Abstract
Model confidentiality attacks on convolutional neural networks (CNN) are becoming more and more common. At present, model reverse attack is an important means of model confidentiality attacks, but all of these attacks require strong attack ability, meanwhile, the success rates of these attacks are low. We study the time leakage of CNN running on the SoC (system on-chip) system and propose a reverse method based on side-channel attack. It uses the SDK tool-profiler to collect the time leakage of different networks of various CNNs. According to the linear relationship between time leakage, calculation, and memory usage parameters, we take the profiling attack to establish a mapping library of time and the different networks. After that, the smallest difference between the measured time of unknown models and the theoretical time in the mapping library is considered to be the real parameters of the unknown models. Finally, we can reverse other layers even the entire model. Based on the experiments, the reverse success rate of common convolutional layers is above 78.5%, and the reverse success rates of different CNNs (such as AlexNet, ConvNet, LeNet, etc.) are all above 67.67%. Moreover, the results show that the success rate of our method is 10% higher than the traditional methods on average. In the adversarial sample attack, the success rate reached 97%.
1. Introduction
The application of CNNs has formed a business model of machine-learning as a service (MLaaS). Lecun et al. proposed a confidentiality attack method based on query connection. For a model with n parameters, they need n queries to get n equations, then the parameters of the model can be calculated by solving the equations [1]. Moreover, they can use the reversed model to generate data that is close to the original data to make a great threat to the model, but these attacks require higher abilities from the attackers. In response to these contact attacks, some scholars proposed methods for protecting model confidentiality based on homomorphic encryption and secure multiparty computing [2, 3].
Due to the shortcomings of traditional reverse attacks, methods based on the side-channel have been applied to the model reverse attack. Side-channel attacks are mainly used in the key recovery of different encryption algorithms in cryptography, symmetric encryption algorithms. Neural network algorithms are similar to encryption algorithms in the structure and calculation process. In terms of algorithm structure, symmetric encryption algorithms are connected by round functions, and CNN is composed of multiple network layers, both of them have linear components and nonlinear components, in the operating mode, the input of the next layer or round relies on the output of the previous layer or round [4]. Based on the similarity, side-channel attack has been applied to the reverse model.
At present, there are two major methods for model reverse attacks. One is based on the model API. It constructs a shadow dataset to restore a fake model T' with the same function as the initial model T. Since the two models have the same architecture, T and T′ predicted probability distributions are also similar, the attack on model T′ can also be regarded as an attack on the target model T. However, the construction of the shadow dataset requires prior knowledge of the edge distributions of the different features or the attackers need to know the distribution of the original training dataset in advance. In experiments, the training set of the shadow model can usually be generated by sampling from the edge distribution of each feature independently, which is very difficult in practical attacks. Another way is through traditional contact side-channel attacks, such as cache attacks, power attacks, and electromagnetic attacks [5–7]. For the SoC board, they all require professional measuring instruments to collect leakage information, and the leakage information usually contains a large amount of noise, which requires preprocessing work to clean the data. Therefore some relevant information of the model will be missed.
In this paper, we propose a novel timing attack based on side-channel attacks in the SoC platform for model reversion. We make use of the timing difference between the theoretical and the practical to determine the actual model parameter value based on a mathematical library (mapping relationship) built on the model parameter value and the timing. Our method does not need to know the distribution of the original training data or the edge distribution of different features, but we assume that the attacker knows about the category of the target model and does not know about the details of the model architecture, and they use the profiling tool in the SDK of the SoC board to collect timing leakage. For the same CNN model, the reverse success rate has also improved. The result of adversarial attacks shows that our method has a certain effect on the original model. Table 1 shows the different attacks of the reverse success rate to the ResNet-20.
The structure of the paper is as follows: Section 2 describes the current work on model inversion; Section 3 describes how we acquire time leakage information on SoC boards; Section 4 is the principle and design of the experiment; Section 5 is the experiment and its effect; and finally, we summarize our job and prospect the work of our method.
2. Related Works
The reverse attacks on CNNs are mainly divided into the following forms.
2.1. Method Based on Model APIs
The attack based on model API uses the query-based method [8, 9]. This method needs shadow data as training data, so that it can reverse it by simulating the original model. The shadow data is generated by sending data cyclically. Once we send data to the original model, we can get a response to speculate on the parameters or functions. Therefore it can generate a close or even identical function. In other words, the shadow model and the original model have to equip the same architecture, we use to generate a dataset x, take x into the model and get the results y, and then the take pair (x, y) to train the shadow model . Because the two models have the same architecture, the predicted probability distribution is also similar. The attack on the shadow model can be regarded as a reverse attack on the original model. However, if the shadow model and original model are too different in architecture, the functions of them will be much different. Even though the shadow data and the original data are similar, the shadow model is also quite different from the original model in accuracy.
2.2. Method Based on Side-Channel
FeiYan et al. [10] reversed the structure of CNN based on FPGA by measuring the memory access information when the model is running. In the method, if the model structure based on priori assumptions is too different from the original model, it will cause a great error in the model weight. It will also cause the confidence level of the inference task to be too low at last. Nair et al. [11] transferred the research to a general-purpose GPU platform, measured the kernel events and memory information generated on the GPU and transformed the structure reversal problem into a sequence recognition problem. They use LSTM, which is commonly used in ASR and CTC, to reverse the GPU-based deep neural network (DNN) model structure, but in order to obtain the kernel events, attackers need stronger capabilities. In addition, when attackers use kernel events to restore the weight, they have to collect the side-channel data corresponding to the weight calculation operation. In a neural network, the operation on the weight is multiplication. Therefore, the weight reversal requires the attacker to collect the granularity level of the multiplication instruction. It can hardly be achieved in real attack scenarios. Xu et al. [12] successfully reversed the number of layers and neurons in each layer, the activation function category, and other information of the DNN implemented on the ARM Cortex-M3 platform by measuring the electromagnetic signals. But the memory usage of DNN on ARM Cortex-M3 is not available to the attacker. Therefore, since the attacker cannot complete the mapping of the model weight value to the side-channel leakage value, the weight value of the model cannot be reversed from the side-channel data. Joseph et al. described timing attacks against the common table-driven software implementation of DNN. When DNN loads the table of lookup, a cache miss occurs if the values are not in the cache and the time to reload the data from memory is greater than the time to access the cache directly. Therefore, it will have an impact on the overall execution time difference if the same value is looked up in two operations during a single encryption process [13]. Shinpei Wada et al. focused on the specific physical structures of the printed circuit board (PCB) near the cryptographic module and conducted an electromagnetic analysis where the distribution of electric or magnetic fields containing the secret key information is dominant. They validated that the whole secret key information can be efficiently extracted by conducting the electromagnetic analysis using a single electric or magnetic probe on the location where a specific field is dominant due to the physical structure. By reversing the physical structure of the PCB, including the cryptographic module, and considering the dominant field, the secret key information from the cryptographic module can be efficiently extracted [12].
3. Introduction to Profiling
In this section, we first introduce the application of the SoC platform in the field of artificial intelligence, then we specifically introduce the ZedBoard, which has become a good carrier of neural network models. Next we introduce the profiling file of the board-side auxiliary development tool we used in our paper. Finally, we show how to obtain the timing leakage from the profiling file.
3.1. SoC Platform
The convolution operation occupies more than 85% of the calculation in CNNs, which leads to a slow inference rate, so it requires additional resources to accelerate. Deep CNN has shown extremely high accuracy in computer vision tasks. Sim et al. [14] proposed a Deep CNN acceleration system. They applied a dual-range multiply-accumulator (DRMAC) to perform low-energy convolution operations, whereas they adopted a tiled manner to use data blocks and a compressed convolution kernel for convolution product operations to reduce the bandwidth requirements of the off-chip memory. Moreover, there are hardware accelerators that utilize the inherent error resilience of artificial intelligence algorithms. Xiao et al. [15] proposed a bionic computational model and revealed the inherent fault tolerance of relative cortical networks. The key of their model is to use a fixed scheme to protect the results of function calculations. Artificial intelligence algorithms can essentially resist temporary or permanent errors. Jiuxiang et al. [16] proposed and implemented an artificial intelligence acceleration chip that can endure multiple errors. It can use a variety of algorithms based on CNN to achieve some high-performance tasks. Like other customized chips, they can also improve energy efficiency by 2 orders of magnitude compared to general-purpose chips.
3.2. ZedBoard Chip
The ZedBoard is one of the typical SoC platforms. It is a development board based on the Xilinx ZynqTM-7000 all programmable SoC (AP SoC). It combines the Corex-A9 processing system (PS) and the 85000 series 7 programmable logic (PL) unit. The Zynq chip has both high-performance processing capabilities and a flexible programmable configuration. The ARM part is called PS (processing system), the FPGA part is called PL (programmable logic) and these are interconnected through the on-chip bus AXI. In practical applications, developers can modify the chip through the PL part, instead of replacing and redesigning the chip. The flexibility of the design and the powerful combination of extended functions make it a great carrier for neural network models [17–19].
3.3. Profiling File
The profiling analysis collects data of the application at a fixed interval or by the sampling frequency. Compared with static code analysis, profiling is a dynamic analysis method for studying programs. It is mainly used to locate the part of the program which needs to be optimized to improve the running speed of memory usage [20].
The TCF profiler is a performance analysis tool of the Xilinx SDK. The tool uses the statistical sampling method of the periodic inspection system to sample the system program by JTAG, which is a debugging interface. The sampling rate is 100Hz, and it shows the proportion of the execution time of each function in the entire application. The TCF profiler tool provides the functions shown in Table 2 [21–23].
3.4. Reverse Principles
The basic structure of CNN consists of an input layer, a convolution layer, an activation function, a fully connected layer, and an output layer. The fully connected layer is actually a convolution operation in which the size of the convolution kernel is the upper layer's feature map, and it stretches the feature map into a one-dimensional vector to reduce the dimension. Finally, it outputs the classification results. Therefore, the parameters of the fully connected layer account for the highest memory usage [24]. The memory usage of the fully connected layer can be obtained by the memory access information. The number of neurons in the fully connected layer can be calculated by linear regression. For the activation function, Nair et al. verified that unsaturated nonlinear functions can solve the problem of gradient explosion or gradient disappearance and accelerate the convergence speed compared with saturated nonlinear functions [25]. In most CNNs, the ReLu unsaturated nonlinear function is commonly used as the activation function. Therefore, the following experiments will mainly reverse the parameters of the convolutional layer.
The principles of our experiments are based on the five formulas of the CNN convolutional layer proposed by Le Cun et al. [26], as follows:
The meaning of each notation is shown in Table 3.
The formulas (1) and (5) are the initial constraints of the convolutional layer. Every CNN model needs to obey them. The formulas (2)–(4) are the constraints of model carriers (the correlation constant coefficient is determined by model carriers). For ZedBoard, we can find these three constraints from the Xilinx software development kit (XSDK). According to the above formulas, reverse engineering can be performed as Algorithm 1:
| ||||||||||||||||||||||||||||||||
4. Experiment
The main innovation by us is to use the time difference between the theoretical time and the actual time to determine the specific parameter values based on a mathematical library (mapping relationship) established on the model parameter values and time. The larger the mathematical library established, the more models can be reversed in theory. We designed two experiments. In the simulation experiments, we run a small number of models on the SoC board to find the fixed parameters , which are constant for a fixed board. Then, we derive the theoretical time of each network layer according to Section 3.4. Meanwhile, we find that the padding methods “SAME” and “VALID” are indistinguishable, which means that when we build our mapping library, we only need to choose one pattern in a physical experiment. In the physical experiment, we collect the timing leakage of each network layer of all models by using profiling tools, then we compare the practical time with the theoretical time to determine the specific parameter settings of network layers.
4.1. Experimental Configuration
The experiment is divided into two parts: host end and device end. The host end is mainly for generating the model, training the model, and transforming the model from the host end to the device end. Furthermore, it controls the function of profiling and turnarounds the profiling file. The device end mainly completes the inference and deployment of the model and returns the profiling file to the host end. Tables 4 and 5 are the experimental configurations of the host end and the device end.
4.2. Simulation Experiment
In order to verify the linear relationship between the calculation and the time of the convolutional layer, actually to find out , we select the single-channel image with the input size of 28 28, the size of the convolution kernel are 3, 5, 7, the convolution step length is 1, and the padding mode selects ‘SAME’ and ‘VALID’. For each candidate value of the convolution kernel, the output channels come from 64 to 128, so we have 768 different convolutional layers. As for the output channels, they can be calculated from the convolution kernel and the input channels. Then, we take the output channels as a self-variable. The convolution kernel and the number of input channels are known. We take 65 sets of output channels to make the least square regression to get every of different convolutional layers, and finally, we can acquire a table of .
We plot the data when , and the padding method is fixed to “SAME” in Figure 1. From the figure, it can be seen that the calculation time increases as the calculation increases. Moreover, there are a few singular points because of some accidental errors. In addition, we calculate the correlation coefficient between the calculation and the time when we take different values, as shown in Table 6.
The correlation coefficients are all above 0.83, which indicates that there is a proportional relationship between the calculation and the time.
At the same time, in the simulation experiment, we discover that the padding model has little influence on measured time. In other words, when the padding model is set to ‘SAME’ or ‘VALID’, it can hardly distinguish the correct parameter settings by only relying on the leakage of the timing information under the current accuracy. It shows the error between the collected data when the padding adopts different models in Figure 2.
In order to verify the findings, we select the single-channel image which the input size is 28 28, kernel size is 7, the convolution step is 1, the output channels come from [64, 128] and the padding is “SAME” and “VALID,” so we have 256 sets of data, in order to verify the above findings we take the -test and test on the data.
4.2.1. T-Test
Introduction to T-Test. In the field of side-channel, the T-test is often used as a discriminator to verify whether two datasets share the same distribution. Welch’s T-test considers two-sample sets and assumes H0: the data of the two sets obey the same normal distribution. Given a constant value (confidence level) then judge whether to accept or reject H0 under the confidence level, the methods are as follows [24, 28].
Let and are two datasets. The data size, mean, and variance of the two datasets are (, , ) and (, , ). The test statistics t and the degrees of freedom are calculated as follows:
If the test statistic t satisfies the T distribution, the probability density function of accepting H0 and the probability of accepting H0 are as follows:
is the gamma function and .
T-test is mainly to test whether the mean of the data is the same or not. When the difference in the means between dataset and dataset is large, the probability P of accepting H0 is small. When the difference of the means between dataset and dataset is small, the probability P of accepting H0 is large [29]. Meanwhile, the T-test ignores the influence of different variances on the data distribution, so it has a better detection effect for data with a large mean difference.
The Result of T-Test. The data we collect in the simulation experiment obeys the normal distribution or approximately obeys the normal distribution. We take a two-sample T-test and assume H0 as the data collected by the two padding models has little difference under the current accuracy. Then we output the test results with 90% confidence. The test results are shown in Table 7.where , , and mean that H0 is accepted with 90% confidence, and represent the estimation of the degree of freedom and the sample standard deviation.
4.2.2. -test
Introduction to-Test. The data is expressed as a contingency table of the number of leakages. The row j represents the amount of leakage, the column i represents the number of sample classifications, and represents the times of leaks corresponding to (i, j) and the total number of samples , the test statistics X and the degrees of freedom are calculated as follows [30]:
In the above equation, is the gamma function.
If the statistic f is larger, the probability P-value is smaller, that is, the probability of accepting H0 is smaller, then H0 can be rejected with a high probability.
The Result of-Test. We take a two-sample F-test on the two sets and H0 is assumed to be that the data collected by the two padding models has little difference under the current accuracy. The test results are shown in Table 8.where , , and mean that accepting H0 with 90% confidence. and represent the test degree of freedom 1 and freedom 2.
After the T-test and -test, the two sets can be regarded as coming from the same distribution. We now verify the indistinguishability of the time leakage collected by the two padding models under the current accuracy, which provides a theoretical basis for simplifying the reverse process and reducing the sampling complexity in physical experiments.
4.3. Physical Experiment
We take the above formulas to find the linear relationship between the calculation and the time of different convolutional layers, then reverse the convolutional layer parameters. The reverse steps are essentially iterative processes of solving for the optimal solution of candidate values.
As to the table, we get it in the simulation experiment. Now, we have 4680 models. For every model, other structures remain the same except for the above parameters. According to Table 9, we select the common single-channel input image size, convolution kernel size, input channels, output channels, step length, and padding models to establish a mapping library.
4.3.1. Attack Library
We generate the above models and train them, then send the models to the device end. It is worth noting that in order to reduce the accidental errors, we make every model perform the task of image recognition 100 times, so for each convolutional layer, we can collect 100 groups of data, as to all the models we have 4680 100 groups of data.
According to formula (4), we use collected data to calculate the table by the least square regression and show the residual of , as shown in Figure 3.
It can be seen from the figure that fluctuates within a small range for different input picture sizes. Among them, numbers 1 to 24 are models with an input image size of 7, numbers 25 to 48 are models with an input image size of 14, and the rest are models with input image size of 28. Figure 4 shows the fluctuations under different input sizes.
We select a from candidate values, the feature map size , and the number of input channels of the first input layer are two known values for CNNs. So, we know , then we can get a theoretical time according to Section 3.4. For each assumed , we calculate the difference between theoretical time and the measured time. The minimum difference corresponding to the will be regarded as the real convolution kernel size. Once the convolution kernel size is obtained, all subsequent convolution layer parameters can be calculated.
4.4. Inversion of Common Models
After the attack library is established, we make an attack on some popular CNN models: an 8-layer AlexNet, an 18-layer SqueezeNet, LeNet, and ConvNet. For the reverse success rate, we define it as the following: for every assumed , we can calculate all the parameters by formulas, if these parameters match the real parameters, the inversion is said to be successful. Table 10 summarizes the number of possible structures identified by our algorithm. From the table, the reverse success rate of common CNNs is higher than 66.67%.
4.5. Experimental Performance Analysis
In the simulation experiment, we get the and theoretical time of different models. In physical experiment, we make use of the profiling tool to collect the timing leakage, then we build the mapping library. In order to prove the validity of our mapping library, we make a strict comparison and process between the profiling data: T-test and -test. Next, we mainly describe the effects from the following indicators: (1) the complexity of our inverse algorithm; (2) the distribution consistency of the measured value and the predicted value; (3) inverse success rate of convolutional layer; and (4) the bias of reversing and the success rate of adversarial attack.
For (1), the attacker only needs to get the profiling file to obtain the timing leakage. As to the profiling file, it is a tool for tuning. Any developer has access to it on the SoC board. Malicious attackers only need to buy the same type of device on the market. After obtaining the profiling file, the attacker can make attacks on the device [31].
For (2), the complex operation is mainly concentrated in matching the , so the complexity is . Compared with the traditional model reverse attacks, the differences are shown in Table 11.
4.5.1. Consistency of Measured and Predicted Value Distribution
The measured time refers to the time leakage collected on the device, and the theoretical time refers to the time data calculated from the formulas. The trend of theoretical time and measured time is shown in Figure 5. The red line represents the theoretical time and the blue line represents the measured time. It can be seen from the figure that the red line almost completely covers the blue line, that is, the difference between the theoretical time and the measured time is extremely small.
We take t-test and χ2 test on the theoretical time and the measured time, the results are displayed in Tables 12 and 13.
It can be concluded from the test results that the measured time and the theoretical time can be regarded from the same distribution.
4.5.2. Inverse Success Rate of Convolutional Layer
Table 14 shows the success rates when the input image sizes are 7, 14, 28, 56, and 112. From the table, we can see that for convolutional layers with different input picture sizes, the reverse success rate is above 75%.
4.5.3. The Bias of Reversing and the Success Rate of Adversarial Attack
In order to further evaluate the reverse performance, we take MAE (mean absolute error) and RMSE (root mean squared error) to be the evaluation indicators of the inverse bias of the models. Meanwhile, we make an adversarial sample attack on the reversed model to evaluate the threat to the original model. Specifically, the accuracy of the reverse is measured by calculating the deviation between the theoretical values and the measured values. Moreover, the threat level of the inverse model to the original model is measured by the success rate of the adversarial attack [32].
MAE is the deviation between the theoretical time and the measured time, defined as follows:
RMSE is defined as follows:
In the above equation, and represent measured time and theoretical time. represents the total groups of the theoretical time. MAE and RMSE are sensitive to large errors. So, the smaller the two values are, the better the inverse performance is. Table 15 shows the values of the two indicators.
Furthermore, we use the reversed model to generate a certain number of adversarial samples and make an adversarial attack on the original model. The result shows that the original model outputs the different label with a 97.73% probability compared with the former outputs after adding adversarial samples to the same picture, so the inverse model can produce enough threats to the original model.
5. Conclusions
We study the time side-channel leakage when CNN is running on the SoC system and propose a reverse attack method based on the time side-channel. It uses the SoC system performance analysis tool to collect the time information of each network layer of CNN. According to the linear relationship between the calculation, the parameters, and the memory usage in different network layers, we use a profiling attack to establish a mapping library corresponding to different network layers and time, and finally we can reverse the entire model. The results show that the reverse success rates of common convolutional layers are above 78.5%, and the reverse success rates of common CNNs (such as AlexNet, ConvNet, LeNet, etc.) are all above 67.67%. Moreover, the 97.73% success rate of the adversarial attack shows that our reverse algorithm can make enough threats to the real model [33].
However, the method we proposed still has some problems. We assume that the attacker can obtain the memory access information in the profiling file, but not all SDK tools of the device can provide the purchasers with access to the information, so it will be a strong assumption in a real environment. However, the memory usage can be calculated by batch size and other settings, such as optimizers like SGD, momentum, and Adam. Attackers can take the above settings to calculate the memory usage by formulas to complete the attack [34]. Secondly, if the accuracy of sampling is lower than 0.01 s, the reverse effect will be greatly reduced. Next, we will focus on applying the reverse algorithm to actual application scenarios and evaluating the attack effect. [26].
Data Availability
The dataset and codes are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This research was supported by the National Natural Science Foundation of China under Grant no. 61972295, the Wuhan Science and Technology Project Application Foundation Frontier Special Project no. 2019010701011407.