Abstract
An IoT system often consists of many sensors to collect data in different aspects. Meanwhile, all these sensors describe the IoT system’s functional status, to which it belongs. The correlations between subsystems are always emphasized for a complex system that contains several IoT subsystems. At the same time, there are still no good ways to calculate these types of correlations since that (1) multiple sensors describe an IoT system as a matrix while the correlation between matrices cannot be calculated by the traditional methods (i.e., vector ways such as Pearson correlation coefficient) and (2) AI methods such as neural networks were introduced to resolve this problem; however, these black-box approaches cannot explain the mathematical mechanisms, and lots of memory or time are consumed. This paper proposed a novel approach named the matrix-oriented correlation computing method (MOCC) to learn the correlations between IoT systems. The critical problem of this proposed method is calculating the correlation between two curved surfaces, which are modeled as matrices, since an IoT system often contains many sensors which characterize different aspects of this system and continuously generate data in time series. By our MOCC method, the correlation or interaction between any two subsystems can be accurately measured, which means that we can predict the state of a system by its most important related system. Missing data value prediction based on our MOCC method is also presented in this paper. We verified the efficiency and effect of our proposed method via a satellite, a typical IoT system consisting of massive sensors, and the experimental result was proved to outperform existing methods.
1. Introduction
A complex system often contains several IoT subsystems. For example, a satellite platform consists of at least energy, attitude, propulsion, thermal control, data transmission, and payload six subsystems; and each subsystem is designed as the integration of smaller subsystems. The correlation between two subsystems is always an essential consideration in the maintenance or analysis of a complex IoT system [1]. By the correlation analysis, we can determine the most relevant factor of an IoT subsystem. For example, we can find that a system’s data anomaly will affect other systems’ functional status if there are high correlations between them [2]; or a system’s data missing can be predicted based on its high relevant systems [3–7]. Generally, this type of correlation can only be calculated by the sensor data of IoT systems since the sensors are designed for monitoring data for all different properties. If a subsystem consists of sensors that have collected data in a time series of length , then the data would be modeled as an matrix, and the correlation between two systems is actually the correlation between two matrices.
There are many existing correlation computing methods, such as Pearson correlation coefficient (PCC) [8] and cosine coefficient (COS) [9], and new correlation measuring methods have been presented in recent years [10–12]. However, most of the existing methods can only measure the correlation between one-dimensional vectors but not matrices. Figure 1 compares the vector and matrix oriented similarity measurement.
(a)
(b)
AI methods such as neural networks were introduced to resolve this problem in recent years [13]; however, these black-box approaches cannot explain the mathematical mechanisms, and lots of memory or time are consumed.
This paper proposed a novel approach named the matrix oriented correlation computing method (MOCC) to learn the correlations between IoT systems, and this method can also be used for any multidimensional observation objects. After the sensors of an IoT system have produced data in a time series of length , we can get an matrix. Since this matrix is constructed along with time, the MOCC method should measure the correlation between two matrices considering the time factor.
The rest of this paper is organized as follows. Section 2 presents our MOCC method. Section 3 describes a critical application—missing data prediction of our MOCC method. Section 4 describes our experiments. Section 5 concludes the paper, and Section 6 shows related work.
2. Proposed MOCC Method
For ease of explanation, we use the following matrix shown in Table 1 to describe the data produced by sensors in a length time series.
This section will illustrate the mathematical steps of our MOCC method to compute the correlation between any two matrices of this type. MOCC is inspired by a significant and novel math concept identified as distance correlation. The concept of distance correlation extends the correlation calculation from one-dimensional space to two-dimensional space, but it does not consider the time factor. This section presents the mathematical principles and advantages of distance correlation and then improves it by bringing the time factor into it, which eventually resulted in our MOCC method.
2.1. Distance Correlation
2.1.1. Mathematical Principles
We use the distance correlation concept to measure correlation considering multiple sensors of an IoT system integrally. Distance correlation—a statistics and probability theory-based concept—was proposed by Szekely et al. to measure the statistical dependence between two random vectors of arbitrary, not necessarily equal dimension [14, 15]. Therefore, distance correlation can measure the correlation between any two matrices more accurately and comprehensively. The distance correlation is defined as follows.
Let and denote two different IoT systems consisting of and sensors, respectively. If these two systems continuously produce data in a time series of length , then, we can get an observed random sample from the joint distribution of random vectors in and in as follows:
For example, if is an IoT system shown in Table 1, then is the -th column of Table 1.
From the definition of and , we can get that two systems that need to measure correlation need not have the same number of sensors but have to be observed in the same time series.
And define:
Similarly,
Before giving the distance correlation between and , their distance variance is defined as follows.
Based on this, the distance correlation between and is defined as follows.
Then, the distance correlation between and is measured as .
Some of the mathematical properties of distance correlation are (1)(2)(3) if and only if and are independent(4) implies that and have the equal dimensionality and , wherein is a vector, is a real number, and is an orthonormal matrix
2.1.2. Advantages of Distance Correlation
First, from the definition of distance correlation, we can get that its most important advantage is that it can measure correlation between multidimensional vectors.
Additionally, distance correlation can illustrate correlation between vectors more accurately than most of existing methods. We use an example shown in Figure 2 to compare the correlation between and got by distance correlation method and the other two important methods—PCC and COS, respectively. Figure 2 presents two vectors, wherein ; and wherein . We get from Figure 2 that the value of allows for a nice estimation of the value of , vice versa. In another words, and are very relevant to each other. However, the correlation between and is 0 measured by PCC and 0.75 by COS. As the comparison, the correlation between and is 1 measured by distance correlation.
2.2. Enhance Distance Correlation by Time Factor
It is reasonable to consider that time can affect the correlation between two systems. It often means that the later the data is obtained, the more significant the impact on the correlation, vice versa. Therefore, this section combines time factor and distance correlation to measure correlations more accurately. The improved distance correlation is identified as MOCC correlation.
If two systems were observed in the same time series of length , and let the 1st observation is the earliest, then, the -th observation should have the largest weight on their correlation, and the -th has the second largest. Therefore, if we bring time factor into distance correlation, equation (14) is revised as wherein is the time factor. The bigger or indicates the later time, therefore, has the bigger weight. Introduce of equation (16) into equation (15), the MOCC correlation will be got.
3. Applications of MOCC Correlation
A significant application of MOCC correlation is missing data prediction, which is also widely studied in the research field of IoT systems. MOCC correlation is a transform of the distance correlation. Although distance correlation is widely adopted by many researchers, the missing value prediction based on distance correlation has rarely been studied before.
An IoT system often consists of massive sensors, and data missing is a common phenomenon for system running. Data missing may be caused by network packet loss, or some sensors’ transient exception. If it occurs, then some data in Table 1 will be missing, the prediction of missing data for an IoT system consists of the following two steps: (1) finding its high correlated systems based on history data and (2) predicting missing value based on its high correlated systems.
3.1. High Correlated System Finding
For ease of presentation, denote the set of all the IoT systems as , if one system has data missing, its high correlated systems have to be found for missing data prediction. Denote is the system that needs missing data prediction, the set of its all high correlated systems can be denoted as: wherein is the MOCC correlation between system and , THRESHOLD is a constant to indicate whether these two systems are high correlated or not. In practice, we let .
3.2. Missing Data Prediction
Before the missing data prediction, we should determine another essential issue: how a system is impacted by its high correlated systems. By experiments, we found two phenomena: (1)If two systems have a high MOCC correlation, then, their MOCC correlation will hardly change with their data amount growing with time(2)On the contrary, if two systems have only a low MOCC correlation, then, their MOCC correlation would change notably with their data amount growing
This experiment will be detailed and presented in Section 4, and this section only use the corollary of this experiment to predict missing data. From the above phenomena, we can get a corollary as follows.
Corollary 1. If is an IoT system with missing data and is high correlated to , then, the prediction of the missing data should keep almost unchanged.
Take the IoT system in Table 1 as an example, is the observed value of on , if this value is missing, denote the prediction of it as .
Based on the observations of and from to , we have got their MOCC correlation denoted as . If the observation on missed the value , then, the prediction should satisfy wherein is got by predicting the missing as .
If we denote as , and its high correlate system as , after the observations on , we rewrite equation (14) as
By predicting the missing as , namely, that we have completed the observations on , if the following satisfied then, equation (20) will be satisfied. Finally, we can get that the value that satisfied equation (22) is the needing prediction of . Although equation (22) seems complicated, it is just a multiple-order equation with only one variable we can solve it quickly with the facility of three-part tool, such as Apache Commons math library.
For the missing value of , we can get different predictions based on its different high correlated system. We can make the prediction more reasonably by combining all the results of all its correlated system in , shown as follows: wherein, is the prediction value made by system , and is the final prediction value of .
4. Experiments
In this section, we perform experiments to validate our MOCC method and compare the results with those from other correlation computing methods. Our experiments are intended to (1) verify the rationality of Corollary 1 that is presented in Section 3.2 and (2) compare efficiency of MOCC method with other correlation computing methods.
4.1. Experiment Setup
This experiment was constructed by employing the data of a typical IoT system—the FY-3D weather satellite, one of the most advanced weather satellites globally. This satellite consists of energy, attitude, propulsion, thermal control, data transmission, cabin, and payload seven subsystems. Each subsystem can also be divided into smaller systems; for example, the attitude control consists of stellar positioning, gyroscope, and flywheel systems. There are more than 10000 sensors deployed on this satellite. All the sensors’ monitoring data are transferred to the ground station periodically fourteen times one day.
It is important to work to analyze the correlation between two subsystems of the satellite since one’s status often impacts another one. Satellite communications are vulnerable to interference. Therefore, data missing or abnormality usually occurs. This motivated the work of this paper.
4.2. Experimental Proof of Corollary 1
In Section 3.2, Corollary 1 says: if is an IoT system with missing data and is high correlated to , then, the prediction of the missing data should keep almost unchanged.
To prove this corollary, we have made statistics of all subsystems of FY-3D weather satellite for their correlations. For any two subsystems, we called it a system pair. This experiment is constructed as the following six steps: (1)Determine a period with no missing data; use this data to do steps 2 to 6(2)Calculate the MOCC correlation of any two systems, and determine their correlation belongs to which range. There are 10 ranges in total, i.e., [0, 0.1)… [0.9, 1](3)For each system pair, randomly remove some data observed simultaneously. It can be seen to delete a column from Table 1 randomly, but the deletion of two tables should be at the same column position. The deletions were finished only 5% columns left. (Since we cannot know the actual values of the future, we deleted some values from the original dataset. Then, the dataset that deleted more values can be seen as “the past,” and the dataset that deleted fewer values can be seen as “the future.” By which, we can simulate the change of two systems’ correlation along with time.)(4)Once a deletion for a system pair finished, we calculated their new MOCC correlation and got the change value comparing their last correlation(5)When all the deletions were finished, the average change of each system pair was got(6)We find that if a system pair originally belonged to a relatively large range in step 2, then, their correlation would rarely change when deleting their data. The experimental result is shown as Figure 3
4.3. Comparisons
We compare the predictive accuracy for missing data of our MOCC method with other two well-known prediction methods. The two compared methods are the follows: (1)Time-Aware Method (TA). This type of methods makes prediction based on the time factor, which was proposed in reference [16](2)Matrix Factorization Based Method (MF). This type of methods makes prediction by factorizing the dataset into matrices, and the reconstruct the dataset by multiplying these matrices. We selected the method proposed in reference [17] to compare with
We use the mean absolute error (MAE) and root mean squared error (RMSE) to measure the prediction accuracy. MAE and RMSE are defined as (22) and (23), respectively: where is a value in the dataset, and is its prediction value.
The predictions were made as follows: (1)Randomly select some values which are not missing to predict, such we can compare the predictive value to the real value(2)The dataset was made to different sparse ratio to test the prediction performance on spare data
There are many subsystems deployed on FY-3D satellite, we choose the battery system to make comparisons, and this system consists of the following sensors: (1) voltage, (2) current, and (3) temperature.
The prediction accuracies of MOCC and the comparisons with other methods are shown in Table 2. With reference to Table 2, we can see that MOCC is more accurate than all of the other methods for the two chosen datasets. As the data increases from 5% to 50%, the MAE and RMSE values become smaller.
5. Conclusion
We have enhanced the concept of distance correlation by bringing the time factor into it, which results in our MOCC method. This method considers all sensors of an IoT system as integration and can measure the correlation between subsystems accurately. We have also presented how to predict missing data values by our MOCC method. The prediction is based on an experimental proofed corollary, i.e., two highly correlated systems will rarely change their correlation with the data amount growing.
We also performed experiments to verify our corollary and our method’s efficiency.
6. Related Work
Correlation between systems indicates their dependency, which is very important in system analysis. Based on correlation computing, we can determine the most relevant factor of the systems subsystem. For example, we can find that a system’s data anomaly will affect other systems’ functional status if there are high correlations between them; or a system’s data missing can be predicted based on its high relevant systems. IoT systems consist of massive sensors producing data in matrix form. Therefore, the correlation must be calculated in multiple-dimensional ways.
There are many existing correlation computing methods, such as Pearson correlation coefficient (PCC) [8] and cosine coefficient (COS) [9], and new correlation measuring methods have been presented in recent years [4]. Many enhancements were brought into PCC and COS in recent years. By adding the weights to determine the different effects of different correlated objects, Zheng et al. [18] proposed an improved PCC correlation computation method and employed this method to predict missing data. Sun et al. [19] proposed a normalized correlation computing method to avoid the disadvantage that traditional PCC or COS neglects the mathematical features of observed vectors. However, all existing methods can only measure the correlation between one-dimensional vectors but not matrices. Figure 1 compares the vector and matrix-oriented similarity measurements.
Prediction of missing data has been widely studied in many fields, especially in the field of QoS prediction for service recommendation [20–22]. Correlation analysis is a crucial way to make a prediction. However, the missing value prediction based on distance correlation has rarely been studied before this paper.
AI methods such as neural networks were introduced to resolve multidimensional correlation analysis in recent years [13]. However, these black-box approaches cannot explain the mathematical mechanisms, and lots of memory or time are consumed.
Data Availability
Data are available on request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported by the National Key Technology Research and Development Program of China (2021YFB3901000 and 2021YFB3901005) and NSFC (61602126).