Abstract
The signals in numerous complex systems of engineering can be regarded as nonlinear parameter trend with noise which is identically distributed random signals or deterministic stationary chaotic signals. The commonly used methods for parameter estimation of nonlinear trend in signals are mainly based on least squares. It can cause inaccurate estimation results when the noise is complex (such as non-Gaussian noise, strong noise, and chaotic noise). This paper proposes a calibration method for this issue in the case of single parameter via nonstationarity measure from the perspective of the stationarity of residual sequence. Some numerical studies are conducted for validation. Results of numerical studies show that the proposed calibration method performs well for various models with different noise strengths and types (including random noise and chaotic noise) and can significantly improve the accuracy of initial estimates obtained by least squares method. This is the first time that the nonstationarity measure is applied to the parameter calibration. All these results will be a guide for future studies of other parameter calibrations.
1. Introduction
The signals in numerous complex systems of engineering can be regarded as nonlinear parameter trend with noise such as the multiexponential model for nuclear magnetic resonance (NMR) signal [1], the optical-sensing membership function model [2], the autoregressive exogenous (ARX) model [3], the separable nonlinear function models [4], and the hydrodynamic model for water level and water flow [5]. The noise can be identically distributed random signals or deterministic stationary chaotic signals [6–10]. The parameter estimation (or fitting) of nonlinear parameter trend with noise has been an important topic for signal analysis in complex systems of engineering. There are some classical methods and their improvements for this topic such as Gauss–Newton method and Levenberg–Marquardt method [11–13]. Most of these methods are based on least squares. But if the nonlinear trend has complex noise such as strong noise, non-Gaussian noise (e.g., power-law and Cauchy noise) and chaotic noise, there are no effective methods for its parameter estimation. It is rather difficult for parameter estimation in this case. For non-Gaussian noise, due to the heavy tails of power-law and Cauchy distribution, the variance of noise does not exist. This causes that the sum of squared residuals (the optimization objective of least squares) is always large and not sensitive to the search of parameter. It can also suffer from the same problem when the noise is strong. It is well known that some chaotic dynamical system exhibit stochastic behaviors, which can be regarded as chaotic noise [14]. For chaotic noise, due to deterministic inherent quantitative relations in noise, the dependence in noise is strong, which can pollute the function signal and may produce spurious trend. These problems eventually lead to inaccurate estimation results obtained by least-squares-based methods.
The noise considered in this paper is stationary, which means that the joint probability distribution of noise does not change over time. Otherwise, it is nonstationary. For signals with stationary noise, an accurate estimate makes its residual sequence more like noise and be stationary. From this perspective, the stationarity of estimated residual sequence can be used for the calibration of inaccurate estimate in the case of complex non-Gaussian noise. To measure the stationarity of whole sequence, Ding et al. [15–17] proposed the nonstationarity measure (NS), which is constructed based on ergodic theory and Shannon entropy, and is sensitive to nonstationary signals. The values of NS are in . The smaller the NS value of sequence, the more stationary the sequence. Lan et al. [18] used nonstationarity measure to select irrigation water consumption model. Tan and Ding [19] analysed the lottery data using nonstationarity measure. Wu [17] analysed the Stock Returns using nonstationarity measure. Tan et al. [20] studied the decomposition of noise and trend via nonstationarity measure. Liu et al. [21] obtained an objective estimate of protein dynamic states with single molecule fluorescence data via nonstationarity measure. Zhao [22] conducted a nonstationarity analysis of high frequency water level data based on nonstationarity measure.
Due to NS’s sensitivity to nonstationary signals, we propose a calibration method to achieve accurate estimation results via NS from the perspective of the stationarity of residual sequence. This paper considers single parameter in order to introduce our calibration method conveniently at the first time. After obtaining the least square estimate, the proposed calibration method increases or decreases the obtained estimate to decrease NS value of residual sequence and stops when the NS value satisfies a given value condition. Various models with different noise strengths and types are simulated and the simulated data are used to validate the proposed calibration method. Results of validation show that the proposed calibration method performs well for all the models and can significantly improve the accuracy of initial estimates obtained by least squares method. It can not only solve the problem of large sum of squared residuals but also deal with strong dependent deterministic noise, which both cause inaccurate estimation results. The nonstationarity measure is first applied to parameter calibration in this study. All these results will be a guide for future studies of other parameter calibrations.
The remaining parts of the paper are organized as follows: the methods are introduced in Section 2. The numerical results are reported and discussed in Sections 3 and 4. Our work is concluded in Section 5.
2. Methodology
2.1. Parameter Estimation for Single Parameter Trend in Signal
Let be the observed signal which can be expressed as follows:where is a nonlinear function of depending on parameter and denotes the trend in signal; is the noise signal. Here, the single parameter is considered in order to introduce our calibration method conveniently at the first time.
It is important to estimate the parameter of signal with single parameter trend. The most classical method for the estimation of is Gauss–Newton method. This method linearizes the nonlinear function via the first order term of its Taylor series to use linear least squares. The algorithm can be described as follows.
Let be the observed signal series. For given an initial value , the can be approximately expressed in the following linear form via Taylor expansion in the neighborhood of ,
The approximate linear form is further used to obtain the input of next iteration via linear least squares, that is to replace in the sum of squared residuals , and minimize the following approximate form with respect to ,
Let and . The minimiser . Thus, the input of next iteration .
Repeat the abovementioned steps until convergence. Let be the obtained value in the step. The limit of is the estimate of . We denote it by .
This estimation method is mainly based on the least squares and linear approximation. When the noise signal is large or non-Gaussian (heavy tails), the sum of squared residuals is always large. The estimate obtained by similar least square methods is inaccurate in these cases. Thus, after obtaining the least square estimate, the parameter calibration is considered from another perspective via the stationarity of estimated residual series.
2.2. Model Selection Based on Residual Sequence
Model selection is the task of selecting a model from a set of candidate models after giving data. In its basic forms, model selection is one of the fundamental tasks of scientific inquiry. Usually, the data points are collected under random noise, and the model we are interested in is as follows:where stands for the relationship between and , and is the random noise. . means independent and identically distributed. The residual sequence for the th observation in the dataset iswhere is the estimate of . If we rewrite the residual sequence as can be seen as an estimation of the ., random noise [23]. If the model fits to the data correctly, the residual sequence would be closed to the random noise. If the residual sequence appears to behave randomly, it suggests that the model fits the data well. On the other hand, if nonrandom structure trend is evident in the residual sequence, it is a clear sign that the model fits the data poorly. In fact, any trend in the residual sequence indicates the deviations between and .
To check if a fit is good or not, the nonstationary measure (NS) can be computed from the perspective of the stationarity to check if the residual sequence behaves randomly. The nonstationarity measure (NS) is first introduced by Ding et al. [15, 16] and constructed based on ergodic theory and Shannon entropy. It is used to measure the stationarity of sequence and is sensitive to nonstationary signals. The values of NS are in . The smaller the NS value of sequence, the more stationary the sequence will be.
Thus, NS can be regarded as an index to illustrate the deviation of the estimated model and true model . If NS is not small, the deviation between and cannot be neglected; the estimation should be improved. In the single parameter estimation model, the model is characterized by the parameter ; to select good model is equivalent to find the best parameter. The deviation between the estimate and true parameter will be inflected by the NS value of the residual sequence . If is larger than a prescribed threshold , the is not closed to because such a deviation of and induces an appeared trend in the noise . The estimated parameter must be calibrated to reduce the effect of the trend till .
2.3. Parameter Calibration via Nonstationarity Measure
Based on the perspective of model selection via residual sequence, we introduce a calibration method for the inaccurate least square estimate under complex noise via nonstationarity measure.
Due to its sensitivity to nonstationary signals, the NS can be used for the calibration of least squares estimate via measuring the nonstationarity of estimated residual sequence. Based on previous studies [15, 16, 20], the values of 0.05 can be used as the threshold to determine whether there is any trend in the estimated residual sequence. If the NS value of estimated residual sequence is less than 0.05, it indicates there is no trend in the residual sequence and the corresponding estimate of is accuracy.
Let be the estimated residual sequence with estimate . denotes the NS value of . The calibration procedure is described as follows:(i)Obtain the estimate of noise-bearing model via Gauss–Newton method as the initial value for parameter calibration. Redenote it by .(ii)Compute the NS value (greater than 0.05 in general).(iii)Increase or decrease the value of by a fixed small size (set 0.01) along the overall direction of decreasing NS value.(iv)Let be the obtained value in the step. Repeat the previous step until the following case happens: and .(v)The obtained corrected estimate .
3. Numerical Analysis with Random Noise
This section focuses on the numerical study of the proposed calibration procedure in Section 2.3 in the case of random noise. The signals for numerical study are generated according to equation (1). If not specified in this section, the noise added to signals is independent and identically distributed standard Cauchy variables, whose probability density function iswith and .
The signal-to-noise ratio (SNR) is used to measure the signal and noise strength in this paper and is defined as follows:
The relative error (RE) is defined as follows for measuring the accuracy of estimation and calibration:
3.1. Performance of Calibration
The following three models of are adopted to validate the performance of proposed calibration:
Model 1 is not smooth, it admits a fold point at . Model 2 has a log singularity. Model 3 has a complex trend. They can represent typical situations in applications. For Model 1 and Model 2, the signals are generated at every 0.1 from −3 to 3. For Model 3, the signals are generated at every 0.1 from −10 to 10. For further analysing the effects of parameter values on calibration, we choose four values of in Model 3 for validation. One generated signal is shown in Figure 1. Results of calibration for the three models are shown in Table 1. All the results are obtained via the average of 200 generated signals.
(a)
(b)
(c)
(d)
(e)
(f)
From Table 1, the value of NS is bigger than 0.05, and the value of should be adjusted close to the true value. It can also be seen that the proposed calibration via NS improves the accuracy of estimate significantly for all three models in Table 1. For the effects of parameter values, it is found that the initial estimated values deviate from the true parameters with the increase of parameters in Model 3. Figure 2 shows one of the calibration procedures for Model 3. It can be seen in Figure 2 that the estimates of different parameter can be calibrated to the exact value. When the calibration stops, . SNR is negative when the parameter is small, but the parameters can be still calibrated to the exact value via NS.
The abovementioned analysis validates the proposed NS-based calibration method. Next, we fix Model 3 with for analysing the effects of various factors on the proposed calibration method.
3.2. Effects of Noise Strength
In this subsection, the added Cauchy noise is set to , , and to obtain different noise strengths. Model 3 with is chosen for validation. Figure 3 shows one generated signal with each noise strength. Results of calibration are shown in Table 2. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 4.
(a)
(b)
(c)
From Table 2, it can be seen that the smaller the SNR, the larger the RE of will be. This indicates that the strong noise can reduce the accuracy of estimation. The calibration method performs well for all noise strengths. Figure 4 indicates that the estimates for different noise strengths can be calibrated to the exact value. When the calibration stops, . Figure 4 also shows a fast convergence rate which confirms the successful calibration procedure.
3.3. Effects of Noise Type
In this subsection, three types of noise are added to signals for analysing effects of noise type. The selected noises are independent and identically distributed and have uniform, Cauchy and power-law distributions. The Cauchy noise is set to . The uniform noise is distributed as . The power-law noise is distributed as the following density:where is a positive constant and valued such that is a density. Model 3 with is chosen for validation.
Figure 5 shows one generated signal with each noise type. Results of calibration are shown in Table 3. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 6.
(a)
(b)
(c)
From Table 3, it can be seen that the RE of initial estimate is large in the case of Cauchy noise. Comparing with other noise, it suffers from a more inaccurate estimate via least squares method. After calibration using NS, the estimates for Cauchy and power-law noise are corrected almost equal to the true parameter. The accuracy in the case of uniform noise is also improved by more than 30%. It is shown in Figure 6 that the calibration for power-law noise can distinguish the accuracy of a smaller step. Figure 6 also reveals a consistent conclusion that the NS for uniform noise decreases slowly in calibration procedure.
3.4. Effects of Sample Size
We fix the range of from −10 to 10 and change the sampling interval to obtain different sample sizes. The sample size means the number of points were sampled uniformly in the range of . Model 3 with is chosen for validation. One generated signal with each sample size is shown in Figure 7. Results of calibration are shown in Table 4. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 8.
(a)
(b)
(c)
(d)
(e)
(f)
From Table 4, it is found that the more the sample points, the larger the NS values. It indicates that when the sample points are sufficient, the NS is more sensitive to the estimated value. This method can distinguish the estimate from the real parameter and then correct the parameter more accurately. This conclusion is consistent with results of Figure 8, which shows changes in estimate of parameter during calibration.
3.5. Effects of Model Complexity
In this subsection, a more complex model extended from Model 3 is considered for effects of model complexity. It is denoted by Model 4 as follows:where is set to 0.2. The signals are generated at every 0.1 from −10 to 10. Different types of noise are used for this analysis. The settings of noise are the same as in Section 3.3. One generated signal for this model is shown in Figure 9. Results of calibration are shown in Table 5. All the results are obtained via the average of 200 generated signals. One of the calibration procedures is shown in Figure 10.
(a)
(b)
(c)
From Tables 5 and 3, there is no significant difference between the results of Model 3 and Model 4. Comparing with results of Model 3, the accuracy of initial parameter estimation is improved slightly. But the accuracy of calibration is slightly inferior to the simple model. Figure 10 shows changes in estimate of parameter during calibration. It can be seen in Figure 10 that parameters can be calibrated to an accurate value under various noises, which indicates the successful calibration procedures for complex model with multitype noise.
4. Numerical Analysis with Chaotic Noise
First, we introduce the chaotic noise. Let be a transformation. The iterations of are a discrete dynamical system. After giving an initial value , put
The obtained is an orbit of with initial value and length . Suppose is chaotic and admits an absolutely continuous invariant probability measure (distribution) whose density function is , i.e.,
The distribution is stationary because it is invariant. According to the classical Birkhoff individual ergodic theorem, for almost all initial value , the histogram of v will approach to as . So, almost all long orbits of exhibit stochastic behavior. Typically, looks like a sample of the distribution ; it can be regarded as a chaotic noise.
It is known that the chaotic noise has deterministic inherent quantitative relations which result in strong dependence in noise and can disturb the function signal. Thus, parameter calibration via NS under chaotic noise is challenging and must be able to deal with such strong dependence in noise. In this section, we focus on the numerical analysis of the proposed NS-based calibration method in the case of chaotic noise.
Model 2 with independent standard Gaussian noise is chosen for this numerical study. Three values of parameter are selected. The signals are generated at every 0.01 from 2 to 8. Two different chaotic noises are further added to this setting for numerical analysis with chaotic noise. The analysed model is expressed as follows:where denotes the noise signal, denotes the independent standard Gaussian noise, and denotes the added chaotic noise which is generated respectively via the following two kinds of iteration. The initial input of iteration is randomly selected from the uniform distribution .
The transformation of the first dynamical system is . In 1947, Ulam and von Neumann [24] showed that admits an absolutely continuous invariant probability measure whose density is
So, is chaotic discrete dynamical system and its typical orbit exhibits stochastic behavior. The transformation of the second dynamical system is
It is a piecewise linear Lorenz map with constant slope [25]. admits an absolutely continuous invariant probability measure . Furthermore, the mean value of is because of the equality . The mean value of the first invariant probability measure is also because of the equality . In both cases, let ; is zero mean chaotic noise for almost all initial values .
Due to disturbance from strong dependence of chaotic noise, the estimator via least-squares-based method in this case is rather inaccurate and very far from the truth parameter. In order to focus on the validation of proposed calibration method and speed up calibration, the initial value is directly given near the truth for calibration instead of least-squares-based method. Results of analysis and calibration are shown in Figure 11 and Table 6. All the results are obtained via the average of 200 generated signals.
(a)
(b)
Figure 11 shows the change of NS with given parameter near the truth parameter . It can be seen that although the NS for changes slightly at locations that are larger than the truth, all the minimums of NS values occur at location that is very close to the true . This validates the availability of our proposed calibration method under chaotic noise. Table 6 shows the results of calibration with the given initial value. It can be seen that the calibration makes the given initial value close to the truth and improves the accuracy of given initial value, which indicates that the proposed calibration method is also effective under chaotic noise as the case of random noise.
5. Conclusions
It is important to estimate the parameter of single-parameter nonlinear trend in signals. The commonly used methods for this topic are mainly based on least squares. It can cause inaccurate estimation results when suffering from large noise and complex non-Gaussian noise. Thus, we proposed a calibration method for this issue via nonstationarity measure (NS) from the perspective of the stationarity of residual sequence. Some numerical studies were conducted for validation. Results of numerical studies showed that the proposed calibration method performed well for various models with different noise strengths and types could significantly improve the accuracy of initial estimates obtained by least squares method. It can not only solve the problem of large sum of squared residuals but also deal with strong dependent deterministic noise, which cause inaccurate estimation results. For some detailed conclusions, the performance under uniform noise is worse than those under Cauchy and power-law noise. When the sample points are sufficient, the proposed calibration method is sensitive to the estimated value and becomes more accurate. The strength of model complexity can improve the accuracy of calibration slightly. The proposed calibration method is also effective under chaotic noise as the case of random noise despite the rather inaccuracy of least-squares-based method. The nonstationarity measure is first applied to parameter calibration in this study. All these results will be a guide for future studies of other parameter calibrations such as the bias correction of Hurst parameter estimations [26–28].
Data Availability
No data were used to support this study.
Conflicts of Interest
The authors declare that there are no conflicts of interests regarding the publication of this paper.
Acknowledgments
This work was supported by the Fundamental Research Funds for the Central Universities (JBK1806002).