Abstract
Overcomplete dictionaries occupy an essential position in the sparse representation of signals. The dictionary construction method typically represented by singular value decomposition (KSVD) is widely used because of its concise and efficient features. In this paper, based on the background of transient signal detection, an adaptive sparse estimation KSVD (ASE-KSVD) dictionary training method is proposed to solve the redundant iteration problem caused by fixed sparsity in existing KSVD dictionary construction. The algorithm features an adaptive sparsity estimation strategy in the sparse coding stage, which adjusts the number of iterations required for dictionary training based on the pre-estimation of the sample signal characteristics. The aim is to decrease the number of solutions of the underdetermined system of equations, reduce the overfitting error under the finite sparsity condition, and improve overall training efficiency. We compare four similar algorithms under the speech signal and actual shock wave sensor network data conditions, respectively. The results show that the proposed algorithm has obvious performance advantages and can be applied to real-life scenarios.
1. Introduction
For the transient environmental field signal detection (such as explosion shock wave field testing, vibration detection, and acoustic field analysis), the deployment of distributed wireless sensor networks is the mainstream method [1, 2]. Transient signals have sizeable sudden energy changes in the time domain, short rise time, and comprehensive frequency coverage, often accompanying the release of destructive energy and harmonics, noise, and other interference [3]. Wireless data acquisition systems require complete data sampling before feature extraction and identification. Limited by the system bandwidth and energy consumption, it can no longer meet the high sampling and high concurrent measurement requirements by optimizing the network structure. Therefore, using the data itself characteristics and propagation laws to achieve source compression or downsampling is a current research hotspot [4, 5]. Applying compressed sensing to wireless sensing networks is one of the methods to achieve source downsampling. When we know the sparse characteristics of the target signal, the acquisition node only needs to transmit a small amount of downsampled measurements to accurately reconstruct the original signal at the receiver [6]. CS includes three components: sparse representation, encoding measuring, and sparse reconstruction. Sparse representation is an essential part of that [7, 8], which has been widely used in signal processing and is the leading research background of this paper.
The common sparse representations used in CS are fixed and dynamic learning sparse basis [9]. The fixed sparse basis such as discrete cosine transform (DCT) and discrete wavelet transform (DWT) is beneficial for extracting the signal time-frequency domain features. However, limited by the broad spectral range of transient signals, the reconstruction error is significant with limited sparsity, and the number of measurements required for reconstruction increases dramatically with it [10]. On the other hand, dynamic learning sparse basis is data-driven dictionary constructed based on target features, which is theoretically more conducive to the sparse representation of signals. The representative ones are MOD and KSVD and their series of derivative algorithms.
In recent years, various types of learning dictionaries combined with CS have been studied in many fields such as image classification and video denoising with mature technology and excellent performance [11–15]. However, relatively little attention has been paid to 1D signals, and most of them are focused on signal processing. [16] demonstrated that KSVD could effectively reduce the storage space for the input signal in a speech synthesis system and perform this feature classification at the same time. An underdetermined blind source separation algorithm based on an improved double sparse dictionary is proposed in [17] for reducing the computational effort of blind source separation. [18] proposed a data compression algorithm with a segmented training dictionary of signal features based on the matching degree of atomic features, which significantly improves the data compression rate.
We found KSVD has the following two main problems in 1D signal processing through the summary of related work. Firstly, it is difficult to build a universal data set because of the single characteristics of 1D signals and the apparent difference between various types. Secondly, the length of the 1D signal samples is usually long, leading to a large number of solution dimensions [10]. We often need to preprocess the target signal in segments, which further reduces intra-sample correlation and increases the difficulty of dictionary training. In most cases, it needs to be coupled with a priori sparsity range. And it takes longer for multiple iterations to obtain the results [19, 20].
Therefore, the adaptive improvement for KSVD and target signal has become a hot research topic. Most of the current studies revolve around the sparse coding part of the dictionary by improving the solution process of the underdetermined system of equations to enhance training efficiency and reduce sparsity [21, 22]. Introducing an adaptive estimation strategy for sample sparsity in the encoding process can effectively improve dictionary training efficiency. The literature [21]proposed a KSVD dictionary training method based on sparsity adaptive matching tracking (SAMP-KSVD), and the literature [23] proposed a KSVD introducing segmented orthogonal matching tracking strategy (StOMP-KSVD). Both methods demonstrate the feasibility of this improved strategy. Since the CS reconstruction phase has the same mathematical model as KSVD in the sparse coding phase, many reconstruction algorithms based on adaptive sparse strategies [22, 24] are also derived, which can all be applied to the improvement of KSVD dictionary training methods. We summarize the advantages and disadvantages of related works and propose an ASE-KSVD algorithm that combines the estimation method of sparsity in greedy iteration [25–27]. The aim is to improve the dictionary training efficiency and reduce the sparsity error while maintaining a lower sparsity.
The characteristic of the ASE-KSVD algorithm introduces an adaptive sparsity estimation strategy in the sparse encoding phase, which performs sparsity estimation on the target sample columns and adjusts the number of iterations required for dictionary training to eliminate the redundant iterations during stepwise computation. In this work, we firstly use the Hankel matrix to translate and segment the target signal [28] to solve the problem of high dimensionality for the 1D signal construction dictionary. Secondly, the improved ASE-KSVD is applied to the CS framework and compared with the same type of dictionary construction algorithms.
The remainder of this paper is organized as follows: in Section 2, the basic construction principle of KSVD and its application model in CS are described to induce the theoretical framework. Section 3 mainly describes the improved ASE-KSVD dictionary construction algorithm based on adaptive sparsity estimation proposed in this paper and analyzes its improvement strategy and characteristics. In Section 4, we verify the feasibility of the algorithm of this paper in the CS framework by using a comparison experiment combining numerical simulations and measured shock wave signals. Finally, Section 5 concludes the paper with an outlook for future work.
2. Model Descriptions
2.1. KSVD Sparse Model
The KSVD algorithm is a widely used dictionary learning method. The main idea of that is to solve the optimal linear representation of the original sample in dictionary matrix . In this paper, we define . As , where is a sparse vector, the mathematical model is the optimization problem solved in
The KSVD algorithm includes two stages: sparse coding and dictionary update stage. Firstly, the dictionary is initialized according to the training sample set or a fixed sparse basis (such as DCT and DWT), and the sparse coding algorithm such as orthogonal matching pursuit (OMP) is used to obtain the sparse coefficients under the current dictionary. Secondly, in the dictionary update phase, using the KSVD algorithm updates the atoms column-by-column.
As an example for the -th column, define as the -th column vector of the dictionary and as the -th row vector of the sparse matrix , then equation (1) can be extended as
In equation (2), is the residual of this iteration, and the optimization problem can be described as
Use the singular value decomposition for the residuals . We will get , the first column in is the newly generated , and the first row of the is used as the new . Update the atoms , complete a single iteration, and repeat the steps to finally obtain the optimal representation dictionary . In this work, the dictionary is constructed to pursue a better linear representation of the signal, thus obtaining a lower sparsity to serve the subsequent CS downsampling and reconstruction process.
2.2. CS Theory Model
In the CS framework, the -point 1D signal is mapped to the -dimensional vector space defined by as target signal, which is downsampled in denotes the measurement vector, is the downsampled measurement matrix, and is the sensing matrix. Define as the compressed measurement ratio (CR).
To ensure the unique reconstruction of the original signal from the downsampled data, the original signal needs to have sparsity in the sparse basis with . This condition has been guaranteed in the construction of KSVD. In addition, the sensing matrix should also satisfy the restricted isometry property (RIP) shown in
From [29, 30], it is known that the exact cannot be calculated in most cases, the degree of correlation between and is determined by the correlation coefficient of equation (6), and the value of is negatively correlated with the probability of satisfying the RIP constraints.
Finally, under the RIP constraint, the problem of the underdetermined equations is transformed into the minimum norm optimization problem shown in equation (7), which can be uniquely solved using greedy algorithms and convex optimization.
In the ideal reconstruction process, when the number of measurements , the target sparse signal can be reconstructed without distortion [31]. The value of is positively correlated with the value of . Therefore, the focus of this paper is on the accurate and fast optimal sparse representation of the target signal using KSVD. The lower the sparsity , the lower the number of measurements required for reconstruction, and the smaller the system bandwidth occupied by the final wireless network transmission accordingly.
3. Improved KSVD Algorithm Based on Adaptive Sparsity Estimation
This section describes the steps and characterization of the ASE-KSVD algorithm proposed in this paper. The main design idea is to introduce an adaptive sparsity estimation strategy in KSVD sparse coding phase. The estimated atomic set is obtained by correlation matching test on the premise that the dictionary matrix satisfies the RIP constraint. The number of atoms in this set is the estimated sparsity for this samples, . The algorithm removes manually entered sparse parameters and avoids redundant iterations caused by excessive sparsity of different sample features. For the problem of long signals, we use a variant form of the Hankel matrix to segment and stitch the target signal to reduce the computational effort. At last, we add a dictionary filtering process to avoid residual rebound during the iterative process, which is used to ensure the final output is the optimal dictionary with minimum residuals.
3.1. Adaptive Sparsity Estimation Strategy
The main idea of the adaptive sparsity estimation strategy is to make a matching test between the sample and the current dictionary. Define and take the index of the first maxima in as the set , , and use correlation projection matching to determine whether the current is less than the true sparsity of this sample. If the condition in Lemma 1 is satisfied, the current is used as the sparsity estimate for the column; otherwise, the value of is incremented and recalculated.
Lemma 1. If satisfies the RIP constraint with and there exists , then .
Proof. Take the first maximal indexes in to obtain the set . When , then , and there exists .According to the definition of RIP, the singular value of is between and . If we denote the eigenvalue of by , we have , so we can obtainCombining with the definition of the RIP property, it can be known that . Substituting into equation (9), we can obtain . According to the contrapositive proposition of the above conclusion, when , then . From this, the initial estimation method for in the sample is obtained, with initial . If , then the value of is incremented until the inequality does not hold. This sparsity estimation strategy effectiveness is verified in the following experiments, and the results are similar to the theoretical expectations. By reasonably setting value, the sparsity of unknown samples can be effectively estimated.
3.2. Algorithm Description
The ASE-KSVD algorithm is divided into four stages: parameter initialization, sample sparsity estimation, sparse coding, and dictionary update. Before the algorithm starts, input variables include sample set , initial dictionary , sparse estimation parameter , and maximum number of iterations , and initialize each iteration parameter.
Take the -th iteration as an example. At first, calculate the projection correlation of on column-by-column and select the top most relevant columns to form the index set . The matching test is performed by calculating the inequality of equation (10) with stepwise increment to obtain the sparsity estimate of on .
When the sparsity estimate of the current column sample is obtained, it is input into as a parameter to calculate the sparse coding estimate of that column sample on the current . This cycle will continue until all column sparse coding estimates are completed.
Then, in the dictionary update stage, the same column-by-column update as KSVD is used. Take the -th column in as an example, , remove this update column in the sample set, and calculate the residual matrix in equation (11), where denotes the -th row vector of .
Because only nonzero values in the dictionary update have impact on the process, so just take the nonzero value position index of , add it to the set , add its corresponding row vector to the set , and then update the corresponding column of nonzero values in to .
Next, the singular value decomposition of , , take the first column in as the -th column in the dictionary , and is the updated , updated to in accordance with the corresponding position, so as to complete the current front dictionary update, cycle until all columns of the dictionary updated.
Finally, calculate the dictionary residuals , and determine whether the current is the minimum residual; when the number of iterations meets the maximum number of iterations , end the iteration. The optimal value in the overall iterations is taken as the final output dictionary.
The algorithm steps are summarized as shown in Algorithm 1.
|
4. Simulation Results and Analysis
We conducted simulation experiments on the proposed algorithm to verify the algorithm’s performance. In Subsection 4.1, to ensure the generality of data, the Gaussian sparse signal is used as simulation input to test the sparse adaptive strategy of the proposed algorithm and determine the optimal parameters. Sections 4.2 and 4.3 perform algorithm performance tests under two data conditions: shock wave sensor network and speech signal. We set up classification-KSVD (CC-KSVD) [18], progressive decrease of residual-KSVD (PDR-KSVD) [32], SAMP-KSVD [21], StOMP-KSVD [23], and DCT as sparse basis generation algorithms for comparison. The experimental analysis is carried out in terms of the sparsity characteristics of the dictionary constructed by the improved algorithm and the CS reconstruction characteristics, respectively.
The CC-KSVD algorithm is based on the KSVD algorithm to classify the sample training set according to the signal characteristics, train the learning dictionary that fits the class of samples separately, and cascade the various dictionaries into a redundant dictionary with all signal characteristics. This method reduces the dictionary size, shortens the training time of the dictionary, is suitable for signals with noticeable feature differences, and requires a known sparsity, and the reconstruction quality is closely related to the target signal classification method.
The PDR-KSVD algorithm adds a judgment condition to each class of samples when training the dictionary and bounds the number of training sessions according to the rate of change of the residuals during the dictionary update. It is solving the problem of unknown target sparsity.
SAMP-KSVD and StOMP-KSVD algorithms are derived and improved from the CS reconstruction algorithm. Both algorithms are improvements to the method for solving underdetermined systems of equations during KSVD training and neither require a known sparsity. The SAMP-KSVD algorithm uses a stepwise sparsity matching strategy, and StOMP-KSVD works with a threshold parameter to achieve a pre-estimation of sparsity.
All experimental simulation parts of this paper were done on MATLAB R2016b software based on Windows 10 system (Intel(R) Core(TM) i5-5200U CPU @ 2.20 GHz, 4.0 GB).
4.1. Analysis of Sparsity Adaptive Strategy
In the performance analysis of the sparsity estimation strategy, we use Gaussian sparse signals to ensure the generality of the parameters across signals of the same length. Theoretically, any target signal can be superimposed by combining Gaussian sparse signals with equal length.
The experiments compare the accuracy and stability for target sparsity estimation under different values of , respectively. Firstly, considering the case where the signal segmentation length is 128 points, set a Gaussian sparse signal with a same length in this part of the experiment. Set sparsity incremented from 1 to 25 in step 1 and observe the accuracy of sparsity estimation. Secondly, fix the sparsity , and the data are generated randomly in a group of 20 times to test the stability of the sparsity estimation. Each data set was averaged after 500 Monte Carlo experiments.
The results in Figure 1(a) show that as the sparsity of the target signal increases, the estimates tend to grow linearly and then tend to be steady state. The prerequisite of the weak matching strategy in sparsity estimation satisfies the RIP condition [26, 33]. In most cases, we require . Therefore, with a fixed , the smaller the , the better the accuracy and linearity of the estimation effect. When the matching condition reaches the estimated upper limit, we can adjust the segment length to solve it. The two parameters showed a positive correlation. Combining the results in Figures 1(a) and 1(b), when the input parameter is taken as , the adaptively estimated sparsity is closest to the signal’s sparsity.
(a)
(b)
However, when solving the underdetermined equation, the size of the value will affect the screening range for the support set in the subsequent iterations. If the estimated value is smaller than the actual value, the underestimation may make the error larger. If the K is larger than the actual value, an irrelevant smaller value may be calculated for an originally zero point. The error cost caused by the smaller value is much less than the missed estimate. When , the sparsity estimate is slightly larger than the actual value and has the best linearity in the estimated range.
4.2. Dictionary Performance Analysis under Shockwave Sensor Network Data
In this part of the simulation, we analyze the characteristics of the proposed ASE-KSVD algorithm in the CS framework in terms of the algorithm’s efficiency, the sparsity effect of the test set signals, and the measurement matrix matching.
4.2.1. Experimental Conditions Setting
This experiment uses the Endevco 85XX series 15psi range sensor shock wave field test data set to evaluate the performance of the improved algorithm dictionary. We intercept 4096 sampling points of the target signal at a sampling rate of , totaling 120 for analysis, of which 100 are used as the sample set and 20 as the test set.
Equation (15) shows that an improved Hankel matrix [20] is used to preprocess the signal by segmentation, dividing the into . The total size of the sample set is , which meets the requirement that the number of training samples in the redundant dictionary is greater than the length of the signal. Furthermore, the signal is reshaped to its original length after the segment is sparse, according to
In signal sparsity verification, the normalized mean absolute error () [34] is shown in equation (17), and sparsity is used to evaluate sparse performance.
4.2.2. Algorithm Efficiency Analysis
In this part of the experiment, it is verified that the sparsity adaptive estimation strategy improves the efficiency of the dictionary sparse coding stage. The KSVD fixed sparsity is set in the experiments, both dictionary coding strategies are OMP algorithms, the training sample set is the same, the maximum number of iterations , and a total of 200 Monte Carlo experiments are conducted to take the average value. In terms of the dictionary training speed, the sample set is incremented from to in steps of 100 columns to compare the dictionary training time of the KSVD algorithm and the ASE-KSVD algorithm.
The average number of iterations for each column in the overall iterative process is shown in Figure 2(a). Sparsity estimation is performed before iteration to determine the approximate sparsity of each column of samples and match the differences between sample sets. Compared to the fixed sparsity of KSVD, ASE-KSVD can effectively reduce the overall number of iterations without affecting the sparse coding process. Combined with the analysis of the mean calculation for the training set size in Figure 2(b), it also indirectly proves that eliminating redundant iterations can significantly improve the dictionary training efficiency.
(a)
(b)
4.2.3. Analysis of Measurement Matrix Matching Degree
In this section, the experiments focus on applying the ASE-KSVD construction of sparse dictionaries in the CS framework. Furthermore, use the correlation coefficient to measure the matching degree of the measurement matrix and experimental screening of several types of measurement matrices that are currently more commonly used to determine the optimal combination of the current parameter characteristics. The dictionary constructed by the ASE-KSVD algorithm is used as the fixed sparse matrix for this experiment. The reconstruction error and the intercorrelation of the sensing matrix are taken into account. The comparative downsampling measurement matrices are Gaussian random matrix, Bernoulli matrix, sparse random matrix, Toeplitz matrix, partial Hadamard matrix, and circular matrix.
The number of measurements is increased from 320 to 1920 with a step value of 160. The OMP algorithm is used as the reconstruction algorithm to compare the reconstruction effect of different measurement matrices on the same signal. Calculate the correlation coefficient between ASE-KSVD and each measurement matrix, and take the average value after 200 Monte Carlo experiments for each set of data. As shown in Figure 3(a), each random matrix with an ASE-KSVD learning dictionary can reconstruct the original signal if the number of measurements is appropriate. The correlation coefficient between the sparse basis and the measurement matrix can reflect the final reconstruction performance. After experimental comparison in Figure 3(b), the Gaussian random matrix has the lowest correlation coefficient and works best in the experimental reconstruction comparison.
(a)
(b)
4.2.4. Sparse Performance Analysis of Shock Wave Signals
In this section of experiments, the sparse performance of the ASE-KSVD algorithm training dictionary is verified. Take 20 sets of shock wave signals in the test set as the target signal, perform the sparse transformation on them, and define as an acceptable error. After the signals are sparse, they are spliced and restored to the original length. After they are arranged in descending order, the inverse transform error of each sparse basis at a different sparsity is intercepted and compared with DCT, KSVD, PDR-KSVD, CC-KSVD, PDR-KSVD, SAMP-KSVD, and StOMP-KSVD algorithm. Each group of signals performs 100 repeated experiments and finally calculates the average value.
The results in Table 1 also show that in the test set signals, ASE-KSVD reduces the average sparsity by 42.64%, 23.91%, 8.58%, 15.26%, 4.74%, and 26.89% compared to DCT, KSVD, CC-KSVD, PDR-KSVD, SAMP-KSVD, and StOMP-KSVD, respectively, and has the best effect among the compared algorithms.
The results in Figure 4 show that the sparse error share of single-point sparsity decreases as the sparsity value increases. Compared with the comparison algorithm and the fixed sparse basis, the sparse coefficient energy in ASE-KSVD is more concentrated. Under the same sparse normalization error, it can maintain a lower sparsity, which is more conducive to retaining the main information in the signal.
4.2.5. Performance Analysis of CS Framework Reconstruction
Under the same test conditions, the DCT, KSVD, PDR-KSVD, CC-KSVD, SAMP-KSVD, and StOMP-KSVD algorithms were used to generate different sparse bases for comparison and validation. The measurement matrix is set as Gaussian random matrix, and the number of measurements is increased from 320 to 1920 with a step value of 160, and the downsampling data are reconstructed separately using the OMP algorithm.
As shown in Figure 5, with the increase of the number of measurements, the reconstruction error of each algorithm shows a downward trend, and the time required for reconstruction shows an upward trend. Among them, ASE-KSVD works best for the sparse representation of the signal with the lowest sparsity. Under the same error conditions, the reconstruction time and the number of measurements are also the lowest. Compared to KSVD, CC-KSVD, PDR-KSVD, SAMP-KSVD, and StOMP-KSVD, the number of measurements was reduced by 48.18%, 15.56%, 20.83%, 16.79%, and 25.97% for the same degree of error (), respectively. Figure 6 shows the reconstructed results of four sets of signals randomly selected from the test set.
(a)
(b)
4.3. Dictionary Performance Analysis under Speech Signal Data
The same dictionary parameter settings are used in this section to develop experiments on the performance of the ASE-KSVD algorithm under speech signal data conditions. The purpose is to verify the algorithm generality further.
4.3.1. Experimental Condition Setting
All data are from Valentini-Botinhao et al.’s open speech signal data set [35] and sampled at 48 kHz. We randomly selected 120 signals from a clean speech library consisting of 28 speakers and intercepted 4096 points containing valid information as the sample set signals. The data segmentation, test set assignment methods, and parameter settings are consistent with the previous section.
4.3.2. Sparse Performance Analysis of Speech Signals
The results in Table 2 also show that in the speech test signals, ASE-KSVD reduces the average sparsity by 31.02%, 15.65%, 12.02%, 13.2%, 6.19%, and 20.91% compared to DCT, KSVD, CC-KSVD, PDR-KSVD, SAMP-KSVD, and StOMP-KSVD, respectively, and has the best effect among the compared algorithms.
4.3.3. Performance Analysis of CS Framework Reconstruction
Figure 7 compares the CS reconstruction performance of the above seven sparse basis construction methods in speech signals. The obtained results are similar to those achieved in the case of shockwave data. Compared with KSVD, CC-KSVD, PDR-KSVD, SAMP-KSVD, and StOMP-KSVD, under the same degree of normalized mean absolute error (), the number of measurements of ASE-KSVD algorithm was reduced by 48.75%, 28.18%, 49.44%, 55.16%, and 61.73%, respectively. The performance of ASE-KSVD algorithm is optimal among them. Figure 8 shows the reconstruction results of two randomly selected speech signals in the test set under 0.25 times downsampling ().
(a)
(b)
The relevant experimental results show that the ASE-KSVD algorithm proposed in this paper can be used to process actual sensor network test data with excellent performance.
5. Conclusions
In this work, we propose a novel ASE-KSVD dictionary training method. The method aims to adaptively adjust the number of iterations required for dictionary training by introducing an adaptive sparsity estimation strategy in the sparse encoding phase of dictionary training. By analyzing the algorithm characteristics and the comparative performance of CS reconstruction under two different data set conditions, it is confirmed that the ASE-KSVD algorithm can effectively handle the sparse signal representation and CS reconstruction problems. The algorithm significantly reduces the dictionary training time and sparse error compared to the fixed sparse basis algorithm. The reconstruction performance in the CS framework is better compared to similar algorithms. The upper limit of the sparsity estimation strategy in the current ASE-KSVD algorithm is determined by the parameter . The sparsity of the target signal is large under noise interference. In this case, the sparsity estimation will be smaller than the actual value, thus increasing the reconstruction error. In the subsequent research process, we will further explore the matching relationship between algorithm parameters and signal segment length to improve the algorithm’s generality and robustness.
Data Availability
Example code for the algorithms in this article can be obtained by contacting the corresponding author via email address: hantl@cust.edu.cn. But, the raw data required to reproduce these findings cannot be shared at this time as the data also form part of an ongoing study.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
Conceptualization, methodology, data curation, formal analysis, writing—original draft, M.J.; validation, writing—review and editing, and supervision, T.H. and H.L.; writing—review and editing, visualization, validation, and software, R.Y. and M.Z.; writing—review and editing, B.X. and X.L.
Acknowledgments
This project was supported by Key Research and Development Projects of Jilin Province Science and Technology Department (20210201042GX), China.