Image Noise Removal Method Based on Improved Nonlocal Mean Algorithm

Huang, Keya; Zhu, Hairong

doi:https://doi.org/10.1155/2021/5578788

Complexity

On this page

Abstract Introduction Analysis of Results Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2021 | Article ID 5578788 | https://doi.org/10.1155/2021/5578788

Image Noise Removal Method Based on Improved Nonlocal Mean Algorithm

Keya Huang¹and Hairong Zhu²

Academic Editor: Hocine Cherifi

Received06 Jan 2021

Revised30 Apr 2021

Accepted17 Jun 2021

Published05 Jul 2021

Abstract

Aiming at the problem of unclear images acquired in interactive systems, an improved image processing algorithm for nonlocal mean denoising is proposed. This algorithm combines the adaptive median filter algorithm with the traditional nonlocal mean algorithm, first adjusts the image window adaptively, selects the corresponding pixel weight, and then denoises the image, which can have a good filtering effect on the mixed noise. The experimental results show that, compared with the traditional nonlocal mean algorithm, the algorithm proposed in this paper has better results in the visual quality and peak signal-to-noise ratio (PSNR) of complex noise images.

1. Introduction

The data signal in the interactive system will be interfered by noise during the collection and transmission process. Random noise reduces image quality and conceals detailed feature information, which is not conducive to further analysis of feature recognition and target tracking. To improve the image quality of the interactive system and ensure the accuracy of the interaction, the current denoising algorithm needs to be optimized [1, 2]. Hassouni et al. removed image noise based on the HOS sequence. The motion trajectories are obtained recursively by a region-recursive estimation method [3]. Many scholars have designed detection methods for extracting image edges from noisy images [4, 5]. At present, in the image denoising problem of interactive systems, most of the research is to denoise a single type of noise. However, mixed Gaussian and salt and pepper noises are common in interactive environments, so it is of great practical significance to study the problem of mixed noise filtering [6–8].

Denoising methods are mainly divided into three areas: spatial domain denoising, transform domain denoising, and statistical method denoising. Spatial domain noise reduction refers to the use of spatial filtering algorithms for noise reduction processing. This algorithm acts directly on the observable pixels and depends on the target pixel and neighboring pixels. Spatial filters include linear filters [9] and nonlinear filters. Common linear filtering algorithms include Wiener filtering and median filtering [10]. The disadvantage of Wiener filtering is that the image data must be smooth, so its application is limited. The median filter replaces the intermediate gray value of the target pixel with the gray value of the pixel in a certain window, which is particularly effective for removing salt and pepper noise [11]. The mean filter is a common type of nonlinear filter, which replaces the target pixel intensity with the average pixel intensity of the target neighborhood in a specific window. There is also a bilateral mesh denoising algorithm that can effectively filter the image [12]. The above algorithms are simple and can effectively remove the corresponding noise. However, due to the different mathematical characteristics of different noises, it is difficult for these algorithms to be effective for multiple noises at the same time, and they will all cause the edges of the image to become blurred to a certain extent. As the window increases, the image will become more blurred [13].

Nonlocal mean (NLM) algorithm is an excellent spatial denoising algorithm [14–16]. The algorithm uses the characteristics of many similar blocks in the image and the average value of similar blocks to suppress image noise. At the beginning, Buades et al. [17] proposed a nonlocal average denoising algorithm based on the similarity (periodicity) of image features. Before this, the periodicity of the image was mainly used for texture synthesis. In recent years, there has been a lot of research and improvement on NLM algorithm denoising methods, which are mainly used for compressed sensing, feature extraction, pattern recognition, and image denoising. With the development of mathematical theory and image processing technology, good mathematical characteristics make its denoising advantages more obvious, and this algorithm has become one of the well-known denoising algorithms. There are also many scholars who have improved the classic NLM algorithm, especially the steps of dividing the pixel area of the traditional algorithm, so that it can retain more image structure information [18]. For example, the patch-based NLM algorithm is used in the field of medical image recognition to improve the fidelity of the image [19]. Although the principle of NLM algorithm is simple and the denoising effect is obvious, the calculation is not simple. The main reason is that the calculation of the similarity weight is complicated and takes a lot of calculation time. Therefore, how to remove pixels with little correlation while maintaining the noise reduction effect to reduce the amount of calculation will bring practical performance improvements to the NLM algorithm. The NLM algorithm has a weak filtering ability for salt and pepper noise. Therefore, it is necessary to improve the NLM algorithm to make it equally effective for salt and pepper noise, thereby enhancing the denoising performance and improving the image quality of the interactive system.

Aiming at some of the shortcomings of the traditional NLM algorithm, this article conducts in-depth research on it and introduces an adaptive median filtering (AMF) algorithm, and an improved NLM algorithm for the processing of mixed color noise is proposed, which is used to better denoise the image information after information fusion.

2. Denoising Algorithm Based on NLM Algorithm

In the process of weighted enhancement of pixels using the NLM algorithm, similar pixels are selected in a considerable range (which can be the entire image) instead of just taking a small neighborhood like traditional spatial filtering. The NLM algorithm was originally proposed for Gaussian noise, and the Gaussian noise model is

represents the image to be processed, and represents the original clear image. is the range of pixel coordinates of the image. (Ω) is the image noise. Assume that represents the denoising result for any pixel coordinate . The NLM filtering result of the gray value of the pixel is the weighted average of the gray values of all pixels in the noise-containing image . The corresponding mathematical model is as follows:

represents the image block centered on the pixel point i, the filter coefficient is set to , and represents the Gaussian weighted dissimilarity of the image block and the pixel. Its specific definition is as follows:

is the neighborhood of the pixel coordinates with the origin as the center point and the radius d, andwhere is a Gaussian kernel with a standard deviation of α. Equation (3) also satisfies

Compared with other traditional spatial filtering, the NLM algorithm uses the similar periodicity of nonlocal images and does not have strict requirements on image characteristics. It can be seen from equation (2) that the range of its calculation search box is an entire image in the defined interval Ω, and the rectangular window has the highest efficiency after testing. From the measure of dissimilarity in equation (3), the NLM algorithm calculates the weight through similar regions of the image instead of a single pixel. It has been tested that or rectangular windows have the highest efficiency.

The formula of NLM algorithm to remove noise iswhere h is the filter parameter and Y_i, Y_j is the image block centered on i, j. is the normalized factor, is the measure of dissimilarity between similar blocks of the image, where is the pixel neighborhood with d as the radius and the origin as the center, and is the Gaussian kernel with standard deviation α.

Convert the digital image collected by the sensor into a grayscale image. Let X represent the original image without noise and Y represent the image containing noise. represents the value range of pixel coordinates, and the value range of image gray is [l_min, l_max].

N is Gaussian noise whose mean is 0 and the standard deviation is σ. Then, the image model with Gaussian noise is

The transmission of interactive images generally contains salt and pepper noise. In the salt and pepper noise model, for ,where is the intensity of salt and pepper noise.

The above is the definition of Gaussian noise and impulse noise. The mixed noise model of the two noises is defined as

Through research, it is found that the median filter in the spatial domain, especially the AMF, has an outstanding effect on removing salt and pepper noise. Median filtering is generally divided into one-dimensional median filtering and two-dimensional median filtering. The algorithm principle of one-dimensional median filtering is to first sort the gray values of the pixels in the neighborhood of the target pixel from large to small and then take the median value from the ranked gray values to replace the gray value of the target pixel [20].

AMF has been widely used as a classic excellent filter for filtering salt and pepper noise [21–24]. represents the coordinate window of a rectangle of size , and is the maximum size of the rectangle window during the algorithm. The process steps of the AMF algorithm are shown in Algorithm 1.

	Initialize and start
(1)	Initialize window size L = 3.
(2)	Calculate the minimum, median, and maximum of pixel gray values.
(3)	If the minimum value of the gray value < the median value < the maximum value, go to step 5; otherwise L = L + 2.
(4)	If the window size is less than the maximum value of the window, go to step 2; otherwise replace the entire gray value with the median value.
(5)	If the minimum gray value < gray value < maximum gray value, then the gray value can be retained in the interval; otherwise the median value is used to replace the entire gray value.

From the algorithm flow chart in Figure 1, it can be found that there are two main strategies of the AMF algorithm:(1)Adaptive window adjustment strategy: it can automatically adjust the window size and automatically filter useful pixels that are not disturbed by noise in the neighborhood of the picture pixels.(2)Selective enhancement strategies: the AMF algorithm keeps the gray value of uncontaminated pixels unchanged and only enhances the pixels that may be contaminated by noise.

The principle and process of the NLM and AMF algorithms are analyzed above, and then the NLM filter will be improved, and the algorithm will be applied to interactive images to improve image quality.

In the calculation of the NLM algorithm, the weight calculation between pixels and is based on the measure of dissimilarity between their associated image blocks and . For Gaussian noise, the dissimilarity satisfies the following equation:

However, the dissimilarity measure of equation (10) does not hold for salt and pepper noise. To solve this problem, inspired by AMF, the calculation of this dissimilarity measure can be directly established on the image space containing non-salt and pepper noise.

First, define the spatial set of non-salt and pepper noise points in the image. is a noisy image, and and are any two pixels. Let denote the range of gray values in the image block and graphic block ; the non-salt and pepper noise point index space of these two image blocks is defined as follows:where represents the coordinate neighborhood of the pixel with the radius d and the origin as the center point. is the index space of non-salt and pepper noise points of the entire noisy image Y.

Based on this index space, the new measure of dissimilarity between and is as follows:

The dissimilarity measure of the classic NLM algorithm is based on all pixels in and , while the new dissimilarity measure is based on the non-salt and pepper noise points of the two associated image blocks. For salt and pepper noise, there iswhich is

Equation (14) can explain that the new dissimilarity measurement method proposed above maintains a certain degree of consistency between the noisy image and the original image. This consistency is also true for Gaussian noise .

is the number of non-salt and pepper noise point index spaces. The larger the , the stronger the consistency, and the smaller the , the weaker the consistency. To obtain an accurate dissimilarity measure, it is necessary to adjust the size of the pixel neighborhood K and then select an appropriate . Let and denote the minimum radius and maximum radius of the window area, and T is the allowable threshold of . The specific algorithm steps for obtaining in the improved algorithm are shown in Algorithm 2.

	Initialize, select any 2 pixels i and j
(1)	Set minimum radius d = d_min
(2)	If d ≤ d_max, calculate K_u, otherwise go to step 6
(3)	If j ∉ K_u, go to step 6
(4)	If , then let d = d + 1, and then go to step 2
(5)	Let , end
(6)	Let , end

Therefore, the final filtering result of the improved NLM algorithm is

It can be seen that the calculation complexity of the NLM algorithm is mainly determined by the weight calculation method. Compared with the classic NLM algorithm, the improved NLM algorithm has the advantages that the image block size can be adjusted adaptively, the weight calculation between pixels is based on the image block measurement, and the pixel weight average is based on non-salt and pepper noise points for enhancement.

3. Analysis of Results

This section designs two sets of experiments. The first group compares the calibration interface diagrams with different noises in the interactive system. The second group selected a classical grayscale image and a picture collected by a camera, introduced different degrees of noise and mixed noise, and then analyzed the image quality evaluation coefficient. Finally, the analysis of experimental results proves that the improved NLM algorithm has excellent denoising performance.

Experiment 1. Open the interactive system and select the interactive point positioning map of the system calibration interface, and compare and observe the calibration effect map after using different denoising algorithms. The red line trace represents the interaction point of induction. If there is noise in the image, it will affect the recognition of gestures. This experiment is mainly used as a reference for the recognition of noisy images.
Figure 2 shows the calibration interface after using different noise reduction algorithms. Figure 2(a) is the original image without any algorithm, Figure 2(b) is the positioning effect diagram after the NLM algorithm, Figure 2(c) is the positioning effect diagram after the ALM algorithm, and Figure 2(d) is the improved NLM algorithm effect diagram proposed in this paper.
It can be clearly seen from the results in Figure 2 that the system sensing points in the image after adopting the denoising algorithm are more accurate. The improved NLM algorithm has the best effect, but it is still not obvious. Considering that the complexity of the interactive environment is changing, it is necessary to quantify the data. Thus, the following experiments introduce different degrees of different types of noise for comparison.

(a)

(b)

(c)

(d)

Experiment 2. Figure 3 is a denoising comparison experiment. Figure 3(a) is an ordinary grayscale image, and Lena in Figure 3(b) is a classic grayscale image. The quantified evaluation of image noise generally uses the peak signal-to-noise ratio (PSNR), and the formula is as follows:Among them, is the mean square error. The larger the PSNR value, the better the denoising effect. Of course, there are many performance indicators that can measure the fidelity of the image, such as brightness, contrast, and structural similarity [25]. This paper mainly uses PSNR to judge the image fidelity.
In the experiment, the denoising effects of the improved NLM algorithm, NLM algorithm, and AMF algorithm are compared. The different parameter settings of different filters in the experiment are shown in Table 1.
For the NLM algorithm, , the radius of the image block is 3 and the radius of the search window is 8. For the improved NLM algorithm, is better, the radius of the search window is set to 8, and a, b, and T are set to 3, 5, and 14.
Next, the image will be polluted to different degrees, and then the denoising effects of each algorithm will be compared.
First, we introduce different degrees of salt and pepper noise to compare the effects of the two images. Table 2 shows the denoising effect data of Lena images polluted by different salt and pepper noises. Table 3 shows the denoising effect data of ordinary human images polluted by different salt and pepper noises. Figure 4 shows the denoising effect of Lena image polluted by salt and pepper noise. Figure 5 is a denoising effect diagram of an image of an ordinary person polluted by salt and pepper noise.
It can be seen from the above Figures 4 and 5 and Tables 2 and 3 that the improved NLM algorithm proposed in this paper has good denoising performance against salt and pepper noise in both images. The AMF algorithm is also very powerful in removing salt and pepper noise, which is not much different from the algorithm in this paper. In contrast, the NLM algorithm is not ideal for removing salt and pepper noise.
Then, introduce different degrees of Gaussian noise and then compare the experiments.
From the above Figures 6 and 7 and Table 4, it can be concluded that the improved NLM algorithm has excellent filtering effect on Gaussian noise, and PSNR value and visual effect are all better than other algorithms. Although the RSNR of the proposed algorithm is slightly lower than that of the traditional NLM algorithm, the difference is not big, and the final filtering effect is not much difference.
Finally, the mixed noise is introduced into the image, and then the performance of the denoising algorithm is considered from the aspects of PSNR and visual effect. The results are as follows.
From the performance of each denoising algorithm in the above Figures 8 and 9 and Tables 5 and 6, the improved NLM algorithm proposed in this paper has obvious advantages over other filters in removing mixed noise.
Two conclusions can be drawn from the above experiments:(1)The improved NLM filtering algorithm proposed in this paper has excellent performance in both mixed noise removal and single noise removal. Both the visual effect and the PSNR have reached a good level with obvious advantages. However, both the AMF algorithm and the classic NLM algorithm can only filter specific noise in the image and cannot deal with the image containing complex noise.(2)Based on the improved NLM algorithm, the precise positioning of the interactive system calibration interface is effectively improved, and the interactive point positioning is accurate, which further proves the effectiveness of the algorithm.

(a)

(b)

(a)

(b)

(c)

(d)

(a)

(b)

(c)

(d)

(a)

(b)

(c)

(d)

(a)

(b)

(c)

(d)

(a)

(b)

(c)

(d)

(a)

(b)

(c)

(d)

4. Conclusion

To solve the problem of Gaussian and salt and pepper noise in the image processing stage of an interactive system, this paper proposes an improved NLM algorithm for filtering mixed noise. Compared with the traditional NLM algorithm, the improved algorithm can better remove the mixed noise in the image, significantly improve the PSNR of the image, and get a clearer image, which is more conducive to the interactive system to extract useful gesture or body information from the image. The next work will explore the algorithm for color image processing effect, and whether we can continue to improve the algorithm to reduce the time spent on image denoising to reduce the interaction delay.

Data Availability

No data were used to support this study.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

The work was supported by the Qing Lan Project of the Higher Education Institutions of Jiangsu Province ([2019]3 and [2020]10) and Basic Science Research Project of Nantong (JC2020154).

References

P. Korus and J. Huang, “Multi-scale fusion for improved localization of malicious tampering in digital images,” IEEE Trans. Image Processing., vol. 25, no. 3, pp. 1312–1326, 2016.
View at: Publisher Site | Google Scholar
L. Yu, J. Hou, and S. Fei, “Gesture tracking and locating algorithm based on federated tracking filter,” Journal of Dynamic Systems Measurement and Control-Transactions of the Asme, vol. 141, no. 12, Article ID 124501, 2019.
View at: Publisher Site | Google Scholar
M. E. Hassouni, H. Cherifi, and D. Aboutajdine, “HOS-based image sequence noise removal,” IEEE Transactions on Image Processing, vol. 15, no. 3, pp. 572–581, 2006.
View at: Publisher Site | Google Scholar
S. Rital, A. Bretto, H. Cherifi, and D. Aboutajdine, “A combinatorial edge detection algorithm on noisy images,” in Proceedings of International Symposium on VIPromCom Video/Image Processing and Multimedia Communications, pp. 351–355, Zadar, Croatia, June 2002.
View at: Publisher Site | Google Scholar
A. Bretto and H. Cherifi, “Noise detection and cleaning by hypergraph model,” in Proceedings International Conference on Information Technology: Coding and Computing (Cat. No.PR00540), pp. 416–419, Las Vegas, NV, USA, March 2000.
View at: Publisher Site | Google Scholar
L. Ji and Z. Yi, “A mixed noise image filtering method using weighted-linking PCNNs,” Neurocomputing, vol. 71, no. 13–15, pp. 2986–3000, 2008.
View at: Publisher Site | Google Scholar
L. Yu and J. Hou, “Large screen interactive imaging system with switching federated filter method based on 3D sensor,” Complexity, vol. 2018, Article ID 8730281, 11 pages, 2018.
View at: Publisher Site | Google Scholar
L. Yu, C. Li, and S. Fei, “Any-wall touch control system with switching filter based on 3-D sensor,” IEEE Sensors Journal, vol. 18, no. 11, pp. 4697–4703, 2018.
View at: Google Scholar
M. Mahmoudi and G. Sapiro, “Fast image and video denoising via nonlocal means of similar neighborhoods,” IEEE Signal Processing Letters, vol. 12, no. 12, pp. 839–842, 2005.
View at: Publisher Site | Google Scholar
S. Shao and K. Zhang, “An improved multisensor self-adaptive weighted fusion algorithm based on discrete Kalman filtering,” Complexity, vol. 2020, Article ID 9673764, 9 pages, 2020.
View at: Publisher Site | Google Scholar
S. Esakkirajan, T. Veerakumar, A. N. Subramanyam et al., “Removal of high density salt and pepper noise through modified decision based unsymmetric trimmed median filter,” IEEE Signal Processing Letters, vol. 18, no. 5, pp. 287–290, 2011.
View at: Google Scholar
S. Fleishman, I. Drori, and D. Cohen-Or, “Bilateral mesh denoising,” in Proceedings of ACM SIGGRAPH 2003 Papers (SIGGRAPH ‘03), pp. 950–953, Association for Computing Machinery, San Diego, CA, USA, July 2003.
View at: Publisher Site | Google Scholar
R. He, Y. Zhang, Z Sun, and Q Yin, “Robust subspace clustering with complex noise,” IEEE Transactions on Image Processing, vol. 24, no. 11, IEEE, 2015.
View at: Publisher Site | Google Scholar
L. Yu, “Image noise preprocessing of interactive projection system based on switching filtering scheme,” Complexity, vol. 2018, Article ID 1258306, 10 pages, 2018.
View at: Publisher Site | Google Scholar
C. Karam and K. Hirakawa, “Monte-carlo acceleration of bilateral filter and non-local means,” IEEE Transactions on Image Processing, vol. 27, no. 3, pp. 1462–1474, 2018.
View at: Publisher Site | Google Scholar
J. Qiyu, G. Ion, K. Charles, and Q. Liu, “Nonlocal means and optimal weights for noise removal,” SIAM Journal on Imaging Sciences, vol. 10, no. 4, pp. 1878–1920, 2017.
View at: Publisher Site | Google Scholar
A. Buades, B. Coll, and J. M. Morel, “A non-local algorithm for image denoising,” in Proceedings of IEEE Computer Society Conference on Computer Vision & Pattern Recognition, IEEE, San Diego, CA, USA, June 2005.
View at: Publisher Site | Google Scholar
Q. Zhao, D. Chen, E. Jiang et al., “Improved weighted non-local mean algorithm filter for image denoising,” Journal of Electronic Measurement and Instrumentation, vol. 28, no. 3, 2014.
View at: Google Scholar
G. Nicolas, N. Kunio, C. Pierrick, S. F. Vladimir, L. A. Douglas, and D. L. Collins, “Non-local means inpainting of MS lesions in longitudinal image processing,” Frontiers in Neuroscience, vol. 9, 2015.
View at: Publisher Site | Google Scholar
A. Kaur, R. Malhotra, and R. Kaur, “Performance evaluation of non-iterative adaptive median filter,” in Proceedings of 2015 IEEE International Advance Computing Conference (IACC), pp. 1117–1121, Banglore, India, June 2015.
View at: Google Scholar
N. I. Chervyakov, P. A. Lyakhov, and A. R. Orazaev, “Two methods of adaptive median filtering of impulse noise in images,” Computer Optics, vol. 42, no. 4, pp. 667–678, 2018.
View at: Publisher Site | Google Scholar
Z. Zhang, D. Han, J. Dezert, and Y. Yang, “A new adaptive switching median filter for impulse noise reduction with pre-detection based on evidential reasoning,” Signal Processing, vol. 147, pp. 173–189, 2018.
View at: Publisher Site | Google Scholar
S. Song, X. Dai, Z. Huang, and D. Gong, “Load parameter identification for parallel robot manipulator based on extended Kalman filter,” Complexity, vol. 2020, Article ID 8816374, 2020.
View at: Publisher Site | Google Scholar
K. Han, “Personalized news recommendation and simulation based on improved collaborative filtering algorithm,” Complexity, vol. 2020, Article ID 8834908, 2020.
View at: Publisher Site | Google Scholar
G. Zhai and X. Min, “Perceptual image quality assessment: a survey,” Science China Information Sciences, vol. 63, no. 11, Article ID 211301, 2020.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2021 Keya Huang and Hairong Zhu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies