Abstract

Aiming at the problem that the image sharpness evaluation algorithm in the photoelectric system has a slow speed in actual processing and is severely disturbed by noise, an improved image sharpness evaluation algorithm is proposed by combining multiscale decomposition tools and multidirectional gradient neighbourhood weighting. This paper applies non-subsampled shearlet transform (NSST) to perform multiscale transformation of the input images, obtaining high-frequency sub-band images and low-frequency sub-band images. In order to enhance the detection of the edge orientation of images, multidirectional gradient processing of the image matrix is added to each sub-band image. In addition, the weight corresponding to the current pixel is obtained by calculating the inverse ratio of the gradient of each direction and the distance of the center pixel. Through calculating the ratio of the gradient neighbourhood weighting operators of high-frequency sub-band images and low-frequency sub-band images, the image sharpness evaluation value can be acquired further. Moreover, the image sequence collected by a certain type of photoelectric system is selected as the image sequence of the noisy real environment for simulation experiments and compared with the current mainstream algorithms. Finally, the experimental draws a conclusion that compared with the mainstream evaluation algorithms, the evaluation results of the proposed method perform better in terms of steepness, sensitivity, and flat area fluctuation, it can better suppress noise and improve accuracy, and its running speed meets the basic requirements of the image sharpness evaluation algorithm.

1. Introduction

With the rapid development of digital image technology, as a core link of photoelectric systems, autofocus technology becomes an indispensable tool for obtaining clear images in a modern high-speed information society, which affects the image quality of imaging and the effectiveness of subsequent image algorithms [13].

The sharpness evaluation function is a core part of autofocus technology. At present, mainstream algorithms for evaluating the sharpness of an image include spatial domain evaluation algorithm, transform domain evaluation algorithm, and statistical evaluation algorithm [46]. Spatial domain evaluation function makes use of images’ features in the spatial domain to distinguish blurred images and clear images. Because the function directly calculates the gray value of the image pixels, the amount of calculation is less and the calculation speed is faster [79]. However, due to the fact that the edge and contour of an object is complex and changeable, the spatial evaluation function only detects the horizontal and vertical gradients of images, which cannot detect the complex edge contour accurately [10]. Bi et al. [11] proposed a spatial domain evaluation function based on HSV weighting, with a strong ability of evaluating the automatic focus control of the embedded real-time optical measurement platform efficiently. Zhang et al. [12] proposed an improved Brenner sharpness evaluation function. Compared with the gray gradient evaluation function, this evaluation function maintains the unimodality and validity of the evaluation curve under Gaussian noise. It is particularly applicable in the fast-focusing occasion, but the practicability in complex noise environments remains to be tested. Li et al. [13] proposed a sharpness evaluation algorithm that uses sharpness evaluation factors to modify the detection results of the Laplace operator. Under high-noise circumstance, this algorithm can attain fast and good evaluation results, but in some special scenes, the running time of denoising is longer, and the evaluation speed of sharpness is slower. Liu et al. [14] proposed an image sharpness evaluation algorithm based on curve fitting and probabilistic model theory. Although this method effectively improves the limitation that one focusing function is only applicable for one certain type of images, it costs too much system runtime to achieve higher accuracy. When the curve’s fitting parameters are complex, the system requires a lot of running time.

According to the analysis of aforementioned image sharpness evaluation functions, aiming at the situation that the edge contours of actual objects in the photoelectric system are complicated and changeable, the image sharpness evaluation is greatly affected by noise. This paper performs multiscale transformation of the input images, introduces multidirectional gradient calculation on the basis of the spatial evaluation function, and obtains accurate image edge contours. From the perspective of optical imaging point diffusion principle, using the correlation among image pixels, combining image gradients, a multidirectional gradient image sharpness evaluation algorithm associating with neighbourhood correlation is proposed.

2. Image Sharpness Evaluation Criteria

2.1. Qualitative Evaluation Criteria

An excellent image sharpness evaluation function should have an evaluation curve with good performance. The ideal digital image focus curve is shown in Figure 1.

It can be seen from Figure 1 that the ideal focus curve has a single peak, and the peak is the best clear image [1517]. In the actual applied process, there exist many interfering factors (such as the vibration of the motor in the photoelectric system). The captured images have noise due to environmental factors, and it may fluctuate up and down in the out-of-focus area, or even multipeak may appear [1820].

2.2. Quantitative Evaluation Criteria

The qualitative evaluation of image sharpness can roughly judge the pros and cons of the evaluation algorithm, but when the performance of the algorithm is close, the performance of an algorithm cannot be intuitively judged through qualitative evaluation. Therefore, a specific quantitative index is needed to reflect an algorithm’s merits and demerits. This paper uses five standards [2123] unified by industry research to quantitatively evaluate the sharpness function of the images. The schematic diagram of the quantitative evaluation standard for the sharpness evaluation function is shown in Figure 2.

In Figure 2, the vertical axis is image sharpness evaluation value, and it is recorded as the FV value; the horizontal axis is the focus motor position, and the value is Z; The maximum extreme value of its focus curve is recorded as point .

2.2.1. Criterion 1: Steepness Width

The numerical characteristics of the focus curve are divided into steep areas and flat areas, and the flat areas of the focus curve are subdivided into left flat areas and right flat areas. The coordinates of the left and right critical points of the steep area are and , and the distance between the critical points is defined as the width of steep area as shown in the following equation:

2.2.2. Criterion 2: Steepness

If two different sharpness evaluation function curves have the same maximum and minimum values, that is, both evaluation functions have the same sharpness ratio. A steepness with better performance resolution capability is needed to accurately evaluate the evaluation function. The equation is expressed aswhere represents the steepness of the sharpness evaluation function (i.e., the steepness of the focus curve), represents the maximum value of the focus curve, represents the minimum value of the focus curve, and represents the width of the steep region of the focus curve.

2.2.3. Criterion 3: Flat Area Fluctuation

The amount of fluctuation in the flat area can be measured in the form of standard deviation, and the amount of fluctuation in the flat area can be expressed aswhere represents the fluctuation of the smooth area of the sharpness evaluation function, represents the standard deviation of the smooth area of the focus curve, represents the value of a single sampling point, and represents the average value of all sample points.

2.2.4. Standard 4: Sensitivity

Sensitivity can be expressed aswhere represents the sensitivity of the sharpness evaluation function, represents the reduction of focus value, and represents the maximum value of focus value. The evaluation algorithm with higher sensitivity is more favorable for peak search.

2.2.5. Criterion 5: Average Calculation Time

The average calculation time is the average time taken by the sharpness evaluation function to process one frame of the image in the image sequence.

3. Proposed Methods

According to Wiener–Khinchin’s theorem [24], two effects of the correlation function are used. First, the gradient function is weighted by the correlation performance generated by defocus blur, distinguishing blurred images and clear images more clearly, and enhances the steepness of the focus curve. Second, correlation is an inherent property of an image. Because there is a certain relationship among pixel content and there is no correlation between noisy pixels and content pixels, the correlation of the pixel content can be used to remove the effect of noise.

3.1. Multiscale Transformation of Images

In recent years, researchers have proposed many effective image multiscale decomposition tools. Compared with curvelet transform and contourlet transform, non-subsampled shearlet transform (NSST) has a parabola scale, and its direction sensitivity is stronger and optimum sparseness [2527]. During the sampling process, NSST avoids losing images’ information and has stability as well as translation invariance. Therefore, NSST is very suitable for expressing detailed information of an image such as contours, textures, and edges.

According to synthetic wavelet theory, when the decomposition dimension , the affine function of the shear wave can be expressed aswhere , represents an integrable space, and are second-order invertible matrices, represents a scale transformation matrix, represents a geometric transformation matrix, and .

When satisfies the Parseval framework, the synthetic wavelet (element in the function system) can be expressed as

When , the shearlet change system can be expressed aswhere are three parameter variables in the shearlet transform system. is the size of the image, is the cutting direction, and is the translation amount. Different images’ details are obtained by setting these three parameters.

Shearlet is a multiresolution and multidirectional transformation system. It is a nondownsampling transformation method of NSST. It uses nondownsampling pyramid filters to scale and filter low-frequency approximate images and high-frequency detailed images. The size of each decomposition image is the same as the original input image.

3.2. Multidirectional Gradient Calculation

The multidirectional gradient calculation is performed on the low-frequency approximate image and the high-frequency detailed image. The gradient of each direction of the image center pixel is shown in Figure 3.

In Figure 3, the gradient vectors of the four directions of the central pixel are , , , and . , and calculate the gradient of the central pixel as shown in the following equation:

In equation (8), , , , and correspond to , , , and vector magnitude, and the distance between adjacent pixels adapts the European formula.

The gradient values of the four directions of the central pixel are added to the entire image matrix, and the gradient calculation formula of the entire image matrix is obtained as

3.3. Numerical Calculation of Image Sharpness Evaluation

For each center pixel, find the cross-correlation value of neighboring pixels. First, calculate the average gray value of neighboring pixels. The calculation formula is

Then, find the cross-correlation value of the pixels to this average. In order to ensure that the cross-correlation value of every pixel is less than 1, the mathematical mean inequality is used to obtain equation (11) for calculating the cross-correlation.

The correlation value is small in clear images, but it is large in defocused images, which has the opposite effect to the expression of the image gradient value. Therefore, the absolute value () of the difference between the cross-correlation value and 1 is taken as the weighting coefficient. The final calculation formula is shown in the following equation:

The final evaluation value F can be expressed aswhere represents the evaluation value of high-frequency detail images and represents the evaluation value of low-frequency approximate images.

Under the effect of correlation weighting on image gradients, the proportion of large gradient values increases and the proportion of small gradient values decreases, which further highlights the difference between clear images and blurred images and improves sensitivity. At the same time, the characteristics of the correlation itself are used to reduce the sensitivity of the gradient to noise and improve the noise resistance.

4. Results and Analysis

4.1. Experimental Platform

The experimental hardware platform used in this paper is CPU Intel Core I7, CPU clock speed 2.20 GHz, RAM 4G, OS: 64 bit Windows. The experimental software platform is MATLAB 2018b.

In order to verify the effectiveness of the proposed method for evaluating image sharpness, this paper used a certain type of photoelectric system to collect an image sequence as the image sequence which has a real noisy environment. The total number of frames of this image sequence is 461 frames, and the resolution is 640 × 512, as shown in Figure 4. The algorithms used for qualitative and quantitative analysis include gray variance evaluation algorithm, Brenner evaluation algorithm, Roberts evaluation algorithm, Prewitt evaluation algorithm, Laplace evaluation algorithm, Tenengrad evaluation algorithm, Vollath evaluation algorithm, entropy function evaluation algorithm, Fourier transform evaluation algorithm, discrete cosine transform (DCT) evaluation algorithm, and wavelet transform evaluation algorithm.

4.2. Qualitative Analysis Results

In order to guarantee the computational efficiency of the image sharpness evaluation algorithm, this paper evaluated only 1⁄4 of the central area in the image. Due to the large amount of image data and many curves, in order to better observe the curve of the evaluation of sharpness, all the experimental results are divided into two groups to display, six kinds of the results per group. The final comprehensive result is shown in Figure 5.

It can be seen from Figure 5 that the overall performance of gray variance evaluation algorithm, DCT transform evaluation algorithm, and entropy function evaluation algorithm curve is poor, the focus curve is less distributed in the flat area, and the whole shows a floating state, which has a large gap compared with other curve trends. These three evaluation algorithms are not suitable for sharpness evaluation in practical applications. Laplace evaluation algorithm can be clearly seen that there are more jitters and a lower signal-to-noise ratio. Because its own second-order differential amplifies gradient and enhances noise, it is very sensitive to noise. The gradient-based sharpness evaluation algorithm is generally consistent with the focus curve trend, but it can be clearly seen that the proposed method in this paper has a better curve trend and a better unimodality, as well as a smooth flat region.

4.3. Quantitative Analysis Results

In order to apply to the focus curves of different evaluation functions, the left and right critical points of the focus curve are selected by where is the maximum focus value of the focus curve, is the minimum focus value, is the focus value of a single sampling point, and is a coefficient, and this paper takes . The sampling points on the focus curve are traversed from left to right. The first sampling point that meets formula (14) is set to the left critical point and is traversed from right to left similarly. The first sampling point that matches formula (14) is set to the right critical point. The results of quantitative analysis based on the five quantitative evaluation criteria are shown in Table 1.

It can be seen from Table 1 that Roberts, Brenner, and the proposed method have a relatively smaller width of the steep region compared with other methods, and the width of the steep region of the proposed method is the smallest. Therefore, compared with other methods, the proposed method has a higher steepness; in the aspect of flat area fluctuation, this method does not have a clear superiority contrasted to Brenner, but it has obvious advantages when compared with other methods, indicating that the proposed method has a better antinoise performance in the flat area; and in terms of sensitivity, the proposed method has a higher sensitivity in the peak region and is sensitive to image out-of-focus state. Finally, the average calculation time of the proposed method is longer than other gradient evaluation algorithms.

5. Conclusion

Aiming at the problem that the image sharpness evaluation algorithm in the photoelectric system runs slowly in actual processing and is seriously affected by noise, this paper proposes a sharpness evaluation algorithm based on multidirectional gradient function neighbourhood weighting, which combines multiscale transformation tools and gradient neighbourhood weighting operators to effectively evaluate the image sharpness. First, NSST is used to perform multiscale transformation on the images to obtain high-frequency detail images and low-frequency approximate images. Second, multidirectional gradients on high-frequency images and low-frequency images are calculated, and the calculation results are weighted in conjunction with neighbourhood correlation. Finally, the ratio of high-frequency images and low-frequency images weighted results is used as the evaluation value for the sharpness evaluation function. This paper selects a certain type of photoelectric system acquisition image sequence as the image sequence of the noisy real environment and implements different evaluation functions of sharpness on the MATLAB experimental platform to compare with the proposed method. Through the intuitive evaluation analysis and quantitative evaluation analysis of the focus curve, the proposed sharpness evaluation algorithm has better unimodality. In contrast with the current mainstream image sharpness evaluation algorithms, the proposed method has a better performance in terms of sensitivity and flat area fluctuation, but it still needs to be improved in terms of running time.

Data Availability

The image data used to support the findings of this study are included within the supplementary information file.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China Youth Science Foundation Project (no. 61802305, name: Research on Feature Extraction Method of Criminal Investigation Survey Image Based on Multi-type Filter Cooperation) and Xi’an University of Posts and Telecommunications Innovation Fund (no. 2018SC-01, name: Application Research of Ncam-based Augmented Reality Tracking and Shooting System in Film and Television Industry).

Supplementary Materials

(1) chapter3_data1.xlsx: normalized sharpness evaluation values of each algorithm. (2) chapter3_data1_1: plot drawn by MATLAB based on the data in chapter3_data1_1.fig, which is also Figure 5 in this manuscript. (Supplementary Materials)