Abstract
It is known that many image enhancement methods have a tradeoff between noise suppression and edge enhancement. In this paper, we propose a new technique for image enhancement filtering and explain it in human visual perception theory. It combines kernel regression and local homogeneity and evaluates the restoration performance of smoothing method. First, image is filtered in kernel regression. Then image local homogeneity computation is introduced which offers adaptive selection about further smoothing. The overall effect of this algorithm is effective about noise reduction and edge enhancement. Experiment results show that this algorithm has better performance in image edge enhancement, contrast enhancement, and noise suppression.
1. Introduction
The presence of noise in image is a major problem that typically negatively affects image analysis and interpretation process. Therefore, to improve the performance of higher level processing stages, a filter method has to be applied in order to reduce noise, enhance edges, and consequently to obtain a better estimate of the ideal image. The purpose of smoothing is of twofold; noise is eliminated to facilitate further processing, and features irrelevant to a given problem are ruled out to reduce the complexity for the subsequent processing. The designs for filters have been conducted for a long time. Many of them may enable to perform some filtering functions with loss of useful information at the same time. In other words, these filters are appropriate when both smoothing and discontinuity preservation objectives are desired. The literature on signal and image filter is vast, and comprehensive review is beyond the scope of this paper. We only introduce several important nonlinear image filter methods which are relevant to our method.
PDE-based image processing methods became widespread after the work of Perona and Malik [1], where they proposed a modified version of the heat equation called anisotropic diffusion that adapted the diffusivity to image features. The anisotropic diffusion equation is also the first variation of an image energy that seeks piecewise constant solutions. A multitude of nonlinear PDE models have been developed for a wide variety of images and applications [2–5]. The nonlinear PDE models have proven to be effective, but only for particular applications where the input data is well suited to the model's underlying geometric assumptions. The parameter tuning is a challenge because it entails fuzzy thresholds that determine which image features are enhanced and which are smoothed away.
Tomasi and Manduchi [6] described a bilateral filter, which does a robust averaging in Gaussian-weighted image neighborhoods. It is successfully applied to image denoising [7, 8], mesh smoothing and denoising [9, 10], and coherence enhancing [11]. A theoretical analysis of this method is presented in [11]. The principle of bilateral filtering is simple. It combines gray levels or colors based on both their geometric closeness and their photometric similarity and prefers near values to distant values in both domain and range. Bilateral filter is a local adaptive filter, but it is not as robust as is claimed. It cannot suppress the impulse outliers effectively. Xu and Pattanaik [12] proposed a noniterative bilateral method for Monte Carlo noise reduction; this method has good performance on salt and pepper noise but fails on Gaussian noise.
The mean-shift procedure [13–15] moves each sample in a feature space to a weighted average of other samples using a weighting scheme that is similar to Parzen windowing. We can also view this as moving samples uphill on a PDF. Comaniciu and Meer [15] proposed an iterative mean-shift algorithm for image intensities, where the PDF does not change with iterations. Each gray scale or vector pixel intensity is drawn toward a local maximum in the corresponding PDF. Mean-shift algorithm utilizes kernel function properties to estimate gradient orientation of the point and find the bandwidth of the kernel function without estimating whole region probability density. It is an effective iterative algorithm.
To efficiently smooth the image while preserving the textures, an enhanced kernel regression method (EKRM) is proposed in [16]. It is noniterative. In this algorithm, the Gaussian kernel regression is explored. Then, Taylor expansion is used in the regression function.
The adaptive filter method applies both linear and nonlinear algorithms but is mainly characterized by a filter parameter adjustment depending on the local features detected over each particular location on the image. A reliable measure for the presence of a discontinuity in the processing window must be available. The structure of the filter then depends on this measure to avoid smoothing over an edge. The adaptive filter method we proposed here is similar to EKRM [16] but with the following improvements. First, a homogeneity criterion is used, which can give the homogeneity image of the original image. Next, the homogeneity image derived from the input image is used as the filter parameter to control the adaptive behavior in the kernel regression, which offers a principle for filtering the pixel. When the homogeneity value is high, Taylor expansion is used on this pixel and the pixel is judged as an edge pixel. When the homogeneity value is low, the pixel is treated as an interior pixel of a region, so the Gaussian filter is applied and the image is smoothed. It is shown to suppress noise while sharpening edge present in the input images.
The paper is organized as follows. Section 2 presents a brief review of classical kernel regression and its application in image filters. Section 3 introduces the local homogeneity measure of images. We propose our new filter method in Section 4, and some comparative results are shown in Section 5. Acknowledgments concludes this paper.
2. Kernel Regression for Image Filter
In this section, we review the EKRM method briefly and analyze its relation with human visual perception.
2.1. Kernel Regression Filter Based on Taylor Expansion (EKRM)
Classical parametric image processing methods rely on a specific model of the signal of interest and seek to compute the parameters of this model in the presence of noise. A generative model based upon the estimated parameters is then produced as the best estimate of the underlying signal. Nonparametric methods rely on the data itself to dictate the structure of the model, in which case the implicit model is referred to as a regression function. Given a set of independent observations coming form a population , where and are called predictor variable and response variable, respectively, we want to find a smooth function , is called regression function. Kernel regression offers a way of estimating the regression function without the specification of a parametric model. In kernel regression, the value of a point can be estimated by a weighted average of this point and its neighbors. This weight function is often referred to as a kernel. Usually, the kernel is continuous, bounded, and symmetric real function that integrates to one. For univariate regression, the weight sequence for a kernel smoother is defined by where the normalized term is the kernel with scale factor .
Gaussian kernel is a popular one in kernel regression. Zhang et al. [16] introduced the second derivatives of Gaussian kernel into kernel regression and applied it in image filter. The new kernel is derived from Taylor expansion theorem. Consider the univariate nonparametric regression model is where is a smooth function and is a random noise which submits to normal distribution .
The kernel function is chosen to be When it comes to estimate , the noise of observations is generally removed by smoothing , and the estimate for is Then, discrete estimate of can be expressed as the widely used N-W estimator form [17, 18] where is the bandwidth and is the sample number.
The th derivative of , denoted by , can be obtained from (2.5) as The discrete estimate form of can be expressed as For a normal density with variance , the following equations are true: Expanding in a Taylor series with respect to , we obtain That can be estimated by Introducing a positive parameter and and substituting for and for , (2.11) becomes Here, and are bandwidths, is sample number, is a fitting coefficient, and is the normal density kernel.
For image filter, we use -dimensional kernel for kernel regression. The coordinates are predictor variables , and their intensity values are response variables . So, the regression model is where , are i.i.d. scalar variables with , , and is called variance function.
The -dimensional kernel function is chosen as Correspondingly, the discrete form of and can be estimated by Then, we have the following equation:
2.2. Its Relation with Human Visual Perception
It has long been known that retinal ganglion cells and cells of the lateral geniculate body have a receptive field (RF) consisting of two regions, an approximately circular center and an annular surround [19]. There are two types of typical retinal ganglion cells—ON center cell and OFF center cell. Figure 1 illustrates these two types of receptive fields. The ON center cell may respond strongly when the center is stimulated by illumination and the response is restrained when the surround is stimulated as shown in Figure 1(a). Diffuse illumination covering both center and surround may excite the cells weakly. For the OFF center cell, it responds strongly when the surround is stimulated and the response is restrained when the center is stimulated as shown in Figure 1(b). It has been found that the ganglion cell's response model is equal to the difference of two Gaussian functions [20], While the antagonistic center/surround organization improves the representation of local contrast, it reduces the sensitivity to global brightness over an extended area while the edge becomes prominent. Li et al. [21] demonstrated an extensive disinhibitory region (DIR) outside the classical inhibitory surround of the receptive field (RF). By adding a disinhibitory mechanism, the average output gray level increases and the transfer property of the low spatial frequency components is clearly improved. It can explain how the visual system transfers the various gradients of luminance of an extended area and represents the mean luminance in visual environment. They use the algebraic sum of three Gaussian functions to represent it by adding a third positive Gaussian representing the outer surround disinhibitory mechanism is the sensitivity profile of the receptive field. , , and are the peak sensitivity values for the center, surround, and outer-surround mechanisms, respectively. The output of the cell response is the following model: where is relative stimulus intensity over space ; represents stimulus area. We denote , and is the center of the receptive field, then According to the three Gaussian models as (2.19), we select (2.21) can be rewritten as Here, .
(a)
(b)
As Taylor expansion in kernel regression, If we select , , and , we can find that (2.23) has the identical form with (2.22), so the EKRM method can be explained in human visual perception theory. It approximates the responsiveness and transfer function of lateral geniculate nucleus (LGN) cells.
3. Adaptive Image Enhancement Method
In this section, we propose an adaptive image enhancement method. Section 3.1 describes the local homogeneity computing method used in our algorithm, which is easy and efficient for computing. The details about the new algorithm are given in Section 3.2.
3.1. Local Homogeneity
The main problem in designing nonlinear filter is how to detect discontinuities. A reliable measure for the presence of a discontinuity in the processing window must be available. The structure of the filter then depends on this measure to avoid smoothing over an edge. The homogeneity is often used as the discontinuity measure. To quantize the homogeneity of a pattern is not a trivial task. In this paper, we use the criterion proposed in [22]; this method is easy to compute and shows pretty good results.
The pixels of an image can be viewed as a set of spatial data points located in a 2D plane with the top left corner being the origin. Denote the location of a pixel as and its intensity as . Let be the pattern to compute homogeneity. Currently, we consider to be a square window of width . Let be the center of the pattern with the attribute being . Each pixel , in corresponds to a vector . Based on , a new vector is constructed as follows: Let be the sum of all the vectors defined in , that is, The measure is defined as the norm of , that is, . In homogeneity region, the values are very small, and in the regions containing several homogeneity regions, the values are usually high. The higher the value is, the more likely that its central pixel is near region boundary. The -image is a gray-scale image whose pixel values are the values calculated over local windows centers on those pixels. The dark and bright areas in the -image represent the region centers and region boundaries, respectively. The figures in [22] demonstrate some example patterns and their corresponding values. We make a linear transform to convert the value into , that is, Then, can be used as the discontinuity measure in our algorithm. Figure 2 gives the example of -image in our method. The original image in Figure 2(a) includes three circles filled with different colors. As we noted that homogeneity can be used as the discontinuity measure. In this image, the circle boundary is the discontinuities we need to find. Figure 2(b) shows that the discontinuity points can be clearly detected using our homogeneity measure. The homogeneity values are high at the boundary of the circles and low in the interior regions.
(a)
(b)
3.2. The Proposed Adaptive Kernel Regression Filter
We apply the local homogeneity in EKRM method and propose an adaptive image enhancement method (AIEM). In EKRM, the image filter is based on Taylor expansion without any selection. So any pixel no matter it is noise or edge pixel is enhanced. In our method, we employ the local homogeneity as a parameter. The pixels in homogeneity region are smoothed using Gauss smoothing and the pixels in inhomogeneity region are enhanced. As proposed in [16], Another parameter is introduced for the bandwidth in the derivative of , then Using the properties of the symmetric positive definite matrix and , we have a more manageable form of (3.4) Here , is an orthogonal matrix with such that Then, (3.4) and (3.5) are written as follows: here, is the new eigenvalue of , and is the new eigenvalue of . In [16], a positive parameter is introduced which is arbitrary. Then, the regression formula is We make an improvement about the regression formula. In our method, we introduce an adaptive positive parameter instead of , which is a local homogeneity measure about the pixel . The criteria used to detect the homogeneity region are based on the comparison of the local homogeneity measurement here . We substitute for and finally get a new filter formula for image processing here
For simplicity, we suppose that is a diagonal matrix with equal diagonal elements, then and (3.11) becomes where , which is easy to be estimated.
In our method, when a region is declared homogeneous, the homogeneity measure is zero, which means Gaussian filter is applied to smooth the central pixel. When no homogeneous region is detected, that is, when trying to estimate over a pixel which is close to a discontinuity, the adaptive filter is applied to preserve this discontinuity.
4. Experiment Results and Discussion
In this section, we show the performances of the proposed kernel regression algorithm on simulate data and real noisy images, then we discuss the merits of our method.
Figure 3 is the result we test on simulate data. There is a sharp boundary between a bright and a dark region as shown in Figure 3(a). The homogeneity value on the boundary is one and approximate zero elsewhere. After applying the proposed method on it, the boundary is preserved, and the interior region is smoothed as shown in Figure 3(b). The proposed method can efficiently smooth out noise and enhance the boundary. Figure 4 gives the examples of our filter results on gray images comparing with other methods. Figure 4(a) is the original input gray image. Figures 4(b)–4(e) are the images filtered by PDE method, bilateral filter, mean-shift filter, and EKRM method, respectively. Figure 4(f) is the image filtered by the proposed method. The original input images are of low contrast and somewhat blurred. Reducing the noise and blurring, sharpening the edge, and increasing the contrast range can improve the quality of the images for human viewing. By comparison, we can find that Figures 4(b)–4(d) method can smooth the images well while making the images a little burred. EKRM method can preserve all details and enhance the contrast of the images, but it also blows up noise. Our method can overcome these shortcomings and has better performance as shown in Figure 4(f). In the homogeneity region such as the floor and the woman's arms in the image, the homogeneity measure is approximate to zero, and the second part in (3.13) almost has no effect on it, which means the image is smoothed by Gaussian filter. So, the outliers are smoothed out. When the pixels are close to discontinuities, such as the textures parts in the image, the homogeneity measure is approximate to one. The second part in (3.13) plays an important role in image filter. The proposed method becomes an adaptive filter which can enhance the pixel values on the boundary. So, the texture part in Figure 4(f) is more clear than the original image. By comparison, Figure 4(f) is more comfortable for human viewing. Figure 5 is the examples of our method on real noisy color images. Besides the merits on gray images, the proposed method gives output color images of well-increased global and local contrasts.
(a)
(b)
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
By introducing an adaptive parameter, the pixels in homogeneity region are smoothed, and significant discontinuities are preserved and enhanced. The filter performs as much smoothing as possible when it finds continuous region without outliers; when a discontinuity is detected, it averages only the pixels that belong to one region. Compared with other methods, it is noniterative, so computation is more efficient. It can remove blurring and noise, increasing contrast, revealing details, and improving perception of information for human viewers. Our method also has the shortcoming that it is not appropriate for low SNR image, which is our future research work.
5. Conclusion
An adaptive image enhancement method based on kernel regression and local homogeneity is proposed in this paper. It introduces the local homogeneity as an adaptive parameter in EKRM method in the second derivative Taylor expansion and could remove blurring and noise, increase contrast, and reveal details. Experiments results show that our method has better performance than other filter methods.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (Grant nos. 60675013 and 10531030) and the National Basic Research Program of China (973 Program) (Grant no. 2007CB311002).