Abstract
Since the seminar work by Rudin, Osher, and Fatami for total variation (TV) model, piecewise smoothing image is favored in a diversity of related fields. To recover a piecewise smoothing image, we propose a high order anisotropic geometrical model which we name total bending (TB) in a multiplicative strategy. TB model measures the bending degree of the image surface by approximating its second fundamental form. The analysis shows that TB is a rotation version of TV model in moving Frenet–Serret frame: TB norm measures the jumps along the normal direction in an adaptive local coordinate while TV measures the jumps along the vertical direction in a fixed Cartesian coordinate. TB reduces undesired staircase effect by recovering the horizontal and slope surface. The evolution diffusion of TB model inherits TV’s edge preserving ability. The experimental results show the performance of the proposed model quantitatively and visually.
1. Introduction
Image restoration is one of the essential tasks in image processing and computer vision. A digital image degenerated by additive white Gaussian noise can be described by the formulaThe goal of image recovery is to restore a true image from its observation version , assuming the noise level approximately known, i.e., Variational method is an important tool for image restoration. Various methods differ in the way the image is modeled and the construction of the energy functional . First order operator based models, such as TV, can recover a piecewise constant image while performing poor for smoothing transition region recovering [1].
A piecewise smoothing image is suitable for image decomposition. It is favored in a diversity of related fields: computer photography [2], medical image processing [3], speckle noise reduction [4], image Retinex [5], image enhancement [6], and image registration [7]. Therefore, various methods have been proposed to restore such an image. For piecewise linear image recovery, the Hessian of an image contains four second order derivatives and has been exploited to construct high order variational models [8]. Some classical curvature operators, including curvature [9], mean curvature [10], and Gaussian curvature [11], were also used. They relief staircase effect but in practice they have been found to tend to blur edges. In an alternative way, low order and high order operators were combined to construct a hybrid additive model: one part to produce flat image and the other part to generate smoothing transition region. Papafitsoros and Schnlieb considered a general additive high order functional which unified some previous hybrid models [12]. Total generalized variation (TGV) is an efficient high order model with a complex structure [13]. It involves high order derivatives and automatically balances the first to derivatives. The connections between some typical additive high order models are also detailed [12]. Other efficient approaches include spatial weighted average schemes and transform based approaches, such as bilateral filter [14], nonlocal means filter [15], BM3D filter [16], and guided filter [17].
In this paper, an edge preserving high order model is presented to recover a piecewise linear image. Geometrical analysis shows that it is a rotation version of the classical TV model. The rest of this paper is organized as follows. In Section 2, we present total bending (TB) model with a clear geometrical intuition. The analysis on TB is carried out in Section 3. Its associated high order nonlinear equation is also derived and discussed in Section 4. In Section 5, experimental results are shown. A brief conclusion is given in Section 6.
2. Total Bending: A Geometrical High Order Model
2.1. Total Bending Energy
To recover a piecewise linear image, we propose the following total bending (TB) energy functional:where denotes a boundary detector and is norm of second order vector : . Contrary to the additive strategy to organize low order information and high order information [12], a multiplicative strategy is adopted here. The advantage for multiplicative strategy is the requirements for piecewise and smoothing can be expressed by two operators. Two operators are independent of each other and no weight function is needed. In our construction, the piecewise requirement is guaranteed by . The discontinuous location is indicated by with an infinite value near boundary between different flat regions. And for flat region, it approaches to . The smoothing requirement is guaranteed by . Operator measures smoothing degree within a region. Zero values indicate a constant or a affine region and heavy oscillation produces a large value.
2.2. The Geometry behind TB
The interaction between and is key to the energy construction. Next, we will reveal the geometrical intuition behind TB model. The kernel function in (2) has a remarkable geometrical meaning when we consider a gray scale image as a surface in three dimensional space. In differential geometry, the surface bending is described by the second fundamental form , where the surface shape replies on three coefficients , and :When , the surface degenerates into a plane . A special case is a horizontal plane for and it is a constant function . The reason for the staircase effect is the smoothing transition region which is expressed by constant function. Inspired by the corresponding between a function and its induced surface, we approximate the image surface bending by defining total bending (TB) norm as follows:Considering the symmetry with some feasible smoothing assumptions, is expanding as the sum of two crossing coefficients and . Based on the discussions above, the geometrical intuition behind TB model is that it tries to produce a surface with zero bending degree almost everywhere.
3. Analysis of TB Norm
It is difficult to investigate nonlinear high order model in theory. Here, we will make analysis on TB norm for one-dimensional case. The analysis will be carried on from discrete and continuous views. Its relation to other well established models will also be revealed.
3.1. Edge Preserving: A Discrete View
Considering a typical step edge for a signal as shown in Figure 1, discrete in point with central difference and forward difference for . Noting near edges, its TB norm isEquation (5) shows that TB norm is a constant near sharp edges, regardless of the spatial location. Therefore, sharp edges can be kept during the minimizing procedure toward a steady state. Two important facts about TB norm can be inferred: (1) is necessary if one wants to preserve edges; the power to is critical [18] and other choice for will not guarantee good edge preserving ability.
3.2. Linear Smoothing Transition Feature Preserving: A Continuous View
Figure 2 shows four monotone increasing functions defined on interval . According to the TV definition, four different functions have the same value. The piecewise constant function and two quadratic functions produce a nonzero TB norm. Only the linear function produces a zero TB norm. Therefore, linear smoothing transition region will be well detected and preserved by TB norm.
3.3. Relations to Some Related Variational Models: A Rotation Version of TV Model
In Figure 3, we illustrate and compare the geometric intuition of TB norm, TV norm, and MC norm. For point on curve and its nearby point , the tangent vector and normal vector for point span a local coordinate system. TV measures the jump along the vertical direction. Point will move toward until the distance is shorten to zero, if TV vanishing locally. Horizontal straight line is generated and staircase effect occurs. TB measures the distance from point to the axis, which is naturally converted to the distance between point and point . Toward to a stable state, TB pushes point to point against the normal direction. As the tangent direction depends on local structure, the straight line may vary in any direction, also including the horizontal lines for TV as a special case. MC norm describes the curve shape by the radius of the osculating circle indirectly: . Therefore, MC model enlarges to infinity and pushes point to point in an indirect way: moving point , the center of the osculating circle, away from point along the normal direction .
The discussions above reveal two characters of TB norm: TB model is a rotational adaptive TV model in the local coordinates frame, known as Frenet–Serret frame in 2D case; TB and MC share a similar idea for generating a plane but TB achieves this behaviour in a more straight forward way. Furthermore, TB has a more concise and symmetrical explicit structure, which is easy to extend to high dimensional case.
4. TB Associated High Order Nonlinear Diffusion Equation
Using a standard technique for finding minimum value of subject to the noise level constrain, we can yield the following Euler-Lagrange equation and obtain a high order anisotropic nonlinear evolution model:where is a second order divergence operator and denotes a vector composed of four normalized second order derivatives: Therefore, the fourth order diffusion term in direction iswhere is -1 for a negative , 0 for the zero , or +1 for a positive . Reminding the diffusion of TV can also be rewritten asEquations (8) and (9) show the diffusions by TB and TV which are driven by the variation of the sign of derivatives, scaled by a boundary detector. They both allow an adaptive diffusion to the local structure: fast in flat regions and slow near sharp edges. They differ in the order of the derivative and the construction of the boundary detector. Therefore, TB shares a similar mechanism with that of TV. Its discontinuous preserving ability is guaranteed. The same conclusion also holds for other directions.
We multiply (6) by , integrate over , and yield
For a rectangular domain, with , the boundary value condition is
As the evolution equation (6) is nonlinear highly, an explicit finite difference method is adopted to solve it. For time discretion, forward difference is used and the space grid size is set as h = 1. Table 1 lists the scheme for time and spatial operators in high order nonlinear equation. Gradient operator in (6) is discrete with two different strategies. For vector , central difference scheme is used to prevent edge shifting. For gradient amplitude calculation in , minmod scheme is adopted to improve the edge preserving ability.
5. Experimental Results
In this section, we conduct several experiments to demonstrate the performance of the proposed model on signals and images. In every test, additive white Gaussian noise is added to clean data. A reasonable selection of the time space seems to be . To keep the balance of the tradeoff between the time space and number of iterations, we fix in every example (an exception is setting in signal denoising). We make comparisons with three well established variational models: TV [1], MC [10], and TGV [19]. To evaluate the performance of the different methods, we make overview comparisons quantitatively and visually. To do a quantitative comparison, peak signal-to-noise-ratio (PSNR) and structural similarity index (SSIM) are used for quantitative comparison. And for visual comparison, we display the 3D induced mesh or zooming image, by which the geometrical and fine features can be observed vividly.
5.1. Signal Denoising
Figure 4 shows the recovery of a one-dimensional signal to test the staircase effect reduction ability. The original clean signal consists of three constant functions, two linear functions, and a quadratic function, degenerated by Gaussian noise with . The denoising result by TV shows obvious staircase effect for linear and quadratic regions. TB provides a more smoothing effect in linear and quadratic functions, avoiding smoothing edges. The points in two ramps share a common tangent vector, respectively, and their bending degree is small.
(a) Clean signal
(b) Noisy signal
(c) TV method
(d) TB method
5.2. Synthetic Images Denoising
Two synthetic images consisting of four regions are used to test the edge preserving ability for piecewise regions. It will also be verified by observing the induced surfaces vividly from a certain view angle in three-dimensional space. Figures 5 and 6 show the denoised images and surfaces for a synthetic image. It consists of four piecewise constant regions, stained by a heavy noise (). The edges by TV and TB are well preserved than TGV and MC. However, minor fault occurs for TV and TB. Some fine features seem to be blurred slightly in TV result and several isolated pixels are visible near edges in TB result. From the 3D surface profiles, it can also be observed that TB produces a competitive antiblurred result near edges when compared with TV method.
(a) Original image
(b) Noisy image
(c) TV method
(d) TGV method
(e) MC method
(f) TB method
(a) Original surface
(b) Noisy surface
(c) TV surface
(d) TGV surface
(e) MC surface
(f) TB surface
Figures 7 and 8 illustrate the denoised images and their induced surface for another synthetic image. The image is composed of four piecewise smoothing regions: a constant region, a linear region, and two quadratic regions. The noise level is . We focus then on staircase effect suppressing effect and edge preserving ability, especially for weak edges. In TV denoised image, staircase is obvious in linear region and quadratic regions. MC oversmooths peaks and edges heavily. Both TGV and MC reduce staircase effect effectively. TGV creates some isolate concave-convex pixels and some weak edges are blurred. The result by TB shows distinct edges between different regions. A good smoothing transition effect can be observed within linear and quadratic regions.
(a) Original image
(b) Noisy image
(c) TV image
(d) TGV image
(e) MC image
(f) TB result
(a) Original surface
(b) Noisy surface
(c) TV surface
(d) TGV surface
(e) MC surface
(f) TB surface
5.3. Natural Images Denoising
Figure 9 shows the denoising results for satellite image. This image contains a white satellite with black background. It is a challenge to recover some white fine objects from the noisy image stained by additive white Gaussian noise with . To show fine details, we illustrate the zooming of a local region, as shown in Figure 10. Some fine line features are blurred by TGV and even missing in MC result, while they are well preserved by TV and TB. For the vertical line, TV and TB perform well. TB performs better for the slope line preservation. The reason for this behaviour is that TB contains two crossing derivatives, which has a good ability to sense the variations in slope directions.
(a) Original image
(b) Noisy image
(c) TV result
(d) TGV result
(e) MC result
(f) TB result
(a) Original image
(b) Noisy image
(c) TV result
(d) TGV result
(e) MC result
(f) TB result
In Figure 11, we display the denoising results for the Pepper image with noise level . The proposed method recoveries smoothing and shining peppers while TV and TGV produce hazy ones. The oscillations occur in pepper boundary in MC result, especially in region marked by red rectangle. Seldom oscillations can be found in such locations by TB and TV. Two regions across two peppers are clipped. One is marked by green rectangle and another is by red. Their zooming 3D meshes are also displayed in Figures 12 and 13, respectively. The mesh in two figures shows the different kinds of regions, separated by discontinuous jumps. The edges are well protected by TB and TV. TB shows a strong desire to produce linear region, especially in the right side of Figure 12. However, in Figure 13, TB seems to be oversmoothing slightly in the bottom of the valley. Its favor of linear region coincides with the geometrical intuition of the TB norm.
(a) Original image
(b) Noisy image
(c) TV image
(d) TGV image
(e) MC image
(f) TB result
(a) Original image
(b) Noisy image
(c) TV image
(d) TGV image
(e) MC image
(f) TB result
(a) Original image
(b) Noisy image
(c) TV image
(d) TGV image
(e) MC image
(f) TB result
In Tables 2 and 3, PSNR and SSIM indexes for different models are reported. It can be observed that TB performs better on PSNR and SSIM values for several test images. TB performs better for two synthetic images. It shows that TB’s performance relies on the image structures. Ideal piecewise smoothing region will promise a high index value.
6. Conclusion
The low order variational framework has been well established and many models have good ability of edge preserving. However, this ability has not been inherited by high order models successfully. In this paper, we try to solve this problem by introducing TB model via a multiplicative strategy for organizing second order derivatives and first order derivatives. The low order gradient information is used to detect and protect sharp edges, while the second order information is used to control smoothing degree. The analysis shows that TB is an adaptive rotational version of the classical TV model in local coordinate system. TB measures surface bending degree and it has a clear geometrical intuition. The construction of TB model strengthens the connection between variational image processing and the differential geometry. The first fundamental form describing the arc length was used to construct low order variational model with area minimizing. The second fundamental form describing the bending degree is used to construct high order models via bending degree minimizing, as presented in this paper. However, TB model still needs some improvements in optimization and theory. As a nonconvex functional, how to devise a fast optimization numerical scheme for TB will be focused in future. Lacking of the maximum principal, the theoretical analysis on TB model is another challenge.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This study was supported by NSFC (U1404103) and Key Scientific and Technological Research Projects of Henan Province Education Department (14A520029).