Abstract
Circular histogram thresholding is a novel color image segmentation method, which makes full use of the hue component color information of the image, so that the desired target can be better separated from the background. Maximum entropy thresholding on circular histogram is one of the exist circular histogram thresholding methods. However, this method needs to search for a pair of optimal thresholds on the circular histogram of two-class thresholding in an exhaustive way, and its running time is even longer than that of the existing circular histogram thresholding based on the Otsu criteria, so the segmentation efficiency is extremely low, and the real-time application cannot be realized. In order to solve this problem, a recursive algorithm of maximum entropy thresholding on circular histogram is proposed. Moreover, the recursive algorithm is extended to the case of multiclass thresholding. A large number of experimental results show that the proposed recursive algorithms are more efficient than brute force and the existing circular histogram thresholding based on the Otsu criteria.
1. Introduction
Image segmentation technology plays a crucial role in the application of image processing, and the effect after segmentation will directly affect the work in the later stage of computer vision. The original segmentation technology is to convert the image into grayscale image and then use various algorithms to segment it [1]. However, with the development of digital information technology, this way has not been able to meet the needs. Therefore, many researchers began to turn their attention to color image segmentation technology [2].
The color image provides more information than the gray image. To segment the color image, researchers should not only select the appropriate color space but also adopt the segmentation algorithm suitable for this space. Among the many color spaces, RGB (i.e., red green blue) color space [3] is the most commonly used one, because the colors perceived by the human eye are a mixture of red (R), green (G), and blue (B), which are commonly referred to as the three primary colors. However, RGB is only suitable for the display system, but not for image segmentation and analysis. Because the R, G, and B components are highly correlated, that is, as long as the brightness changes, the three components will change accordingly. On the contrary, the HSI (i.e., hue saturation intensity) color space model [4] overcomes the defect of the coupling of the two in the universal RGB color model by representing the color attribute and the light intensity, respectively, by the three mutually independent components hue (H), saturation (S), and intensity (I).
The three basic components of the HSI color space model are hue (H), saturation (S), and intensity (I). HSI matches the color perception of human eyes, which is particularly useful in some situations with uneven illumination, so many researchers use the HSI color space model in image segmentation [5]. Where the H component is a reflection of color “quality,” each color has the corresponding wavelength and the corresponding hue. Because the hue is independent of the highlight and shadow, the hue is very effective to distinguish objects with different colors. Therefore, the segmentation of the image can achieve the desired results by making full use of the hue component information of the color image.
Thresholding of H component was firstly studied by Tseng et al. [6], who firstly constructed a circular histogram by hue for the image, then transformed it into a traditional histogram after smooth filtering, and finally used the principle of maximum variance for recursive thresholding. His results showed that circular histogram produces better segmentation results than traditional histogram. Subsequently, Wu et al. [7] used the iterative Otsu algorithm on the circular histogram to segment the white blood cell image, which was proved to be effective by experiments. Dimov and Laskov [8] divided the circular histogram into two classes using the Otsu method and expanded it to divide the image into any number of classes. Lai and Rosin [9] discovered a unique property of circular histogram thresholding based on the Otsu criteria and an efficient Otsu-based circular histogram thresholding algorithm was given, which takes time, where is the number of the histogram bins, and applied it to optical flow data, indoor/outdoor image classification, and nonphotorealistic rendering.
Entropy was initially extended to the information field by Shannon [10], who defined the information gain of an event as the negative of the logarithm of the occurrence probability of the event. Entropy reflects the information content of symbols that are independent of any particular probability model. Image analysis adopts the concept of entropy in the sense of information theory, and entropy is used to quantify the minimum description complexity for smooth regions [11]. Since entropy can provide a good level of information to describe a given image, the entropy of the histogram distribution can be calculated and an appropriate segmented image can be obtained. Therefore, the entropy method has been widely used in image segmentation, which does not need any prior information, and the optimal threshold value for image segmentation can be selected where the maximum entropy corresponding to the probability distribution of target composition and background composition is known as the maximum entropy thresholding [12].
Recently, Kang et al. [13, 14] proposed two ways on circular histogram thresholding based on the maximum entropy principle. In the first way, the circular histogram was converted into a linear histogram at the appropriate breakpoint using Lorenz curve, and then the traditional maximum entropy thresholding method was used to segment. In the second way, the maximum entropy expression was given directly on circular histogram. Compared with the first way, the second way does not require auxiliary breakpoint selection information, but its exhaustive search brings a large amount of computation.
It can be found from the literature [13, 14] that the entropy-based method on circular histogram may be much better than the Otsu-based method. However, separating the circular histogram into two classes needs two thresholds, so it takes time to use maximum entropy thresholding on circular histogram. This is an order of magnitude more than the time of the traditional linear histogram binarization with maximum entropy and the efficient circular histogram thresholding method based on the Otsu algorithm in the literature [9]. Moreover, if maximum entropy thresholding on circular histogram is extended to classes , the computational complexity of brute force will increase to . It can be seen that the maximum entropy thresholding has low segmentation efficiency on the circular histogram, and it has limitations in real-time. Therefore, it is necessary to find a fast calculation method for maximum entropy thresholding on circular histogram. In this paper, a recursive algorithm for the maximum entropy thresholding on circular histogram of two-class thresholding is proposed and also extended to the case of multiclass thresholding.
The remainder of this paper is organized as follows: in Section 2, the criterion for selecting the optimal threshold pairs for the maximum entropy thresholding on the circular histogram is described, then it is simplified according to the symmetry, and a recursive formula for the maximum entropy thresholding on circular histogram is derived. In Section 3, the maximum entropy thresholding on circular histogram is extended to the case of multiclass thresholding, its threshold selection criterion is given, and then a recursive formula is derived as well. Experimental results and discussions are shown in Section 4. Section 5 concludes this paper.
2. Maximum Entropy Thresholding on Circular Histogram
Initially, Pun [15, 16] introduced entropy into the image for thresholding. Kapur et al. [12] pointed out some errors of Pun and rederived it and then proposed a new image thresholding algorithm based on entropy, namely, maximum entropy thresholding. The maximum entropy is that when it is necessary to predict the probability distribution of a random event, the prediction should satisfy all the known conditions and not make any subjective assumptions about the unknown situation. When the information entropy of probability distribution is the largest, the probability distribution is the most uniform and the prediction risk is the least. This method is more global and universal in image thresholding [1]. Besides histogram, no prior knowledge of images is needed.
The HSI color space well matches the way that humans describe and interpret colors. The hue in HSI space describes the color attribute of pure color (pure red, yellow, or green). Figure 1 shows the circular plan of hue in HSI color space. On this plane, it can be seen that the color changes with the change of angle, and the direction of change of chroma is anticlockwise. The hue of a point is determined by the angle between the line from the point to the center of the circle and the horizontal line. Obviously, the range of this angle is . Usually, means red, means green, and means blue. Because the hue component is closely related to the way of human perception of color, it is very suitable for the analysis and processing of color images.
As can be seen from Figure 1, the hue component of a color image is a natural circle, so it has some different characteristics: periodicity and continuity. Then, when the hue component is processed, it will be different from the traditional linear histogram, and the statistical method will also be changed to circular statistics [17]. However, using entropy to segment an image only needs to know the probability distribution of the histogram and does not need to calculate the numerical characteristics such as mean and variance. Now, a circular histogram is assumed, with an element index from 0 to , that is, . Because of the periodicity of the circular histogram, element 0 must be adjacent to element . In order to unify the way of representation, the definitions in the literature [9] are referred to as follows: and are two points on the circular histogram, respectively, so the part from to on the circular histogram is represented as . If , then it is equivalent to . And circular sum is defined as , which is equivalent to
Moreover, the threshold selection of circular histogram is different from that of traditional linear histogram. Threshold pair is required in circular histogram to divide the histogram into two parts. The first threshold can be considered as an appropriate starting point, and the other threshold is responsible for dividing the two parts and .
For convenience, the circular maximum entropy thresholding is used to refer to the maximum entropy thresholding on the circular histogram. In this section, first of all, the selection criterion of the optimal threshold pair of circular maximum entropy thresholding will be introduced, that is, the general form of circular maximum entropy thresholding. Then, according to the symmetry of the circle, it can be found that the general expression of circular maximum entropy thresholding also has symmetry, which can reduce the workload by half compared with the brute force method. At the end of this section, a recursive algorithm of circular maximum entropy thresholding is proposed.
2.1. The Selection Criterion of the Optimal Threshold Pair
For an image with size and hue value , the hue value at point is represented by . The number of pixels in the hue value of the image is denoted as , so the number of pixels in the entire image is . is used to represent the frequency of hue value .
For a circular histogram , without any prior information, a threshold pair is used to divide the circular histogram into two parts. Then, the threshold pair can be selected at points where the entropy corresponding to the probability distribution of the two parts is maximized, so as to make the probability distribution of the two parts as uniform as possible.
Suppose that the circular histogram is separated in the threshold pair (note: the direction of anticlockwise here, , , where the symbol represents the anticlockwise order), two subprobability distributions can be formed: , .
Note that is the sum of anticlockwise order (i.e., the circular sum introduced in Section 2), then
Therefore, the Shannon entropy of these two probability distributions can be expressed as
Then, the selection criterion of the optimal threshold pair of circular maximum entropy thresholding is [13]
2.2. Symmetry of the Selection Criterion
Because the hue histogram is a natural circle, it has a very important property that is symmetry. Therefore, the discussion is as follows. For equations (2) and (3), the following equation can be obtained:
Similarly, there are .
For Shannon entropy expression equations (4) and (5) and the above equation (7), the following equations can be obtained:
Therefore, the following can be obtained:
From the above equation, it can be seen that the circular maximum entropy summation expression also has symmetry. Since the linear order relationship between and is only or , it is only necessary to search the range on the circular histogram, which can reduce the computation by half. Therefore, the selection criterion of the optimal threshold pair can be changed to the following equation:
2.3. Recursive Algorithm
Equation (4) can be derived as follows:where .
Similarly, equation (5) can be derived as follows:where and .
Then, the following can be obtained:
The following is the specific processes of recursion formula generation, denoting , firstly:(1)When ,. When ,. When , .(2)When ,. When ,. When , .(3)When ,(4)When ,
Based on the above statements, the recursion formula of circular maximum entropy thresholding can be obtained as follows:
When using the recursive algorithm, the calculation time can be saved greatly. If the recursive algorithm is not used for calculation, it can be seen from equation (14) that each cycle is repeated to calculate and , which will increase the computational complexity of the algorithm. In order to avoid unnecessary repeated calculation, two new storage spaces are created, and , and and are converted into the expressions of and . In this way, when calculating and , the corresponding calculation results can be obtained directly from the two storage spaces and quickly get the values of and , which greatly reduces the calculation complexity.
3. Multiclass Thresholding of Circular Maximum Entropy
For some images, in order to meet the requirements, they need to be divided into multiple parts, namely, the circular histogram is divided into multiple classes. In this section, the circular maximum entropy thresholding is extended to the case of multiclass thresholding; that is, the circular histogram is divided into parts . In the same way, the general form of threshold selection criterion is given, and then the recursive algorithm is proposed.
3.1. The Selection Criterion of Optimal Thresholds
For a circular histogram , without any prior information, thresholds are used to divide the circular histogram into parts. Then, thresholds can be selected at points where the entropy corresponding to the probability distribution of the parts are maximized, so as to make the probability distribution of every parts as uniform as possible.
Considering the symmetry of the expression of circular maximum entropy, it is possible to assume that , so that 0 will be included in the . Then,
Therefore, the Shannon entropy of every probability distributions can be expressed as
The selection criterion of the optimal thresholds in the case of multiclass thresholding of the circular maximum entropy is as follows:
3.2. Recursive Algorithm
Equation (22) can be derived as follows:
Similarly, equation (23) can be derived as follows:
The following is the specific processes of recursion formula generation, still denoting ,:(1)When ,(2)When ,(3)When ,(4)When ,
Based on the above statements, the recursion formula of circular maximum entropy in the case of multiclass thresholding can be obtained as follows:
The same as the recursion formula of two-class thresholding, two new storage spaces and are created firstly, and then and are converted to the expression of to avoid duplicate cumulative calculations. Thus, the Shannon entropy expressions and for the probability distribution of each part are also converted to the expressions of and , so that the corresponding values can be called directly from the two storage spaces and in the calculation. The calculation results of and are obtained quickly.
4. Experimental Results and Discussion
In order to compare the operational efficiency and segmentation results of various algorithms and the proposed recursive algorithm, all experiments were tested on a personal computer with MATLAB 9.3.0 (R2017b).
The experiment is mainly divided into three parts. The first part is to compare the running time of different calculation methods of circular maximum entropy thresholding under the condition of circular histogram divided into two parts: brute force method, brute force method considering symmetry, and the proposed recursive algorithm. In order to prove that the proposed recursive algorithm is efficient, the running time of the circular Otsu thresholding model in the literature [9] and recursive algorithm of maximum entropy thresholding for circular histogram linearization are also tested. Then, the segmentation results of various algorithms will be compared in the second part. In the third part, for the circular histogram of test images, the running time of the brute force method considering symmetry with that of the proposed recursive algorithm will be compared under the condition of multiclass thresholding.
All of test images are from the Berkeley database, and Figures 2(a)–2(h) in Figure 2 show the original color image in RGB color space. For our test, all images need to be converted to the HSI color space for display, and Figures 3(a)–3(h) in Figure 3 show the corresponding test image in Figure 2 in HSI color space.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
The H component of each image is extracted in the HSI color space, and then a circular histogram is drawn. The circular hue histogram of each image is shown in Figure 4, where Figures 4(a)–4(h) in the figure correspond to Figures 3(a)–3(h) in Figure 3.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
The running time (in seconds) of each test image using different algorithms is recorded in Table 1, including circular maximum entropy thresholding brute force method, brute force considering symmetry, and recursive algorithm proposed. For comparison, the running time of using the circular Otsu thresholding model in [9] for segmentation will also be recorded. The running time of the recursive algorithm of maximum entropy thresholding for circular histogram linearization is also recorded. Here, two different methods will be used to linearize the circular histogram: one is to traverse the breakpoints in an exhaustive way and then use the recursive algorithm of the traditional maximum entropy thresholding for the linearized histogram, and the other is to use Kang et al.’s [13] Lorenz curve method to find the optimal breakpoint, then break the circular histogram at that point, and then use the recursive algorithm of traditional maximum entropy thresholding. More intuitive to show their differences, the average running time of each method in these eight images will be calculated, and the bar chart is shown in Figure 5. For ease of representation in charts and tables, method 1 refers to the brute force method of circular maximum entropy, method 2 refers to the brute force method of the circular maximum entropy thresholding considering symmetry, method 3 refers to the circular Otsu thresholding method [9], method 4 refers to a recursive algorithm that traverses breakpoints on a circular histogram and then uses the traditional maximum entropy thresholding for each linearized histogram, method 5 refers to the recursive algorithm of using the traditional maximum entropy thresholding for linear histogram after the circular histogram is broken by Kang et al.’s [13], and method 6 refers to the proposed recursive algorithm of circular maximum entropy.
From Table 1 and Figure 5, it can be seen that the efficiency of the circular maximum entropy thresholding using the brute force method considering symmetry is more than twice that of the brute force method. When the proposed recursive algorithm of circular maximum entropy thresholding is used to segment images, the segmentation efficiency is nearly 335 times that of the brute force method and is almost 145 times that of the brute force method considering symmetry. Although the circular Otsu thresholding is efficient, it can be seen that the proposed recursive algorithm of circular maximum entropy thresholding efficiency is almost 5 times that of circular Otsu thresholding. And the recursive algorithm of circular maximum entropy is more efficient than the two circular histogram linearization maximum entropy thresholding. This is sufficient to demonstrate the efficiency of the proposed recursive algorithm of circular maximum entropy. Therefore, the recursive algorithm can be used to replace the brute force method in image segmentation by circular maximum entropy thresholding.
Next, the segmentation results of various algorithms will be compared. Since the proposed recursive algorithm rededuces the formula of the brute force method to improve the segmentation efficiency of the circular maximum entropy thresholding, the segmentation results obtained by methods 1 and 2 are the same as those obtained by the recursive algorithm proposed by us, so no comparison is made here. Figures 6 to 13 show the segmentation results of different algorithms corresponding to the eight test images, Figures 3(a)–3(h) in Figure 3. In each set of images, (a) is the ground truth, (b) is the segmentation result of method 3, (c) is the segmentation result of method 4, (d) is the segmentation result of method 5, and (e) is the segmentation result of method 6, which is the proposed recursive algorithm.
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
It can be seen from the segmentation results in Figure 6 that Figure 6(b) is very close to the ground truth, but there are some defects in the tail and mouth of the parrot. Figures 6(c) and 6(e) are exactly the same, and the parrot and a small part of the fence are divided into the same class, so their effect is relatively poor, while a large part of the parrot in Figure 6(d) is misclassified. The Figures 7(b), 7(c), and 7(e) in Figure 7 are similar, and they are close to the ground truth, while Figure 7(d) is obviously poor. It can be seen that Figures 8(b) and 8(d) in Figure 8 are the same, and in addition to the target, there are a lot of redundant information included in the segmentation results. Figures 8(c) and 8(e) are the same, but there is only a small amount of redundant information in the segmentation results except for the target. Figure 9(b) is the closest to the ground truth, but the target is still incomplete. Figures 9(c) and 9(e) are the same, and their target also contains some redundant information. And Figure 9(d) is obviously the worst of all. The four segmentation results in Figure 10 are not significantly different. It can be seen that Figures 11(b), 11(c), and 11(e) in Figure 11 are similar, and they are all better than Figure 11(d). Each of the segmentation results in Figure 12 can almost segment the airplane well, but there is some redundant information in Figure 12(d). Figures 13(c) and 13(e) are the same, and their effects are better than those of Figures 13(b) and 13(d).
To sum up, it can be seen from these eight groups of segmentation results that method 4 has the same results as the proposed recursive algorithm, while the results of method 5 are slightly inferior to that of the proposed recursive algorithm, while the results of method 3 are close to that of the proposed recursive algorithm. In order to see the results more accurately, the pixel accuracy value [18] of each segmentation result will be calculated separately, and the average value of the eight segmentation results according to different methods will be calculated, which are recorded in Table 2.
It can be seen from Table 2 that the pixel accuracy value obtained by method 4 is the same as that obtained by the algorithm in this paper, and the pixel accuracy value obtained by the algorithm in this paper is significantly better than that obtained by method 5. For a small part of test images, the pixel accuracy value obtained by this algorithm is slightly lower than that of method 3, but it can be seen from the average value that this algorithm is still superior to method 3.
The circular maximum entropy thresholding will be extended to the case of multiclass thresholding for experiments, and the brute force method considering symmetry and the recursive algorithm will be used to test and their running time (in seconds) will be compared. To observe the images in Figure 3 and the corresponding circular histograms in Figure 4, it can be found that circular histograms in Figures 4(d) and 4(g) are the bimodal, the circular histograms in Figures 4(a)–4(c), and 4(h) are roughly three-peak, and the circular histograms in Figures 4(e) and 4(f) are roughly three-peak or four-peak. Table 3 shows the running time of images segmentation into three classes by different methods of circular maximum entropy thresholding. The test images are Figures 2(a)–2(c), 2(e), 2(f), and 2(h) in Figure 2, and the last row of Table 3 shows the average time of segmentation of six images by different methods. Table 4 shows the running time of image segmentation into four classes by different methods of circular maximum entropy thresholding. The test images are Figures 2(e) and 2(f) in Figure 2, and the last row of Table 4 also shows the average time of segmentation of two images by different methods.
It can be seen from Table 3 that if circular maximum entropy thresholding is extended to three-class thresholding, the recursive algorithm proposed still has a high efficiency, which is more than 85 times that of the brute force method. As can be seen from Table 4, if circular maximum entropy thresholding is extended to four-class thresholding, the brute force method has a high computational complexity and low efficiency. It takes an average of 9145.1474 seconds to segment test images, Figures 2(e) and 2(f). However, the segmentation efficiency of the recursive algorithm is 66 times that of the brute force method.
From the above experimental results, it can be seen that when circular maximum entropy thresholding is used to binarize color images, the recursive algorithm is much more efficient than the brute force method and brute force considering symmetry, and its running time is even shorter than the efficient circular thresholding model proposed in [9], which can completely realize real-time image processing. And the segmentation results of the recursive algorithm are the same or better than all the other algorithms compared. After the circular maximum entropy thresholding is extended to the case of multiclass thresholding, the computational complexity of the brute force method will increase exponentially. Although the complexity of recursive algorithm will also increase, it is still much more efficient than the brute force method. Therefore, the proposed recursive algorithm is recommended for circular maximum entropy thresholding.
5. Conclusion
In this paper, aiming at the problem of high computational complexity and low segmentation efficiency when using circular maximum entropy thresholding to segment color images, a recursive algorithm of circular maximum entropy thresholding is proposed. On the premise that it can obtain the same segmentation results as the brute force method, the recursive algorithm will have less running time. In terms of segmentation efficiency, the proposed recursive algorithm with the existing efficient circular Otsu thresholding algorithm was compared. Obviously, the proposed recursive algorithm is more efficient, and its efficiency is 5 times that of the efficient circular Otsu thresholding algorithm. Moreover, it can be seen from the segmentation results that the recursive algorithm is slightly better than the circular Otsu thresholding algorithm. Subsequently, the algorithm was extended and the recursive algorithm of circular maximum entropy thresholding in multiclass thresholding cases was put forward. Experiments showed that recursive algorithm is more efficient than the brute force method. Therefore, when using circular maximum entropy thresholding for color image segmentation, it is recommended to adopt the proposed recursive algorithm, which can save a lot of time. At the same time, this algorithm also provides a good foundation for improving the segmentation efficiency of circular histogram thresholding in the future.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (no. 62071378) and New Star Team of Xi’an University of Posts and Telecommunications (xyt2016-01).