Abstract
In recent years, the incidence of diabetes has been increasing year by year. Since most of the fundus lesions are located near blood vessels, the image information is complex, and the end vessels are difficult to identify. So, a new segmentation method of diabetic retinal vessel images based on particle swarm optimization and salp swarm algorithm is proposed. This paper uses a Gaussian filter to enhance the main blood vessels, and a top-bot hat transform is used to strengthen the end vessels. The preprocessing process is completed by combining and reconstructing the two images through a normalization operation. The improved particle swarm optimization and salp swarm algorithms perform multi-threshold segmentation on the preprocessed vessel images. The best fit value, Structural Similarity Index Measure, Peak Signal to Noise Rati, feature similarity index measure, sensitivity, accuracy, regional consistency, Dice coefficient, Jaccard similarity, and Shannon entropy are selected for comprehensive evaluation and analysis. The results showed that this paper’s improved particle swarm-salp swarm algorithm for segmenting diabetic retinal blood vessel images is more efficient, and the threshold is better. The vascular segmentation method in this paper is applied in medical image processing, which improves the accuracy of medical image processing and reduces the computational effort.
1. Introduction
Diabetic retina (DR) is one of the four significant major cures for blindness worldwide. Studies have shown that the rate of advanced retinopathy blindness in China and India has been as high as 37.4% [1]. The blood vessels inside the eye are densely distributed. Therefore, eye lesions in most diabetic patients are associated with blood vessels.
The segmentation of DR vascular images is relevant to the doctors’ diagnosis of the grade of retinopathy in diabetic patients and the subsequent treatment. Current fundus vascular segmentation methods can be divided into two main categories: supervised learning and unsupervised learning. Supervised learning often requires an extensive training set, where a training model is learned several times based on known images of the vascular pixel class, and the resulting model is then used to split DR vascular images of unknown vascular pixel classes. To make the segmentation of fundus vessels more transparent, many scholars have proposed many neural network-based vessel segmentation methods. Xie proposed a method for segmenting fundus vessels using deep learning to achieve automatic extraction of vessel features and improve the accuracy of the algorithm [2]. Xue et al. proposed an improved U pattern network segmentation method for DR images, which improved the unclear situation of low-contrast vascular images [3]. The above cases are all supervised learning ways and have high data requirements, while unsupervised learning split ways have lower data requirements. Unsupervised learning does not require an extensive training set, making it easier to achieve with lower hardware requirements for the operating platform. Unsupervised learning focuses on segmenting images according to the distribution pattern of vascular image pixels. Li et al. proposed method is matching filtering of fundus vessels using line operators to make the vessel pixels more prominent and Fan Linlin's segmentation method is based on matching filtering to preprocess fundus vessel images and then doing erosion and expansion on the resulting images, which improves the accuracy of the algorithms segmentation to a certain extent [4, 5]. Fan proposed method of improving the accuracy of fundus vascular image segmentation using a combination of minimum wave transform and Gaussian filtering [6]. Along with the development of artificial intelligence, image processing technology has become a popular research direction. Rajinikanth et al.proposed the FCI evaluation procedure using the Spider Monkey Optimization Algorithm (SMOA) [7].
Traditional threshold segmentation methods are unsupervised learning and are mainly based on the OTSU algorithm, the maximum variance between classes criterion for splitting images. The single-threshold OTSU segmentation method is easy to implement when dealing with images with simple information. However, due to the dense distribution of blood vessels in the fundus and the redundant data, the single-threshold segmentation method does not meet the project needs, so the single-threshold OTSU segmentation method needs to be extended to a multi-threshold field. When segmenting DR vascular images based on multi-threshold, OTSU is computationally complex and not easy to implement clinically. To address this problem, many scholars have proposed using population optimization algorithms to solve multi-threshold problems. Nasiri proposed a method for multi-threshold segmentation of fundus vascular images using the Whale Optimization Algorithm (WOA), making multi-threshold segmentation of fundus vascular images easier to implement in clinical practice [8]. Arnay uses Ant Colony Optimization (ACO) to segment fundus images with multi-threshold, and ACO’s unique pheromone update method improves the segmentation accuracy of fundus vascular images [9]. Alrawi and Karajeh use Genetic Algorithm (GA) in the evolutionary algorithm to segment the matched filtered fundus image of DR after preserving more details of blood vessels [10]. Rajinikanth et al. proposed a machine-learning system (MLS) to detect the AMD using the fundus-retinal-images [11]. The unique optimization approach of the population optimization algorithm makes it more suitable for solving complex problems such as multi-threshold segmentation of DR fundus vascular images.
The main focus of this paper is to preprocess DR fundus vascular images using Gaussian matched filtering (GMF) and top-bot hat transforms, respectively, and then combine the two resulting images. Particle swarm optimization (PSO) is used to generate the parameters of the multi-salp swarm algorithm (MSSA) based on the discrete criterion to obtain the improved particle-swarm-multi-salp swarm algorithm (PMSSA). Finally, the multi-threshold segmentation of the pretreated DR fundus vascular image after the improved PMSSA is used. The improved PMSSA segmentation of DR fundus vascular images has higher accuracy, sensitivity, and specificity. The algorithm has better convergence performance and is less likely to fall into a local optimum. To evaluate the performance of the improved PMSSA multi-threshold segmentation of DR fundus vascular images more comprehensively, a comprehensive analysis of PMSSA and same category population optimization algorithms, WOA, Social Spider Optimization (SSO), PSO, and MSSA in terms of appropriate optimal stress values Structural Similarity Index Measure (SSIM), Feature Similarity Index Measure (FSIM), Peak Signal to Noise Ratio (PSNR), Dice coefficient(Dice), Jaccard similarity coefficient (Jaccard), sensitivity, accuracy, Shannon entropy, and regional consistency was done. The results show that the DR fundus vascular image segmentation method in this paper is more accurate and retains more vascular details. The PMSSA has better iterative performance and higher stability.
The contributions of the present work are summarized as follows:(1)In this paper, we propose a segmentation method of diabetic retinal vessel images based on particle swarm optimization and salp swarm algorithm.(2)Enhanced end-vessel based on matched filtering with top-bot hat transform.(3)Multi-threshold segmentation of preprocessed DR fundus vascular images using the improved PMSSA, comprehensive analysis of the performance of the algorithm.
The main purpose of this paper is to segment the DR fundus vascular images more efficiently and accurately, retain more vascular details, and facilitate physicians’ doctors to diagnose the level and diagnosis of retinopathy in diabetic patients.
2. Related Work
The fundus is rich in vascular tissue, and most DR fundus lesions occur near the vascular tissue, so the extraction of retinal vessels is very important in the process of DR fundus diagnosis. Recent work on retinal vascular extraction is mostly based on color fundus imaging, and RGB color fundus imaging is the only noninvasive, direct, and nondestructive way to visualize vascular tissue. Due to individual differences and different imaging environments, the differences in brightness and color of RGB images make it difficult to segment the end blood vessels in RGB images, and a variety of blood vessel segmentation methods have been developed to solve this difficulty.
In the Retinopathy Online Challenge database, Hatanaka et al. proposed a method with high generalization capability based on three microaneurysm detectors for automated microaneurysm detection [12]. Wen et al. proposed the use of principal component analysis and machine learning methods to detect microaneurysms and improve the performance of detection [13]. Dai et al. proposed an automatic detection method for MAs in color fundus images based on gradient vector analysis and class imbalance classification, which improved the sensitivity of detection [14]. Kadry et al. proposed a Computer-Aided-Procedure (CAP) to extract the blood vessel [15].
In summary, the segmentation of retinal vessel images has achieved certain results, but there is still room to improve the segmentation accuracy of retinal vessel images. Therefore, a vessel extraction method based on the improved particle swarm optimization and salp swarm algorithm is proposed in this paper to improve the accuracy of DR vessel segmentation.
3. Materials and Methods
3.1. Image Preprocessing
The RGB channel extraction operation was done on the retinal images in the given DRIVE dataset to obtain the red channel, green channel, and blue channel, respectively, and the results are shown in Figure 1. For the color original image (a), the R, G, and B channel histogram values are shown in Figure 2. Studies have shown that the vascular information is most apparent in the green channel of DR fundus vascular images, so all studies in this paper were carried out on green channel images.
3.1.1. Gaussian Match Filter
The experiments in this paper are based on GMF enhancement of the vascular portion of the green channel images [16]:where is the retinal vessel scale, is the vessel segment length, and is the direction of filter growth in the vessel. The matched filtering results are shown in Figure 3.
(a)
(b)
(c)
(d)
(e)
(f)
3.1.2. Top-Bot Hat Transformation
Top-bot hat operations do open and close operations on DR images, respectively. The top-hat process amplifies and preserves the high-frequency part of the image, while the bot-hat process preserves the low-frequency portion of the image. The closed process involves expansion followed by erosion and the open process involves erosion followed by expansion, neither of which changes the image size [17]. The specific definitions are as follows:
The difference operations between the top-bot hat of the DR image make the pixels belonging to the ends of the tiny vessels in the DR vessel image more prominent, defined as follows:where is the open process on the image, is the closed process on the image, and is the structural element of the open and closed processes. Figure 4 shows the results of a top-bot hat transformation performed on the DR vessel images.
4. Segmentation of Retinal Vessels Based on PMSSA Multi-Threshold Algorithm
Salp is an almost transparent marine invertebrate. In 2017, Mirjalili proposed a population optimization algorithm, MSSA, based on the foraging behavior of salp, which has features such as fast convergence and adjustable exploration and exploitation ratios [18].
In the mathematical model of the algorithm, the chain of salps is mainly divided into leaders and followers. Leaders update their position in a certain number of steps based on the current position of the food, and followers’ positions are updated only about their predecessor’s salp. In practice, the salp algorithm is used to explore n-dimensional space (n is the number of solutions) to store the positions of the salp group in a two-dimensional matrix X, the salp with the best fitness function is selected as food. The formula for updating the position of the salp is as follows.
Leaders update their positions according to formula (4) and followers update their positions according to formula (5). is the bandwidth of the salp exploration space. ， ， and are random numbers uniformly distributed between [0, 1]. In practice, determines the direction of advancement of the salp; determines the length of the advance of the salp; and determines the weight of the salp influenced by food F. This means that the exploration and development ratio of the salp algorithm is adjustable, and to a certain extent it can avoid falling into a local optimum solution:where is the number of current iterations and is the number of total iterations.
4.1. OTSU Algorithm
The OTSU algorithm finds the optimal threshold in an image based on the maximum interclass variance. The image is partitioned into different pixel classes according to a particular rule, where the variance between individual pixel classes is the greatest [19]. The formula is as follows:
The maximum variance between classes is as follows:where is the percentage of greyscale value m; represents the foreground of the image; represents the background of the image.
The maximum variance between classes for multiple thresholds (K categories) is as follows:where is the average grey level of the image.
4.2. Particle Swarm Algorithm
The PSO algorithm is a classical population optimization algorithm proposed in 1995, which is fast in iterations and has better global than local optimization seeking ability [20]. The particle swarm position update equation is as follows:where is the weight of , and and are the individual optimum and group optimum, respectively.
Since in the salp algorithm is a set of random numbers located between [0, 1], which makes the algorithm less efficient, while the PSO algorithm has a fast iteration rate and the exploration ability is superior to local development ability, which means it can get the optimal solution at the early stage of solving simple problems, so the PSO algorithm is used to generate of the Bottlenose Sea Sheath algorithm, thus bringing the improved PMSSA, which enhances the stability of the algorithm so that it has a higher probability of jumping out of the local optimum in the iterative process.
4.3. Improved PSO-Based Salp Multi-Threshold Algorithm for Segmenting Retinal Vessels
The single-salp algorithm (SSA) has fast convergence and adjustable exploration and exploitation ratios, and it is suitable for dealing with low-dimensional problems. is the crucial parameter that determines the distance of the salp from the food location . Traditional MSSA when dealing with multidimensional issues, where is a set of random numbers in [0, 1], which makes the algorithm’s performance in dealing with high dimensional issues and segmenting DR vessel images, MSSA is not superior, low exploration efficiency, and there is a possibility that the exploration of some salp individual is unnecessary. In contrast, the PSO algorithm has the characteristics of fast iteration and global search is superior to local search, so in this paper, PSO is used to generate a set of random arrays fixed at the current number of iterations and is distributed between [0,1] according to the discrete criterion. This allows PMSSA to explore all directions uniformly around the current food during multi-threshold segmentation of DR fundus vascular images, making it less likely to fall into local optima where the PSO algorithm generates with the following fitness function:where is the number of leaders in the PMSSA algorithm. The location update formula for the improved PMSSA algorithm is as follows:
The flow chart of the algorithm is shown in Figure 5. In this paper, the DR vessel images are processed using GMF and top-bot hat transform, respectively, and then combining the two. The results of the preprocessed DR vessel images using PMSSA multi-threshold segmentation are shown in Figure 6.
5. Results and Discussion
It was observed that the 5-dimensional segmentation yielded the most abundant blood vessels. To scientifically investigate the performance of the improved PMSSA multi-threshold segmentation of DR vascular images, several classical population optimization algorithms: MSSA, WOA proposed in the literature [21], social spider optimization (SSO) proposed in the literature [22], PSO, multi-threshold segmentation of DR images in the DRIVE dataset, are selected for cross-sectional comparison experiments with the algorithms in this paper. The final segmentation result is shown in Figure 7.
5.1. Evaluation Indicators
In this paper, we evaluate the quality of DR vessel fundus images after algorithm segmentation based on image quality evaluation indicators: PSRN, SSIM, FSIM, Dice, and Jaccard [23–26]. The accuracy of segmentation was evaluated based on sensitivity, specificity, and accuracy [27, 28]. The bias of the algorithm segmentation was assessed based on Shannon entropy and regional consistency [29, 30].
5.1.1. PSNR
PSNR is the input-to--to-output signal-to-noise ratio and is mainly used to evaluate the loss rate of an image at the signal level. When segmenting an image, a portion of the target information is eliminated while nontarget pixels are removed. Thus the evaluation indicator PSNR is introduced to evaluate the degree of distortion in the signal of the resulting image after segmentation by the algorithm in this paper, as follows:where M and Nare the images before and after the segmentation process, respectively. The PSNR only evaluates the distortion rate of the image input and output, but not the structural change of the segmented image. To remedy this deficiency, the parameter SSIM is introduced in this paper.
5.1.2. SSIM
SSIM is a metric that evaluates the variation of an image in terms of structure. In the field of unsupervised threshold segmentation, SSIM is a vital evaluation indicator that evaluates the deviation of an image in terms of brightness, structure, and contrast in terms of structure, which is specifically defined as follows:where and are defined as follows:
The formula for SSIM is as follows:
Analysis of the data in Tables 1 and 2shows that the improved PMSSA is less likely to fall into a local optimum solution. This advantage becomes more apparent as the dimensions increase. When dealing with images with complex information, the PMSSA algorithm has more obvious advantages.
5.1.3. FSIM
SSIM is an essential parameter for evaluating the deviation of the segmented image from the structure, as image processing is targeted, pixels in different regions should have different weights assigned to them. Therefore, some scholars have proposed a structural evaluation criterion FSIM that assigns different weights to different regional pixels [25], which is specifically defined as follows:of which,
5.1.4. Dice and Jaccard
To evaluate the segmented image and ground truth in detail, this paper introduces the Jaccard index and Dice similarity.
Dice coefficient is also known as the Sorensen-Dice coefficient. Named after Thorvald Sorensen and Lee Raymond Dice, it is an ensemble similarity measure function that is usually used to calculate the similarity of two samples. Dice coefficient is pixel level, the real target (ground truth) appears in a certain area A, the target area of our model prediction result is B, then the Dice coefficient equation is as follows:
Jaccard similarity coefficient (Jaccard) is used to compare similarities and differences between finite sample sets. Jaccard similarity coefficient is defined as the ratio of the size of the intersection of C and D to the size of the union of C and D, the definitions are as follows:
Analyzing the data in Tables 3–7, it was found that the images segmented in 3 dimensions by the improved PMSSA algorithm were optimal in terms of evaluation indicators SSIM, FSIM, and PSNR. And the images segmented in 4 dimensions by the PMSSA algorithm were optimal in terms of evaluation indicators Jaccard and Dice. There is over-segmentation in both 4 and 5 dimensions, which may be because the segmented image has a larger vessel width than the expert segmentation. The vascular width of the image is larger than the expert segmentation. To verify the idea, this chapter introduces sensitivity ( ), specificity (), and accuracy (Acc) as one the evaluation indicators (see Table 8).
5.1.5. Sensitivity, Specificity, and Accuracy
To evaluate the accuracy of pixel categorization in segmented images, this paper introduces evaluation indicators: sensitivity (), specificity (), and accuracy (Acc).
Sensitivity , specificity , and accuracy Acc are defined as follows:where , , , and represent the true example, the true counter-example in the predetermined vasculature, and the false positive, false counter-examples in the predetermined background, respectively.
Combining Tables 9–11 reveals that: compared to the other population optimization algorithms (mentioned above), the sensitivity, accuracy, and specificity of the improved PMSSA algorithm are improved, and the specificity and accuracy are significantly better. The PMMSA algorithm still has an inverse trend in 5-dimensional segmentation, but the PMSSA algorithm has the most minor inverse trend, PMSSA is more stable, and the DR vessel images' quality is better. The algorithm avoids the local solutions from getting by the other population algorithms, and the algorithm’s efficiency is relatively good.
5.1.6. Shannon Entropy
Shannon’s entropy is used to evaluate the richness of the information contained in the segmented image, so the higher the entropy value, the richer the information contained in the image, which is specifically defined as follows:
5.1.7. Regional Consistency
Multi-threshold segmentation of fundus vascular images is fundamentally based on particular rules for clustering pixels in fundus vascular images, which means that there is a certain similarity within the segmented classes and a certain difference among the classes. To evaluate the algorithm of this paper more comprehensively, this paper introduces regional consistency to evaluate the performance of PMSSA multi-threshold segmentation of fundus vascular images, which is specifically defined as follows:
Analysis of the data in Tables 12 and 13 revealed that the Shannon entropy value and regional consistency value of the PMSSA algorithm for multi-threshold segmentation of fundus vascular images soar in superiority with increasing dimensions. The higher the dimension, the more pixels are judged to be blood vessels in the DR image after segmentation. Tables 12 and 13 show that each algorithm has the highest Shannon entropy value in 4 and 5 dimensions. Therefore, the inverse growth of each algorithm in 4–5 dimensions can be judged as an over-segmentation phenomenon. In summary, the 3-dimensional segmentation is the closest to the expert manual segmentation. The indicators of the final result are shown in Table 14.
Combining the above evaluation indicators, it can be found that the specificity and accuracy of this paper for extracting diabetic RGB vascular images are particularly outstanding, which can reach 98.58% and 95.53%, respectively. After GMF and top-bot hat transformation, the improved PMSSA has higher segmentation accuracy in multi-threshold fundus vascular images. The algorithm has better stability in dimensions 2–5 with a more excellent and uniform probability of jumping out of the local optimal solution interval obtained by other algorithms. It also can be seen that the overall performance of DR vessel images segmented by PMSSA is the best, and the advantage becomes more evident as the dimension increases. Thus, PMSSA is more advantageous in processing images with complex vascular distributions like the images (e) and (f) in Figure 1.
5.2. Experimental Discussion
DR lesions are one of the common complications in diabetic patients in the late stage. In the early stage of DR lesions, timely diagnosis and treatment can effectively stop the deterioration of the disease. Fundus lesions are mostly found near blood vessels, with complex image information and difficult end-vessel identification. To address the problem of incomplete end vessel segmentation in diabetic RGB vascular image segmentation, we propose a PMSSA algorithm with significantly improved specificity and accuracy by using the particle swarm optimization algorithm to improve the parameters of the salp swarm algorithm.
This paper uses this algorithm to segment the RGB images after matched filtering and top-bot hat transformation and extracts the blood vessel features; the effectiveness of this paper’s algorithm is verified by doing a comprehensive comparison analysis with other algorithms (MSSA, WOA, SSO, and PSO) on ten evaluation indicators: SSIM, PSNR, FSIM, Jaccard, Dice, Shannon entropy, regional consistency, accuracy, sensitivity, and specificity. GMF enhances the contrast between the vessel pixels and the background pixels in the DR vessel image. The top-bot hat transformation preserves the backbone information, and some of the details of the vessel end in the DR vessel images. The experimental results show that the method extracts complete vascular information with an average accuracy of 95.53% and a specificity of 98.58%. Compared with other algorithms, the accuracy and specificity are significantly improved. In dimensions 2–5, the algorithm is more stable, with a greater and more uniform likelihood of jumping out of the local optimal solution interval established by previous algorithms. Using the performance indicators listed above, it is clear that the overall performance of DR vessel images segmented by PMSSA is the best, and the advantage becomes more apparent as the dimension rises. When dealing with images with complicated vascular distributions, PMSSA is more advantageous.
Therefore, the PMSSA algorithm designed in this paper has excellent prospects in diagnosing and treating retinal disease images, which will help ophthalmologists diagnose retinal disease more effectively and significantly reduce the burden on doctors.
6. Conclusions
DR vascular images have a dense distribution of blood vessels and unclear end information about the end of the blood vessel. Therefore, this paper used GMF and top-bot hat transform to preprocess DR fundus vascular images. Combining the two resulting images enhances the main and partial terminal information of blood vessel, in preparation for PMSSA multi-threshold segmentation of DR vessel images. In this paper, an improved salp algorithm PMSSA based on the PSO algorithm is proposed, which is more efficient in the optimization process of salp individuals to avoid salp individuals from moving toward the nontarget direction in the current iteration process and improve the stability of the algorithm. The improved PMSSA multi-threshold segmentation preprocessed DR fundus vascular images were compared with MSSA, WOA, SSO, and PSO algorithms. Through comprehensive analysis, it is found that the improved PMSSA has a better iterative effect and is less likely to fall into the local optimal solution. The improved PMSSA has better overall performance when processing DR images with redundant information and no apparent obscure pixel distribution. Although the PMSSA algorithm is easier to obtain the optimal solution, the computation time of the algorithm is relatively long. Given this phenomenon, in future work, we will be optimized in terms of algorithms and the structure of the implementation of the algorithm code.
Data Availability
This paper uses the DRIVE dataset from the Retinopathy Online Challenge (ROC) database. The ROC is a library of fundus images for DR screening competitions, made public by the University of Iowa in 2009. This study adhered to the tenets of the Declaration of Helsinki.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This research was funded by the National Science Foundation for Young Scientists of China (Grant No. 61806060), 2019–2021, “Research of Diabetic Retinal Image Analysis Algorithms Based on Fractional Order Differential and the Deep Learning.”