Abstract
China is gradually transitioning from the “tunnel construction era” to the “tunnel maintenance era,” and more and more operating tunnels need to be inspected for diseases. With the continuous development of computer vision, the automatic identification of tunnel lining cracks with computers has gradually been applied in engineering. On the basis of summarizing the weaknesses and strengths of previous studies, this paper first uses the improved multiscale Retinex algorithm to filter the collected tunnel crack images and introduces the eight-direction Sobel edge detection operator to extract the edges of the cracks. Perform mathematical morphological operations on the image after edge extraction, and use the principle of the smallest enclosing rectangle to remove the isolated points of the image. Finally, the performance of the algorithm is judged by the objective evaluation index of the image, the accuracy of crack recognition, and the running time of the algorithm. The image filtering algorithm proposed in this paper can better preserve the edges of the image while enhancing the image. The objective evaluation indexes of the image have been improved significantly, and the main body of the crack can be accurately identified. The overall crack recognition accuracy rate can reach 97.5%, which is higher than the existing tunnel lining crack recognition algorithm, and the average calculation time for each image is shorter. This algorithm has high research significance and engineering application value.
1. Introduction
Tunnels are an important part of the road and railway transportation network. With the continuous increase in the mileage of roads and railways in China, and due to the aging of the structure itself, a large number of tunnels have gradually entered the maintenance period. The tunnel itself is faced with its own structural aging and damage from external factors. Among many tunnel diseases, lining cracks are one of the fundamental problems that endanger the safe operation of the tunnel. It will not only affect the durability of the lining structure, but may even endanger the safety of the structure itself and cause major losses. If tunnel cracks are discovered and repaired in time at the initial stage, the probability of serious consequences will be greatly reduced. At present, manual detection is still the main way to find tunnel cracks, but this method is low in efficiency, is low in detection accuracy, and consumes a lot of manpower and material resources, and the overall detection effect is not satisfactory. With the development of tunnel inspection vehicles, it is possible to use a vehicle-mounted CCD camera to take pictures of the lining of a tunnel in a short time. Then, according to the obtained tunnel lining image, a detection method based on computer vision is used to identify the cracks in the image. Compared with the artificial tunnel crack detection method, the crack detection method based on computer vision has the advantages of fast recognition speed and high detection accuracy and has been widely used in tunnel crack detection [1–3].
Scholars at home and abroad have conducted a lot of research on the identification of tunnel lining cracks and have achieved satisfactory results. Li et al. [4] analyzed the existing problems of the existing tunnel crack detection system and creatively applied the latest technology to the moving train, which can realize the tunnel lining crack identification when the train speed is 50 km/h, but the overall crack identification accuracy is not high. In order to better identify and analyze cracks, Shi et al. [5] proposed an analysis method based on crack width characteristics, which greatly reduces the interference of “false cracks” and improves the accuracy of crack identification. Jang et al. [6] proposed a hybrid image autonomous concrete crack detection technology based on deep learning. The hybrid image combined with visual and infrared thermal imaging can improve the detection ability and detection accuracy of cracks. Win and Thein [7] proposed a related reverse method to identify the location and size of cracks in concrete structures. Nhat-Duc and Peir [8] used a gray intensity adjustment method called Min-Max Gray Level Discrimination (M2GLD) to automatically identify and analyze cracks. Jia et al. [9] proposed a multiscale spatial tunnel lining crack water leakage identification method based on point cloud images and proposed a fusion image detection operator, which can maintain the stability of band-shaped crack image detection. Existing various edge detection algorithms are not satisfactory for image recognition results, and the time and accuracy of crack extraction need to be further improved. Zhu et al. [10] realized the accurate identification of tunnel cracks by fusing the template-based analysis method, the linear structure analysis method based on Hough transform, and the approximate crack structure analysis method based on support vector machine, but the algorithm took a long time. Dai et al. [11] used improved homomorphic filtering to process the collected images and used the XDoG edge extraction method to extract the edges of the cracks in the image. Practice has proved that this method has the advantage of fast calculation speed. On the basis of structural similarity, Yan et al. [12] proposed an image quality evaluation method based on convolution filtering and gradient structural similarity. This method can obtain better evaluation results for dithered and blurred lining images with cracks. Although the existing image preprocessing methods are basically perfect, there are still environments with poor tunnel lighting, and image recognition needs to be enhanced.
In response to insufficient illumination and uneven illumination, Song et al. [13] proposed a multiscale joint low-light enhancement network based on Retinex theory to effectively solve the problems of noise amplification and color distortion. Sun et al. [14] proposed a nonuniform illumination correction algorithm based on multiscale Retinex, which enhanced the measurement accuracy of DIC. Wang et al. [15] designed a low-light image enhancement optimization algorithm based on frame accumulation and multiscale Retinex joint processing; this algorithm improves the overall quality of the image to a certain extent. For the problem of occurrence of halation and overenhancement, Ping et al. [16] fused Gabor filtering with Retinex theory. In order to solve the light scattering in the water and the interference of the impurities in the water on the light propagation, Huang et al. [17] combined the color correction method with the deblurring network, which can better eliminate the blur and enhance the details. Liu et al. [18] used MSRCR and guided filtering methods for defogging and proposed a white balance fusion global guided image filtering (G-GIF) method, which effectively solved the problem of edge detail loss.
Although the existing image preprocessing methods are basically perfect, there is still a lack of image filtering methods for the environment with poor tunnel illumination. Various existing edge detection algorithms are not satisfactory for image recognition results, and there are often more false cracks and noises, which affect the final recognition results of cracks. The time and accuracy of crack extraction need to be further improved. Based on the abovementioned problems, this paper proposes a new tunnel lining crack identification algorithm based on improved multiscale Retinex and Sobel edge detection.
2. Basic Principles of Retinex Algorithm
In 1978, Land [19] first proposed the calculation theory of color constancy perception, also known as the Retinex theory. Different from traditional linear and nonlinear methods that can only enhance a certain type of image feature, Retinex can strike a balance between dynamic range compression, edge enhancement, and color constancy. Therefore, various different types of images can be adaptively enhanced.
2.1. Single-Scale Retinex (SSR)
The Retinex theory can divide the image into two parts, namely, the incident component and the reflected component [20]. The imaging principle of the Retinex algorithm is shown in Figure 1. For a certain image, the relationship between the three can be expressed in the following formula:
The basic principle of the Retinex algorithm is to estimate the reflected component through the incident component and then achieve the purpose of image enhancement. In 1997, Jobson improved the Retinex algorithm and proposed the SSR algorithm [21]. The basic idea of the single-scale Retinex algorithm consists of three steps. The first step is to construct a Gaussian surround function and then use the constructed function to filter the three color channels of R, G, and B of the color image. The filtered image is the incident component we estimated. The third step is to subtract the incident component from the original image in the logarithmic domain, and the data obtained is the reflection component, and the reflection component is the output result image. The formula of the SSR algorithm is shown in the following formula:
In formula (2), represents the color channel in the color model. represents the low-light image input by the first color channel. represents the output of the first channel after single-scale Retinex processing. stands for convolution operation. represents a Gaussian wraparound function, and its function expression is usually as shown in the following equation:
In formula (3), where is the normalization coefficient, since the integral of is always equal to one, therefore, can be obtained by calculation. is the scale constant of the wrap function.
2.2. Multiscale Retinex Algorithm (MSR)
Since the single-scale Retinex algorithm selects Gaussian function as the number of traverse segments, the enhanced image cannot simultaneously guarantee a large-scale compression of dynamic range and contrast intensity. In order to balance the two enhancement effects, a very accurate scale constant must be selected. Therefore, the SSR algorithm usually cannot obtain satisfactory processing results.
In view of the shortcoming of the single-scale Retinex algorithm, some scholars have proposed the multiscale Retinex algorithm [22, 23]. Compared with the single-scale Retinex, the multiscale Retinex algorithm can take into account the two characteristics of dynamic range compression and image contrast. The MSR algorithm is a linear weighted summation of the single-scale Retinex algorithm, and its specific expression is shown in the following formula:
In formula (4), is the weight of different scales, satisfying . represents the number of scales of the wrapping function, and the general value is 3, which represents three scales of small, medium, and large. represents the wraparound function at scale .
2.3. The Improved MSR Algorithm
The traditional MSR algorithm uses a Gaussian function as the surround function, but because the Gaussian operator cannot estimate the illumination well in the transition zone, when the illumination changes greatly, a halo phenomenon will occur, which will weaken the image and affect the enhancement effect of the image. Therefore, this paper chooses to use bilateral filter function instead of the original Gaussian wrap function [24]. Compared with the Gaussian function, bilateral filtering can enhance the spatial domain while maintaining the smoothness of the time domain, which can better solve the problem of misjudgment of the Gaussian function. The edge information of the image can also be preserved, so that the obtained image is more continuous and smooth, and it can also solve the halo problem caused by the Gaussian function to a certain extent. The function of bilateral filtering is
Among them, and are the input image and the filtered image, respectively. and are the spatial domain kernel function and value function. and are the spatial distance difference scale parameter and the pixel difference scale parameter. is the set of pixels with the center of in the input image, and the radius of . is the filter radius, and the larger the , the larger the filter interval, and the more complicated the calculation process.
2.3.1. Gamma Correction
Gamma correction on the image after bilateral filtering can make the transformed image more natural. The specific expression form of gamma correction is shown in the following formula:
Among them, represents the incident component after gamma correction. is the gamma coefficient. If , the overall image becomes brighter; if , the overall image becomes darker; if , the image does not change.
2.3.2. Sigmoid Function
The sigmoid function can simulate the nonlinear and local characteristics of human vision to a certain extent and has good local details and dynamic range compression capabilities [25]. Assuming that the low-illuminance image can be obtained by multiplying the actual image and the interference term , the expression is shown in the following equation:
Using the Sigmoid function to stretch the contrast of the reflected image can enhance the contrast of the image and make the processed image more visible. The formula is shown in the following formula:
When the brightness component of the image is enhanced, the saturation of the image is lower. In order to make the image color more full, the saturation of the image needs to be stretched. However, because the tunnel images obtained under different lighting conditions have certain differences, the corresponding saturation processing levels must also be different. This paper proposes an adaptive nonlinear stretching algorithm for saturation components, and the formula is shown in the following equation:
Among them, and represent the saturation of the image after stretching and before stretching, respectively. , and are, respectively, the average value, maximum value, and minimum value of the (R, G, B) color component of the corresponding pixel of the image.
The overall flow of the algorithm in this paper is shown in Figure 2. First, extract the RGB three color channels of the image, and then convert the RGB color space to the HSV color space, and the H channel remains unchanged. The S channel is adaptively enhanced. The V channel first uses the improved MSR algorithm for image filtering. Then, use the gamma function for color correction. Finally, the HSV space is converted back to the RGB space, and Sigmoid function is used to improve the reflection component and enhance image details. So, we got the image preprocessing result. Next, the Sobel edge detection algorithm is used to detect the edges and remove the independent points in the image to make the crack shape more complete.
2.4. Analysis of Filtering Results
The improved MSR algorithm proposed in this paper is compared with the SSR algorithm and the MSR algorithm, and the obtained tunnel low-illuminance image comparison chart is shown in Figure 3.
(a)
(b)
(c)
(d)
It can be seen clearly from Figure 3 that the filtering results after different types of Retinex algorithms can be seen. The three filtered images can improve the brightness and contrast of the image to a certain extent. However, the algorithm proposed in this paper (MSR-B) keeps the boundary of the image obtained by the SSR algorithm and the MSR algorithm clearer and can better highlight the tunnel cracks and maintain the details of the cracks.
3. Image Edge Extraction
3.1. Edge Extraction of Eight-Direction Sobel Operator
The Sobel operator based on eight directions performs edge detection on the image in the eight directions of , , , , , , , and (counterclockwise rotation along the axis). Through the convolution operation of the operator template in each direction with each pixel, the foreground characteristic color is given to the pixel, whose operation value exceeds a certain threshold. Finally, the feature pixels in the eight directions are gathered into the same image, and then the final Sobel operator detection image is obtained [26]. The operator can be divided into eight according to the direction of the template, and the detection template is shown in Figure 4. The final test result is shown in Figure 5.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
3.2. Removal of Isolated Edges of Cracks
The principle of the minimum enclosing rectangle is to use a rectangle of a certain size and position to exactly completely contain the edge of an object in the image. The rectangle usually refers to the rectangle defined by the largest and smallest abscissas and the largest and smallest ordinates in the image object area as its smallest enclosing rectangle, and then judging whether there are other rectangles in the neighborhood of each enclosing rectangle to determine whether it is an isolated edge.
The specific process of the algorithm is usually as follows: (1) first, perform the morphological expansion operation on the image, which can realize the initial merging of the tiny holes in the image and the connected areas. (2) Obtain the smallest bounding rectangle of all connected regions. (3) Taking the smallest circumscribed rectangle of a connected domain as the center, judge whether there are other adjacent circumscribed rectangles in the size of its eight-neighborhood template; if not, define it as an isolated point and discard it. If there are other circumscribed rectangles, they are judged as nonisolated points. (4) Eliminate the small circumscribed rectangle with less than 50 white pixels in the circumscribed rectangle in the image, after the expansion operation is carried out, and the crack image determined by the isolated point is shown in Figure 6.
As shown in Figure 6, the eight neighborhoods of Sector 3 and Sector 4 do not contain other circumscribed rectangles, so they are judged as isolated edges and discarded. However, Sector 1 and Sector 2 are nonisolated points, and the algorithm is reserved. It can be seen from Table 1 that after removing the isolated edges of the cracks, the number of noncharacteristic pixels can be reduced by 27%, and the final proportion of the number of cracks can reach 94%. The number of noise points in the image is obviously reduced, and the crack skeleton roughly appears.
3.3. Breakpoint Connection and Image Filling
The crack skeleton usually has a certain degree of discontinuity, and the crack image after the edge is extracted by the Sobel edge detection operator does not conform to the actual situation, so the crack needs to be filled. The direction vector crack connection algorithm proposed in the literature [3] is used to connect the broken points of the processed image, and the final result is shown in Figure 7. Then, fill the upper and lower white edges of Figure 7 as a whole. The specific effect is shown in Figure 8. It can be seen from Figures 7 and 8 that, after the breakpoint connection and crack filling operations, the crack image can already be identified, which can lay the foundation for other subsequent steps.
4. Algorithm Evaluation
The algorithm in this paper is based on Intel (R) Core (TM) i5-5200U CPU @ 2.20 GHz, memory is 8.00 GB, operating system is Windows 8.1 Chinese version system, and Matlab R2016a is used for simulation experiment.
4.1. Improved Retinex Algorithm Evaluation
The objective evaluation criteria of image enhancement effect usually include three objective indicators of peak signal-to-noise ratio (PSNR), information entropy, and contrast. Generally speaking, the larger the peak signal-to-noise ratio, the smaller the distortion of the image. Information entropy represents the amount of image information. Generally speaking, the higher the image information entropy, the greater the information entropy, and the more complete the image edges maintained. Contrast can better reflect the relationship between tunnel cracks and concrete lining in the image, and it can also be said that the image is more in line with the intuitive visual effect of the human eye.
4.1.1. Peak Signal-to-Noise Ratio (PSNR)
The formula for calculating the peak signal-to-noise ratio is shown in equations (10) and (11):where represents the mean square error of the current image and the reference image ; and are the height and width of an image, respectively; is the number of bits per pixel, generally taken as 8; that is, the pixel grayscale is 256.
4.1.2. Information Entropy
Entropy is the information entropy of the image, which describes the average amount of information that the image wants. The entropy is also large. The clearer the image texture, the richer the color. The definition of entropy is shown in the following formula:
Among them, represents the probability of each gray level appearing.
4.1.3. Image Contrast
The calculation formula of contrast is shown in the following formula:
Among them, is the grayscale difference between adjacent pixels. is the pixel distribution probability, where the grayscale difference between adjacent pixels is .
4.1.4. Comparison of Recognition Results
Calculate the peak signal-to-noise ratio, information entropy, and image comparison chart of the four images in Figure 3, respectively, and the calculation results are shown in Table 2.
It can be clearly seen from Table 2 that the algorithm in this paper has a large improvement in PSNR, information entropy, and image contrast, and all surpass the SSR algorithm and the MSR algorithm. The obtained image is clearer, the boundary preservation is more obvious, and the image information is richer, which proves the superiority of the image filtering algorithm proposed in this paper.
4.2. Evaluation of Crack Identification Effect
Compare the cracks that can be recognized by the human eye in the original image with the cracks that the computer automatically recognizes. The part of the crack contour drawn manually in the original image is shown in red in Figure 9, all the correct pixels are marked as , and the other redundant parts in Figure 9 are represented in green in the original image, marked as . The total number of pixels is denoted by .
The calculation formula of the recognition accuracy rate is shown in formula (14). After calculating multiple images, it can be seen that the accuracy of crack identification in this paper is approximately 80.4%; that is, the main body of the crack can be fully identified, but there is a certain identification error. The overall recognition effect is good:
In order to verify the accuracy of the algorithm, 200 tunnel crack images were randomly selected from the gallery for identification. If the cracks in the image can be completely identified, the identification is considered successful; otherwise, the identification fails. The image of cracks on the surface of ordinary concrete is selected for comparison with the cracks of the bridge, and the recognition results are shown in Table 3.
From the comparison in Table 3, it can be seen that the accuracy of the proposed tunnel lining crack recognition image in this paper can reach 97.5%, which is higher than other existing algorithms. The recognition effect of ordinary concrete surface cracks is also very good, which may be due to the better light obtained when collecting ordinary concrete surface cracks, and there is basically no other noise interference. However, the accuracy of identifying bridge cracks is still low. The bridge cracks here do not refer to the cracks in the asphalt pavement of the bridge, but the wings and piers of the bridge superstructure. These parts usually need to be added with waterproof paint, and there may be part of the paint peeling off in the obtained crack image, which leads to low recognition accuracy. The recognition rate of ordinary concrete crack images is higher than that of tunnel lining crack images, and bridge crack images have a certain relationship with light intensity. When the light is strong or weak, it will significantly affect the imaging of the crack edge and then have a greater impact on the identification of the crack width. After testing, when the light intensity is in the range of 1400–3100lux, the test accuracy of the width is relatively high, and the average relative error is less than 5% [27].
4.3. Algorithm Running Time Comparison
The running time of an algorithm is an important indicator to evaluate the efficiency of an algorithm. In the case of ensuring the recognition accuracy, the faster the algorithm runs, the higher the image recognition efficiency. Compare the algorithm proposed in this paper with literature [6], crack recognition algorithm based on maximum entropy [28], gradient-based improvement Canny algorithm [29], and SVM-based crack recognition algorithm [30] (the algorithm consumes time and takes the average of the test object). The calculation time comparison is shown in Table 4.
From the comparison in Table 4, it can be seen that the gradient-improved Canny algorithm and the maximum entropy crack segmentation algorithm have the shortest running time, but because the algorithm itself does not preprocess the image through steps such as image filtering, the crack extraction results are relatively general. The SVM-based algorithm takes an average of 8.7 seconds for each picture, but the recognition result is not much different from this article. Compared with the algorithm in literature [6], the algorithm in this paper has a greater improvement in the processing efficiency of each image. It is suitable for a large number of tunnel image processing and has certain engineering application significance.
5. Conclusion
(1)The improved multiscale Retinex algorithm is used to filter the collected images of tunnel lining cracks. The addition of gamma transform and Sigmoid function can preserve more details of the image. Through the comparison of peak signal-to-noise ratio, information entropy, and contrast, it can be seen that the newly proposed image enhancement algorithm can well retain the original information of the image, and the edges of the image are preserved intact, which is conducive to subsequent recognition.(2)The eight-direction Sobel operator is used to detect the edge of the image, and the isolated points are removed by the morphological operation and the principle of the smallest bounding rectangle, and finally the crack is filled. The results show that the proposed crack processing algorithm can make the crack identification accuracy reach 90.4% and can basically realize the complete identification of the main body of the crack.(3)The crack recognition algorithm proposed in this paper has a recognition accuracy of 97.5%, and the calculation time for each picture is relatively short. Subsequent improvements can be made to the Sobel edge detection operator to reduce the recognition of error points. It is also possible to perform semantic segmentation of the image and then identify the segmented cracks to improve the recognition accuracy.Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest.
Authors’ Contributions
Wang Quanlei completed the writing of the main body of the thesis. Zhang Ning, Ma Chao, and Zhou Zhaochen mainly completed the programming work. Jiang Kun made certain revisions to the article as a whole. Dai Chunquan played an important guiding role in the article.
Acknowledgments
The authors are grateful to the financial support from the Humanities and Social Sciences Fund of the Ministry of Education (no. 20YJAZH022).