In recent years, many researchers adopted the DL to locate and classify bridge diseases and achieved good performance. For example, Lei Z. et al. used a convolution neural network (CNN) to detect road cracks. They divided the complete images into several small image blocks and then classified the small image blocks to complete the extraction of cracks [8]. Cha et al. used the CNN and the target detection network of a faster region-based convolutional neural network (Faster R-CNN) to realize the detection of building damage, including cracks [9, 10]. Li et al. adopted the YOLO to perform the detection of concrete cracks [11]. Zhang et al. further used the target detection network of the YOLOv3 and migration learning to complete the damage detection of cracks in concrete bridges [12]. Hoang and Nguyen constructed an automatic model to detect and classify asphalt pavement crack. It could detect cracks and wa able to recognize the types of pavement cracks, including the longitudinal, transverse, and alligator cracks [13]. Meng studied an image segmentation algorithm for concrete cracks based on CNN. Compared with other methods, the algorithm achieved higher detection accuracy and generalization ability [14]. Ding et al. established a concrete crack identification model based on an improved Mask R-CNN. The results showed that the average prediction accuracy of concrete crack identification was better than the YOLOv4 method [15]. Hu et al. studied a pavement crack detection method based on YOLOv5. The experimental results showed that the detection accuracy of the YOLOv5 series models was above 85% [16]. Note that, with the above works, one can detect the location of the cracks but cannot quantify the cracks. In practical engineering applications, the quantitative result of crack is very important, which can be directly used to analyze the stress status of the structure, and thus is an important basis for bridge maintenance decision. Moreover, the entire learning process of the present DL is a dimension reduction process. The crack width of a concrete bridge is usually narrow compared to the image. Hence, the crack information may be lost during the learning process.
The fully convolution neural network (FCN) has been widely studied to realize pixel-level crack detection and measurement. FCN is an end-to-end, pixel-to-pixel convolutional network for semantic segmentation. An FCN is composed of downsampling and upsampling parts. The downsampling part mainly includes convolutional layers and pooling layers. The upsampling part includes deconvolution layers [17]. For example, Yang et al. employed the FCN to realize the pixel-level crack classification and measurement. The results showed that the accuracy of segmentation was 97.96%, and the relative quantification error was within 24.01% for crack width [18]. Li et al. utilized the U-Net to realize the location of concrete cracks in the tunnel. The segmentation accuracy was 92.25%, and the relative quantification error was within 18.57% for crack width [19]. They further adopted the segmentation network of FCN and Naïve Bayes data fusion (NB-FCN) to realize the location of concrete cracks. The cracks were quantified by introducing postprocessing with accuracy being 93.2% [20].
10 strike network diagram 2.1 crack
DOWNLOAD: https://tinourl.com/2vFbNT
The rest of the paper is organized as follows. In Section 2, the network architecture of the modified bridge crack detector based on the DL-assisted image processing approach is presented. The detection, quantification, and visualization algorithms are described. In Section 3, the establishment of bridge damage dataset, offline data augmentation, training process, and crack classification and location results obtained by the YOLOv5 algorithm are introduced in detail. We analyze crack connected component optimization methods, including the mask filter used to remove handwritten marks, the ratio filter adopted to eliminate speckle linear noises, the fusion of the same crack detection bounding boxes, the regional adaptive threshold segmentation, and the connected components search approach based on the crack trend of area. Besides, we compare the crack quantification accuracy with and without these optimization methods. A comparison between the calculated crack width based on the connected components and the results measured by a bridge engineer using the crack width gauge is given. Finally, conclusions are drawn.
The overall architecture of the damage detection network based on the DL-assisted image processing approach is shown in Figure 1. It includes two modular functional parts. At first, the region with crack is identified through the DL-based detection network. Here, YOLOv5 is employed to get the predicted bounding boxes. After that, according to the predicted bounding boxes, the digital image processing techniques are adopted to quantify the length and width of the cracks, and the crack details are further visualized in the image.
Through the connected component labeling algorithm [40, 41], the binary image obtained after removing the handwritten marks is transformed into the corresponding connected component graph, as shown in Figure 3(h). It can be seen from the connected component diagram that the connected component belonging to the crack is approximately linear, and the length is generally much larger than the width. Therefore, line-like noise is further filtered out by the aspect ratio feature of the connected component, and speckled noise is filtered out by the area feature, as shown in Figure 3(i). It can be observed that there are some intermittent connected component fragments, and there are still some unremoved noises. In order to make the visible cracks more complete and clear and in line with the actual trend of cracks, we propose a region connected component search algorithm based on the connected component of cracks. The binary image of cracks after the connection is shown in Figure 3(j).
Figure 6 presents the network detection results of cracks in the experiment. We can find that, with the detection network, the cracks can be successfully detected. But for some cases, a single crack is detected as multiple bounding boxes.
(2) Fusion of Multiple Bounding Boxes for the Same Crack. When the detection network is used to identify the cracks, a crack in the image may have multiple bounding boxes because of the disconnection of the cracks or the inaccurate network identification. In that case, the visualized cracks are not continuous, and the number of cracks cannot be counted correctly. Therefore, we further perform a fusion operation for multiple bounding boxes.
We propose a region connected component search algorithm based on the crack trend. The flowchart of the proposed algorithm is shown in Figure 10(a). The first step is to initialize the index and vertex coordinates of each connected component. The second step is to update the connected component and select the connected component with the largest aspect ratio in the image as the current connected component. In the third step, we consider a downward connection from the current connected component and build a square search box according to the lower vertex of the current connected component. The width of the search box is equal to the width of the image. The schematic diagram of the connection is shown in Figure 10(b). The fourth step is to find the target connected component to be connected in the search box. In the search box, the distance d and the angle θ between the lower vertex of the current connected component and the upper vertex of the next candidate connected component are calculated. Then, the confidence degree c can be calculated as follows:
In conclusion, we proposed a method of concrete bridge crack detection and quantification based on a DL-assisted image processing approach. The detection and the quantification phases are separately designed, which can easily replace the target detection network with an advanced DL algorithm. The target detection network based on DL is used to determine whether there is a crack in the image and then extract the crack area. In addition, we proposed a new digital image quantification and crack visualization method by introducing the region connected component search algorithm based on the crack trend.
2ff7e9595c
Comments