Pavement Pothole Detection Based on Cascade and Fusion Convolutional Neural Network Using 2D Images under Complex Pavement Conditions

Background: The development of deep learning technology has promoted the industrial intelligence, and automatic driving vehicles have become a hot research direction. As to the problem that pavement potholes threaten the safety of automatic driving vehicles, the pothole detection under complex environment conditions is studied. Objective: The goal of the work is to propose a new model of pavement pothole detection based on convolutional neural network. The main contribution is that the Multi-level Feature Fusion Block and the Detector Cascading Block are designed and a series of detectors are cascaded together to improve the detection accuracy of the proposed model. Methods: A pothole detection model is designed based on the original object detection model. In the study, the Transfer Connection Block in the Object Detection Module is removed and the Multi-level Feature Fusion Block is redesigned. At the same time, a Detector Cascading Block with multi-step detection is designed. Detectors are connected directly to the feature map and cascaded. In addition, the structure skips the transformation step. Results: The proposed method can be used to detect potholes efficiently. The real-time and accuracy of the model are improved after adjusting the network parameters and redesigning the model structure. The maximum detection accuracy of the proposed model is 75.24%. Conclusion: The Multi-level Feature Fusion Block designed enhances the fusion of high and low layer feature information and is conducive to extracting a large amount of target information. The Detector Cascade Block is a detector with cascade structure, which can realize more accurate prediction of the object. In a word, the model designed has greatly improved the detection accuracy and speed, which lays a solid foundation for pavement pothole detection under complex environmental conditions.