Steel is the basic material for the development of modern industrial technology. With the improvement of industrial manufacturing capacity, steel is widely used in the construction industry, ocean engineering, and aerospace engineering, and more stringent requirements are placed on steel. However, due to the limitations of the production process, there are inevitably surface defects of various depths in the steel production process, such as scratches, cracks, concave surfaces, etc. These defects will not only affect the appearance of the product, but also lead to a decrease in the wear resistance and toughness of the steel and shorten the service life of the steel. Therefore, analyzing and mastering how to improve the surface quality of steel is an important problem that modern steel manufacturers need to solve urgently.
Based on the Faster R-CNN target detection algorithm, this article proposes a machine vision-based steel surface defect detection method.
basic algorithm
1.1 Faster R-CNN algorithm
The Faster R-CNN algorithm is an end-to-end target detection algorithm. The network structure is shown in Figure 1, which is divided into four parts, as follows.
(1) Backbone network
The backbone network of the Faster R-CNN algorithm is usually VGG-16, ResNet and other networks, which are responsible for extracting input image features. By consulting the references, this paper chooses the VGG-16 network as the backbone network, and its structure is shown in Figure 2.
(2) RPN network
The RPN network is responsible for extracting candidate frames, and its basic principle is shown in Figure 3. When the input model image is extracted through the backbone network and then input to the RPN, the RPN performs a sliding window on the feature map to generate a frame. The part with a high frame drawing score is used as the recommended area, which is the candidate frame.
(3) ROI pooling
ROI pooling is responsible for scaling the feature map to a certain ratio so that the feature map meets a preset size. Currently, the commonly used scaling method is to first divide the region proposal into n equal parts of the same size, then find the maximum value of each part, and finally copy the maximum value and output [6].
(4) R-CNN detection network
The R-CNN network is responsible for classifying and identifying candidate frame sample categories. First, set a threshold a according to the actual situation, then calculate the intersection-over-union ratio (IoU) of the real and predicted target frames, and finally judge the size of IoU and a to determine the sample category of the candidate frame. When IoU>a, it is judged as a positive sample; otherwise, it is a negative sample. The Faster R-CNN algorithm has the characteristics of strong robustness and high detection accuracy, but it is easy to lose structural information in the feature extraction stage. At the same time, the classification and positioning accuracy of the region proposal box in the R-CNN network detection process needs to be improved. . Therefore, in order to further improve the detection accuracy of the network, this paper improves the backbone network and the detection network [7-8].
1.2 Faster R-CNN network improvement
1.2.1 Backbone network
In order to solve the problem that the Faster R-CNN network is easy to lose structural information, this paper realizes the improvement of the Faster RCNN network by fusing the features of the backbone network. First, the conv3-256-1 and conv3-256-3 of the VGG-16 network Block3 are fused, and the conv3-512-1 and conv3-512-3 of the VGG-16 network Block5 are fused. Then fuse the fused features again. After setting each Block, it is the maximum pooling, and the fully connected layer remains unchanged. Through the above operations, the backbone network can retain structural information as much as possible. The structure comparison of the backbone network Block3 and Block5 before and after improvement is shown in Table 1.
1.2.2 R-CNN detection network
The R-CNN network can make the region proposal box more accurate by increasing the IoU threshold, thereby improving the detection accuracy of the model, but excessively increasing the IoU threshold can easily lead to insufficient model fitting. In order to solve this problem and improve the detection accuracy of the model, this paper improves the model by cascading two RCNN detection networks with different IoU thresholds. Set the IoU thresholds of the two R-CNN detection networks to 0.5 and 0.6, respectively, and divide the positive and negative samples according to their IoU thresholds to train and correct the region proposal box, so that a more accurate bounding box can be obtained and the detection effect can be improved[10- 12].
2 Steel surface defect detection based on machine vision
Based on the improvement idea of the Faster R-CNN network mentioned above, this paper will design the steel surface defect detection based on machine vision as shown in Figure 4. First, VGG-16 with fused features is used to extract the steel surface image of the input model to reduce the loss of structural information of the steel surface image. Secondly, the RPN network is used to extract and generate the candidate frame of the steel defect feature map. Then ROI pooling is used to scale the steel surface defect feature map. Finally, the R-CNN detection network with IoU thresholds of 0.5 and 0.6 was used to classify and identify the feature maps, and the detection results of steel surface defects were obtained.
3 Simulation experiment
3.1 Experimental environment
This experiment is carried out on the Linux Centos7 operating system and tested based on the MXNet framework. The CPU of the system environment is E5-2620 at 2.10 GHz, the GPU is NVIDIA1080, the memory is 64 GB, the compiler is Pycharm Community 1.3, and the interpreter is Python 3.7.
3.2 Data source and preprocessing
The experimental data comes from the NEU-CLS data set, a total of 1800 images of steel surface defects, as shown in Table 2.
3.3 Evaluation indicators
In this experiment, the average accuracy (AP) is selected as the index to evaluate the performance of the model, and its calculation method is shown in formula (1). The average AP (mAP) is selected to evaluate the overall performance, and its calculation method is shown in formula (4). Choose the frame rate per second (frame/s) to evaluate the model running efficiency.
In formula (1) ~ (3): TP is a true example; FP is a false positive example; FN is a false negative example.
In formula (4): Nc is the number of detected defect categories.
3.4 Results and analysis
3.4.1 Model Validation
(1) Backbone network verification
In order to verify the effectiveness of the proposed model to improve the backbone network, the experiment compares the model performance of different backbone networks as feature extraction, and the results are shown in Table 3. It can be seen from the table that compared with ResNet-50, the average detection accuracy of the backbone network of VGG-16 as a model is higher, reaching 84.06%, which is an increase of 2.3%; the detection speed is increased from 6.2 frames/s to 12.23 frames/s, almost doubled. This shows that the proposed model uses VGG-16 as the backbone network, which can improve the detection performance of the model, and the model improvement is effective.
(2) Fusion strategy verification
In order to verify the effectiveness of model feature fusion, different fusion strategies were used to construct the model in the experiment, and the average detection accuracy and detection speed were compared. The results are shown in Table 4. It can be seen from the table that the average detection accuracy of strategy B is the highest, reaching 95.92%, and the detection speed is the fastest, reaching 22.45 frames/s, indicating that the fusion strategy can achieve the best detection effect, that is, the fusion strategy of combining Block3 and Block5 is the best. The improvement of the proposed model is effective.
(3) Detection network verification
In order to verify the cascading effect of the proposed model on different IoU threshold detection networks, the experiment compares the performance of the model at different cascading IoU thresholds, and the results are shown in Table 5. It can be seen from the table that cascading a detection network with IoU=0.6 can improve the average detection accuracy of the network, reaching 98.29%, an increase of 2.37%. Compared with cascading an IoU=0.7 detection network, it has certain advantages. This shows that the proposed model has certain effectiveness and is effective for the improvement of the detection network.
The experimental results of the proposed model ablation are shown in Table 6. It can be seen from the table that compared with before feature fusion, the mAP value of the model after feature fusion increased by 11.86%; compared with before cascading network, the mAP value of the model after cascading network increased by 2.37%; compared with the model before improvement , the detection accuracy of the proposed model increased by 14.23%, reaching 98.29%. The reason is that the feature map generated after feature fusion contains both high-level and low-level information, thereby improving the detection accuracy of the model, and the cascaded network can improve the IoU of the model and improve the detection accuracy of the model [13- 15]. This shows that feature fusion and cascaded network can significantly improve the detection accuracy of the model, that is, the proposed model has certain effectiveness.
3.4.2 Model comparison
In order to further verify the effectiveness of the proposed model, the experiment compares the detection effect of the proposed model and the Faster R-CNN model on the experimental data set before and after feature fusion, and the results are shown in Figure 5. It can be seen from the figure that compared with the improvement before, the prediction accuracy of the proposed model and the FFaster R-CNN model for various steel surface defects have been significantly improved, indicating that feature fusion can effectively improve the accuracy of the Faster R-CNN model for steel surface defects. detection accuracy. Compared with the FFaster R-CNN model, the proposed model has improved the detection accuracy of most types of steel surface defects to varying degrees, indicating that the cascaded detection network can improve the detection accuracy of the model to a certain extent. This shows that the use of feature fusion and cascade detection network is effective in improving the detection accuracy of the Faster RCNN model, that is, the proposed model is effective.
In order to further verify the superiority of the proposed model, the experiment compares the detection effect of the proposed model with the traditional detection model, and the results are shown in Table 7. It can be seen from the table that compared with other traditional detection models, the proposed model has certain advantages in the accuracy and speed of steel surface defect detection, with an average detection accuracy of 98.29% and a detection speed of 12.26 frames/s, which can meet the requirements of steel surface defects. Surface defect inspection needs.
3.4.3 Instance Verification
In order to verify the actual application effect of the proposed model, the proposed model was used in the experiment to detect the steel surface with defects, and the results are shown in Figure 6. It can be seen from the figure that the proposed model can detect different types of steel surface defects more accurately and has strong generalization performance.
4 Conclusion
In summary, the proposed machine vision-based steel surface defect detection method can reduce the loss of structural information by fusing the features of Block3 and Block5 of the VGG-16 network, and further improve the average detection accuracy and detection speed of the model. Reaching 95.92% and 22.45 frames/s; by cascading a detection network with an IoU threshold of 0.6 after the detection network, it is possible to more accurately classify and locate the network area suggestion box, and make the average detection accuracy of the model reach 98.29%. The average detection accuracy and detection speed of the improved Faster R-CNN algorithm reached 84.06% and 12.23 frames/s respectively. Compared with traditional detection methods based on Faster R-CNN, SSD and YOLO-V3 algorithms, the proposed method average The detection accuracy and detection speed have been improved to varying degrees, reaching 98.29% and 12.26 frames/s, with certain effectiveness and superiority, and can meet the needs of different types of steel surface defect detection.
references
slightly.
Interested students can download this paper from CNKI
Article source: Lu Jialin, Cheng Ying※, Feng Sai, Li Mingzhe, Zhang Ninghua, Sun Jiayu. Steel surface defect detection based on machine vision[J]. Electromechanical Engineering Technology, 2022,51(07)
Shun doesn’t want to recommend a machine vision platform that can be used for steel surface defect detection—Coovally, which runs codes. Coovally is a machine vision platform that includes a complete AI modeling process, AI project management, and AI system deployment management. The development cycle can be shortened from months to days, and the development, integration, testing and verification of AI vision solutions can be accelerated. Help improve the enterprise's AI stack software development capabilities, so that advanced AI systems can be popularized at a lower cost and faster. "Package its own AI capabilities" for business personnel to use, so as to realize "teaching people how to fish". At present, Coovally has covered multiple application areas, including manufacturing quality inspection, geological disaster monitoring, power industry equipment monitoring, medical special disease diagnosis, smart transportation, smart parks, etc.
In addition to the common falling object recognition, it can also detect: flame detection, smoke recognition, smoking recognition, helmet recognition, pedestrian detection, personnel intrusion recognition, cross-border detection, climbing recognition, leaving the post detection, loitering detection, crowd recognition , crowd density detection, fall recognition, anti-epidemic clothing recognition, mask recognition, goggles recognition, fire extinguisher placement, non-reflective clothing recognition, welding defect recognition, glass product defect detection, corridor debris stacking recognition, trash can overflow recognition, river course Identification of floating objects, identification of rats at night, identification of package movement, identification of electric vehicles entering elevators, identification of taking and putting objects, identification of large luggage on escalators, detection of livestock, detection of bird nests, detection of crop diseases and insect pests, identification of ship intrusion, identification of muck trucks , Bare soil pile recognition, non-motor vehicle recognition, vehicle parking violation recognition, electric vehicle helmet recognition, large truck restriction recognition, water accumulation recognition, manhole cover recognition, road damage recognition, heavy machinery vehicle detection, traffic light recognition, etc. More than 100 types Common AI application scenarios.