Intelligent defect detection system based on AI image recognition, application in the steel industry - technical solution

Note: Subsequent technology sharing, immediate updates, and more timely technical information and learning technical materials will be releasedon the official account CTO Plus . Please follow the official account: CTO Plus

Original text: Intelligent defect detection system based on AI image recognition, application in the steel industry - technical solution (qq.com)

In the previous article of the public account CTO Plus , "Industrial Defect Detection Application System Based on AI Image Recognition (GPU&FPGA)", I gave a rough description of the defect detection application scenarios of my AI image recognition products in the industrial field, including application scenarios and implementation. Cases and advantages. In this article, I will further introduce the technical solution of this product.

If you have any application requirements or questions, please contact the author of this article in the background. Contact information (same as WeChat): 15801030767

Overview

At present, domestic and foreign steel companies mainly rely on manual identification and judgment in the scrap inspection process. Due to the lack of efficient
, accurate, and convenient acceptance tools, scrap quality inspectors often can only rely on personal professional knowledge and experience.

There are many types of scrap steel in scrap steel plants with clear grade subdivisions. The most important way to inspect the quality of scrap steel is to
grade the scrap steel and identify impurities and other unqualified materials through manual visual inspection. Traditional quality inspection judgments are highly subjective, have many objections to judgments, and have insufficient systematic management and traceability.

The intelligent scrap inspection system will realize automated data collection and precise data management, and apply deep
image recognition technology to assist manual scrap grading and identification of unqualified materials. It can identify the proportions of various grades of scrap and provide Pricing and weight of impurities, as well as providing alarms for oily seals.

According to the current actual situation, the project can be verified in 2 parking spaces in the steelmaking workshop first, and then
promoted or moved to other parking spaces after passing the acceptance inspection.

Introduction to the Intelligent Scrap Inspection Program

Automatically identify the vehicle body and directly lock the unloading vehicle through the target detection algorithm to start detection.
The algorithm satisfies a variety of sites and unloading scenarios. Including: aviation crane suction cup, steel grabbing machine grabbing, direct unloading truck unloading and leveling, dock shipping scenes, etc.
Accurately identify scrap steel grade and miscellaneous weight through multiple dimensions (thickness, shape, size, color, old and new).
It can identify a variety of unqualified material types: impurities, sealed parts, oily parts, over-long parts, defective materials, mechanical pig iron, etc., and give real-time alarms.
Multiple surveillance videos can be switched in real time and historical review can be provided. System information is traceable and verifiable.
Supports multiple unloading positions working at the same time, that is, multiple users working at the same time.

Advantages and value of scrap steel intelligent inspection system

Avoid losses caused by manual inspection errors. Manual inspection results are inevitably unstable, and if they are lower than the evaluation benchmark, it will increase the cost of the steel plant.
The system can realize 7*24 inspection and can receive scrap vehicles throughout the day, effectively improving work efficiency.
To improve openness and fairness, the system has video and high-definition picture storage, which can be audited and traced to achieve transparent management.
Effectively avoiding safety hazards, quality inspectors can stay away from the unloading site and work remotely with the system.
Effectively improve the accuracy of identifying unqualified materials. The system algorithm can accurately identify unqualified materials such as sealed parts and oil-stained parts, and strictly eliminate the danger of dangerous materials entering the furnace.
It greatly reduces the work intensity of relevant personnel, helps to improve employee satisfaction and reduces brain drain.

Scrap manual inspection process

Inspection process:

The scrap truck must be weighed and the scrap material type must be declared first.
Drive into the designated discharge detection position and use the suction cup to discharge. Generally, 1-2 inspectors will make the judgment.
The inspector observes the materials unloaded from the car and the suction cup. After unloading the scrap steel, the inspector directly gives the grade and impurity deduction results of the car based on experience, and inputs the results into the handheld terminal and enters the company system.
Empty trucks are weighed and settled.

Scrap steel grade inspection standards

The project verification phase mainly implements intelligent detection of bulk materials such as leftover materials and heavy waste, as shown in Table 2-1. In the future, we will have the opportunity to try the intelligent detection of broken materials together.

Scrap steel inspection results

The current inspection results are input by the quality inspector into the handheld terminal, including license plate, company, type, miscellaneous and other information, and are automatically synchronized to the company's computer system.

Intelligent Inspection and Judgment Solution-Scrap Intelligent Inspection and Judgment Algorithm

Algorithm 1: Scrap steel grade identification algorithm

[Using algorithms such as deep learning and target detection, the image is first pre-processed, histogram equalized and image morphology processed. Then check the image quality. Finally, the output result of scrap steel grade is obtained]

Algorithm 2: Location identification algorithm for unqualified materials

[Using YOLO with Darknet, for each input image, the location of impurities and other unqualified materials can be identified and marked. 】

Algorithm 3: Type identification algorithm for unqualified materials

[Using Fast R-CNN to search for potential impurity targets in photos, delimit the location range, and then identify the category to which they belong]

Algorithm 4: Unqualified material weight output algorithm

[Select and train a deep convolutional neural network from algorithm models such as ResNet, Inception, and VGG, using photos as input, and directly using unqualified materials such as impurities as output to achieve end-to-end content estimation]

Algorithm 5: Sealed Parts Identification Algorithm

【Use algorithms such as histogram equalization and gamma correction to optimize imaging effects and highlight photo targets. Use smoothing to reduce noise in the picture, use edge enhancement to highlight the outline of the seal, and give an early warning. 】

Algorithm 6: Oil-stained parts identification algorithm

[Use algorithms such as histogram equalization and gamma correction to optimize imaging effects and highlight photo target features. Use smoothing to reduce noise in pictures, use edge enhancement to highlight the outline of oily parts, and give early warning. 】

Supplementary algorithm

【Data enhancement】

In view of the small amount of data in the initial stage of system launch, data enhancement methods can be used to expand the training data set. Perform one or more operations on the collected images: random rotation, random cropping, color dithering, noise disturbance, horizontal flipping, and vertical flipping to artificially increase the size of the training set, enrich data diversity, and avoid overfitting. .

【Transfer Learning】

Use public data sets, such as ImageNet, to pre-train neural network models such as YOLO, and transfer parameters and knowledge based on them, thereby achieving support for scrap steel detection tasks with less computing resource overhead and training time.

Intelligent detection scheme-intelligent detection algorithm process

Training and inference process:

Scrap image data collection

Set up cameras at various points in the scrap workshop to collect scrap image data during the unloading process. The collected images have two purposes: extract some samples for image annotation and use them for algorithm model training; during the actual production process, they are used as data sources and passed to the scrap steel intelligent detection algorithm data reasoning system for data reasoning.
Cameras are used on-site for 7*24-hour full monitoring, and they are saved in the form of videos and screenshots are used to select pictures and collect pictures.
Screenshot principle: Each vehicle is unloaded. After the vehicle is parked, the suction cup will suck the steel material several times. In principle, the initial picture after the vehicle is parked is first selected, and the image after each unloading action is completed. A picture. That is, select the picture before the first unloading action and the picture after each action.
Adjustment of camera angle: Unloading is relatively fixed. In order to obtain high-quality pictures, it is not necessary to adjust the angle, focal length and other settings of the camera. Just ensure that the camera can capture the complete unloading process of the vehicle.
Shooting content: Taking the truck currently being unloaded as the main content, you can see most of the front of the truck, most of the wheels, and the complete compartment.

Scrap image data annotation

The scrap steel plant quality inspectors and experts will annotate and guide the collected image data. During this process, our company will dispatch professional annotation engineers to assist on-site, and the on-site time will be arranged according to the specific situation. Data annotation is an important prerequisite for the implementation of the algorithm and requires the experience and guidance of quality inspectors and experts at scrap steel plants.

An example of annotation: the quality inspector can look at the unloading image, circle the boundary and location of the unqualified material, and write the visual weight value; the quality inspector can look at the unloading image and write out the grade of the scrap steel in the picture.

Software function description-intelligent remote operation console

Based on big data and artificial intelligence technology, various information sources in the steel plant are integrated and analyzed to form a visual information platform that is easy to operate and has complete functions. Through role and authority management, the platform helps improve the work efficiency of managers, assists front-line workers in making scientific decisions, and reduces the operational burden of operation and maintenance personnel.

(1) Intelligent monitoring: With the support of underlying AI technology, key link operation monitoring can be realized

Intelligent inspection and judgment: Based on the image data during the unloading process, intelligent analysis is performed through AI algorithms to achieve real-time analysis of the weight of the waste and early warning of the unloading status;
Intelligent analysis of work behavior: In scenarios where people are working, the AI algorithm is called to conduct intelligent analysis of work behavior and protective gear wearing, and prompts for any illegal operations found; (optional)
Free combination of monitoring screens: Monitoring screens in the camera list can be freely retrieved according to business needs;

(2) Console: On-site data is updated in real time to help companies grasp operational dynamics

Intelligent process management: Use cameras and recognition algorithms to integrate data from key links such as entry review, weighing, unloading, settlement and departure, etc., to achieve unified management of the entire process of business data within the site;
Real-time statistics of on-site business: Automatically integrate key business information such as the number of vehicles entering the day, scrap weight, alarm information, etc., and perform real-time dynamic updates;
Intelligent detection algorithm performance: intuitively displays the data and model performance of intelligent detection algorithms used in key business links;
Camera status overview: displays the information and monitoring status of all cameras in the factory;

(3) Data panel

Data panel: Visualize historical business data to help companies analyze operational status
Total order volume, distribution and trends: Customized statistical analysis of the company's historical order volume, including year-on-year, incremental stock, etc.;
Historical performance of intelligent detection algorithms: Visualize the historical performance of intelligent detection algorithm models and easily trace prediction records;
Unloading alarm message statistics: visualize the alarm information of key business and grasp the order quality;
Distribution and trends of recycled steel types: Visualize the weight ratio and trends of various types of steel to intuitively understand the business status;
Intelligent analysis: supports labeling, training and service updates, enabling continuous iteration of algorithms
Data annotation: Use a customized image annotation tool to annotate the image data of the unloading scene, and the system automatically converts the annotated image files into a standard model input structure; (optional)
Model training: The system has a built-in AutoML module. Users only need to simply select the data set and model type to quickly train a new model and test results; (optional)
Service update: Automatically summarize all online and non-online services and their model performance, and easily update services; (optional)

Mobile APP

In order to realize the informatization and intelligence goals of the entire factory operation process, the "Unloading Intelligent Assistant" APP assists quality inspectors at the unloading site to complete multiple tasks such as vehicle positioning in the unloading area, predictive service calls, and unloading alarms.

Hardware Configuration

Hardware diagram
The hardware layout diagram is as follows:

A high-definition network camera is installed above each parking space to provide an unobstructed front view of the car. It can be installed on the walls on both sides of the workshop without interfering with the driving suction cups.

Hardware list

The hardware configuration list is as follows:

Network solution-network topology diagram

Each server considers load balancing and mutual backup, which not only ensures that the system can be upgraded smoothly, but also ensures that if a single server fails, it will not cause a fatal error in the system and stop the service; the communication link between the center and the mobile information center uses a mobile dedicated line. input method to ensure the security, stability and speed of data collection equipment communication.

Each host in the system is connected by Gigabit links, which ensures that the internal network can still meet the data transmission needs between hosts after the system is expanded.

Other requirements regarding the network

The factory network staff can assist in opening the public network port, which allows our company's implementation personnel to remotely control the angle and other parameters of the on-site camera.

The application of this product can be extended to education, medical care, finance, precision manufacturing and other industries. I will share the application scenarios of these industries later when I have time. If you have any needs, please contact the author on the backend of the official account CTO Plus. The contact information is ( Same as WeChat): 15801030767

For more related technical points about Go, please pay attention to the public account: CTO Plus’s subsequent posts. If you have any questions, please leave a message in the background for communication.

Recommended reading :

SteveRocket's blog_CSDN blog - Python advanced, Django advanced, C advanced blogger Steve Rocket is good at Python advanced, Django advanced, C advanced, and other aspects of knowledge https://blog.csdn.net/zhouruifu2015 /

More information· Search [CTO Plus] on the WeChat public account and follow it to get more information. Let’s learn and communicate together.

For a description of the public account, visit the following link

About Articulate "Be a porter of knowledge and technology. Be a lifelong learning enthusiast. Be a technical circle with depth and breadth." I have always wanted to develop skills in the professional field https://mp.weixin.qq. com/s/0yqGBPbOI6QxHqK17WxU8Q