0 Preface
Flatness is one of the important components of various shape tolerances of auto parts, and it is also one of the most important technical indicators in the production testing process. To a large extent, flatness determines the assembly quality of some parts of the car and affects the performance of the car. So how to quickly and accurately detect the flatness of auto parts has become one of the research directions in the auto parts manufacturing industry.
So far, in the parts flatness measurement industry, some manufacturers still use the feeler gauge measurement method, which has low detection accuracy and low efficiency, and can only detect the edge of the part. This is unacceptable for production lines that require mass testing. Some manufacturers use the three-coordinate measurement method [1], the measurement speed is slow, and the measured elements need to be kept at a level, which leads to the fact that enterprises can only conduct random inspections of products but not full inspections. Aiming at the problems of low precision, low efficiency, slow speed, and easy waste of human resources in the current measurement methods, this paper designs an online inspection system for the flatness of auto parts based on machine vision, which satisfies the current domestic requirements for the flatness of auto parts. Accurately measure the demand of large quantities, high speed, and high precision, reduce the cost of enterprises, improve the operating efficiency of enterprises, and have broad application prospects. The visual online inspection system for flatness of auto parts meets the current domestic demand for large-scale, high-speed, and high-precision flatness measurement of auto parts, reduces enterprise costs, improves enterprise operating efficiency, and has broad application prospects.
1 overall scheme design
The overall technical route of this system is shown in Figure 2, which is divided into two parts: vision and machine:
(1) In the visual part, it mainly includes two parts: hardware and software. Vision hardware mainly refers to the laser profiler used for measurement. Vision software refers to the point cloud algorithm and the design of the host computer software. The point cloud algorithm adopts the method of rough registration and then fine registration to realize flatness measurement. The upper computer software design is based on the Visual Studio 2019 software development environment using the C++ programming language and calling the PCL point cloud library and the OpenCV vision library for writing, testing and implementation.
(2) In the machine part, it is necessary to complete the construction of the overall hardware structure, cooperate with the visual part to realize the movement of the object under test and the laser profiler, and design the software of the lower computer at the same time to realize the motion control of the object under test, the laser profiler and the linkage of various mechanisms .
2 System module design
As shown in Figure 3, the system can be divided into three modules according to the function.
2.1 Motion control module design
The motion control module takes the programmable control PLC as the core, and completes the information interaction with the computer, laser profiler and detection sensor by driving the cylinder, the moving motor of the laser profiler, the unqualified plane rejecting mechanism and the movement of two conveyor belts. The motion control process of the DUT detection is as follows:
① Place the object to be tested on the stage and move with the feeding conveyor belt.
②When the loading detection laser displacement sensor detects that an object enters the detection area, and when the stage is in place in the detection area, the sensor transmits the signal to the PLC to control the cylinder to lift the stage upward to ensure that the object under test It is not disturbed by factors such as conveyor belt vibration.
③ Then, the PLC sends a signal to the motor, and the motor drives the laser profiler to move, and the laser profiler starts scanning at the same time. After the surface point cloud information of the object to be measured is obtained, it is transmitted to the computer for the next point cloud display, calculation and processing.
④ After scanning, the object to be tested enters the unloading conveyor belt. The PLC receives the flatness detection feedback signal given by the computer. If the flatness of the object to be tested is not qualified, it will control the rejecting mechanism to reject it. If the flatness of the object to be tested is qualified, the object to be tested is allowed to pass through and enter the subsequent inspection process.
2.2 Design of point cloud acquisition module
The point cloud acquisition module consists of a laser profiler and an object to be measured. This module is mainly responsible for completing the acquisition of point cloud information, selecting the appropriate sampling frame rate, sampling point distance and depth of field to scan the object to be measured.
2.3 Point cloud processing module design
The point cloud processing module is responsible for the display of the point cloud, the registration of the point cloud, the calculation and display of the flatness of the object to be measured, and completes the information interaction with the PLC. Because the laser profiler will obtain large point cloud data during laser scanning, and the commonly used Iterative Closest Point (ICP) algorithm has high requirements for the initial position of the point cloud, and it is easy to fall into the local optimal solution and cannot achieve Actual demand. In order to ensure that the point cloud registration does not fall into the local optimal solution, the point cloud registration is usually divided into two parts, that is, the point cloud registration method that first performs the coarse registration of the point cloud and then performs the fine registration[2]. Coarse registration of clouds can be done through point cloud
The local feature descriptors of the key points establish correspondences to obtain the transformation matrix, and the extraction of feature descriptors determines the accuracy of registration. Rusu et al. [3] proposed a feature histogram (Point Feature Histogram, PFH) descriptor to calculate the geometric position relationship of all point pairs in the neighborhood of key points as the main feature of key points; then Rusu et al. improved the PFH algorithm and retained Most of the identification features of PFH propose a Fast Point Feature Histogram (FPFH) feature. Then use the Sample Consensus Initial Alignment (SAC-IA) method. Although this method significantly improves the registration accuracy, it reduces the efficiency and is often used for initial registration.
Therefore, this paper first extracts the FPFH of the point cloud as the feature description of the point cloud [4]; then according to the feature, uses the SAC-IA method to complete the initial registration; finally adopts the ICP algorithm to realize the fine registration of the point cloud.
2.3.1 Coarse point cloud registration
As shown in Fig. 4 is the influence area of PFH calculation of a query point Pq. With Pq as the center and r as the radius, all points in the circle are connected to each other. The UVW coordinate system is established with P1 as the origin, and the relationship characteristics between P1 and P2 of any two points and their normals n1 and n2 are as follows:
α, φ, d and θ are the feature information required by the FPH descriptor. After obtaining these feature information, the histogram can be constructed and normalized. The disadvantage of this method is that its calculation speed will become slow when the amount of data becomes large. In order to improve the extraction efficiency of FPH descriptor, the FPFH algorithm is proposed.
All points in the point cloud data must first obtain SPFH features. SPFH is used to calculate the association between feature points and neighboring points. Using formula (5) to calculate FPFH features greatly reduces the amount of calculation. Through subsequent weighting processing, The FPFH feature obtains the relationship information between the neighbor points and the neighbor points, fills the loss of the relationship information between the neighbor points and the neighbor points in SPFH, and improves the matching accuracy.
After obtaining the FPFH feature sub, use the SAC-IA algorithm for point cloud P and point cloud Q to perform feature point matching calculation. The SAC-IA algorithm randomly selects k sample points in the point cloud P, finds the closest corresponding points of these points in the point cloud Q, and calculates the transformation matrix according to the relationship between the two point cloud data sets. Use formula (6) to calculate the deviation value of the registration, and use the Levengerg-Marquardt algorithm to continuously optimize the best translation matrix and rotation matrix.
2.3.2 Point Cloud Fine Registration
Point cloud fine registration is the process of accurately obtaining the relative correlation and transformation matrix of point cloud [5]. Because the rigid body transformation matrix obtained by SAC-IA roughly overlaps the data of the two point clouds, but the registration accuracy is far from meeting the requirements of practical engineering applications, so precise registration is performed on the basis of the initial registration. Point cloud fine registration adopts the Iterative Closest Point (ICP) algorithm: first, establish the corresponding point sets P and Q according to a certain method, among which PA, QB, the number of corresponding point pairs is n. Then the least squares method is used to iteratively calculate the optimal coordinate transformation, that is, the rotation matrix R and the translation vector t, so that the error function f Rt ( , ) is minimized until the requirements are met.
3 Experimental results and analysis
The point cloud registration algorithm in this paper is applied to the flatness measurement of the laser profiler, and the flatness measurement experiment is carried out on the auto parts model. As shown in Figure 6, the green point cloud is the source point cloud, and the blue point cloud is the target point cloud. The hardware environment of this experiment is Intel Core i7-9750H @2.60GHZ 8GB; the graphics card is Nvidia gtx 1650; the software environment is Visual Studio 2019. In order to verify the influence of point cloud registration accuracy on flatness measurement results, the SHOT+ICP point cloud algorithm is selected for comparison with the algorithm used in this paper. Use the flatness measurement results of the coordinate measuring machine as a reference value. The flatness measurement results are shown in Table 1.
It can be concluded from the table that the flatness measurement results of the two methods are basically consistent with the measurement results of the three-coordinate measuring machine. Since the repeatability of the measurement results of the method adopted in this paper can be controlled below 0.0023mm, an effective evaluation of flatness can be realized. In addition, the measurement time of the three-coordinate measuring machine is more than 15 minutes, while the flatness measurement time using the algorithm in this paper can be controlled at about 24.5s, so the flatness measurement method in this paper is more suitable for large-scale measurement occasions.
4 Conclusion
The domestic auto parts flatness testing industry is relatively backward, how to quickly and accurately measure the flatness of auto parts is one of the key research directions. The on-line inspection system for the flatness of auto parts based on machine vision designed in this paper is very suitable for the occasions that need to detect the flatness of auto parts in large quantities and has broad development prospects.
Source of the article: Wang Yulong. Design of Auto Parts Flatness Inspection System Based on Machine Vision [J]. Electronic Testing, 2022,36(18)
Of course, there are many large and small machine vision companies on the market at present, and the machine vision inspections are good, but here I would like to recommend a machine vision platform - Coovally, which runs the code. Coovally is a complete AI modeling process, A machine vision platform for AI project management and AI system deployment management. The development cycle can be shortened from months to days, and the development, integration, testing and verification of AI vision solutions can be accelerated. Help improve the enterprise's AI stack software development capabilities, so that advanced AI systems can be popularized at a lower cost and faster. "Package its own AI capabilities" for business personnel to use, so as to realize "teaching people how to fish". At present, Coovally has covered multiple application areas, including manufacturing quality inspection, geological disaster monitoring, power industry equipment monitoring, medical special disease diagnosis, smart transportation, smart parks, etc.
In addition to common defect detection, it can also detect: flame detection, smoke recognition, smoking recognition, helmet recognition, pedestrian detection, personnel intrusion recognition, cross-border detection, climbing recognition, leaving the post detection, loitering detection, crowd recognition, Crowd density detection, fall recognition, anti-epidemic clothing recognition, mask recognition, goggles recognition, fire extinguisher placement, non-reflective clothing recognition, welding defect recognition, glass product defect detection, corridor debris stacking recognition, trash can overflow recognition, river floating object recognition, mouse recognition at night, package movement recognition, electric vehicle entry into the elevator recognition, pick-up and release recognition, escalator bulky luggage recognition, livestock detection, bird nest detection, crop pest detection, ship intrusion recognition, muck truck recognition, More than 100 common types such as bare soil pile recognition, non-motor vehicle recognition, vehicle parking violation recognition, electric vehicle helmet recognition, large truck traffic restriction recognition, water accumulation recognition, manhole cover recognition, road surface damage recognition, heavy machinery vehicle detection, traffic light recognition, etc. AI application scenarios.
For defect detection of auto parts, Coovally is also perfectly fine.