Table of contents
The basic principle of binocular ranging
Advantages and difficulties of binocular ranging
Classification of RGB-D cameras
Disadvantages of RGBD cameras:
3. What are the research directions of point cloud
1. Classification based on point cloud
2. Segmentation based on point cloud
3. Target detection based on point cloud
4. Enhancement and completeness of point cloud data
1. What is a point cloud
A point cloud is a dataset of points in a certain coordinate system.
The point cloud obtained according to the principle of laser measurement includes three-dimensional coordinates (XYZ) and laser reflection intensity (Intensity). The intensity information is related to the surface material, roughness, incident angle direction of the target, the emission energy of the instrument, and the laser wavelength .
The point cloud obtained according to the principle of photogrammetry , including three-dimensional coordinates (XYZ) and color information (RGB).
Combining the principles of laser measurement and photogrammetry to obtain point clouds, including three-dimensional coordinates (XYZ), laser reflection intensity (Intensity) and color information (RGB).
2. How to get the point cloud
Point clouds are not captured by ordinary cameras, but are generally obtained by 3D imaging sensors, such as binocular cameras , 3D scanners , RGB-D cameras , etc.
1. 3D laser scanner
According to different scanning and imaging methods, laser scanners can be divided into one-dimensional (single point) scanners, two-dimensional (line array) scanners and three-dimensional (surface array) scanners. According to different working principles, it can be divided into pulse ranging method (also known as time difference measurement method) and triangulation method.
Pulse ranging method: The laser scanner sends a beam of laser light from the laser emitter to the object at time t1. Since the surface of the object can reflect the laser light, the receiver of the scanner will receive the reflected laser light at time t2. Calculate the distance d=(t2-t1)c/2 between the scanner and the object from the speed of light c, time t1, t2.
Obviously, the measurement accuracy of the pulse ranging 3D laser scanner is limited by the accurate measurement time of the scanner system. When using this method to measure close-distance objects, a large error will occur due to the short time. Therefore, this method is more suitable for measuring long-distance objects, such as terrain scanning, but not suitable for close-range scanning.
Triangulation distance measurement method: use a laser beam to focus on the surface of the measured object at a certain angle, and then image the laser spot on the object surface from another angle. The angle of the light is also different. The angle θ of the chief light can be calculated by measuring the position of the spot image with a CCD (image sensor) photodetector. Then combined with the known baseline length d between the laser light source and the CCD, the distance L≈dtanθ between the scanner and the object is deduced through the triangular geometric relationship.
The handheld laser scanner constructs 3D graphics through the above-mentioned triangle ranging method: through the handheld device, the laser spot or linear laser is emitted from the object to be measured. Two or more detectors are used to measure the distance from the surface of the object under test to the handheld laser product, usually with the help of a specific reference point—usually an adhesive, reflective patch—used as a scanner Positioning and calibration in space. The data obtained by these scanners will be imported into the computer and converted into 3D models by software.
The characteristics of the triangulation method : simple structure, large measurement distance, anti-interference, small measurement point (tens of microns), high measurement accuracy. However, it will be affected by factors such as the accuracy of the optical component itself, the ambient temperature, the light intensity and diameter of the laser beam, and the surface characteristics of the measured object.
2. Binocular camera
The basic principle of binocular ranging
The principle of binocular distance measurement_NineDays66's Blog-CSDN Blog
This blog completely and clearly introduces the entire process and principle of binocular ranging, as well as the comparison and parameter adjustment of several stereo matching algorithms, which is of great value! !
disparity map
Parallax is the difference between the x coordinates corresponding to the same spatial point in the imaging of two cameras . It can be encoded into a grayscale image to reflect the distance. The closer to the lens, the brighter the grayscale; (provided that the two The camera is horizontal, if the two cameras are vertical, use the difference in y coordinates)
Advantages and difficulties of binocular ranging
Advantages of binocular system:
(1) Compared with lidar and other solutions, the cost is lower;
(2) Relying on natural light, it can be used both indoors and outdoors
(3) Directly use the parallax to calculate the distance, and the accuracy is higher than that of the monocular;
Difficulties of the binocular system:
(1) The amount of calculation is very large , and the performance requirements of the calculation unit are very high, which makes it difficult to commercialize and miniaturize the binocular system. Therefore, it is relatively difficult to solve the dual-purpose computing problem on chips or FPGAs. Most of the research institutions or manufacturers that use binoculars in the world use servers for image processing and calculations, and some use FPGAs for processing after simplifying the algorithms.
(2) The effect of binocular registration directly affects the accuracy of ranging.
2.1. Very sensitive to ambient light . The binocular stereo vision method relies on the natural light in the environment to collect images. Due to the influence of environmental factors such as changes in lighting angles and light intensity, the brightness difference between the two captured pictures will be relatively large, which poses a great challenge to the matching algorithm.
2.2. It is not suitable for scenes that are monotonous and lack texture . Since the binocular stereo vision method performs image matching based on visual features, it will be difficult to match for scenes lacking visual features (such as sky, white wall, desert, etc.), resulting in large matching errors or even matching failures.
3. RGB-D camera
What does RGB-D mean?
RGB-D depth image introduction RGBD_donkey_1993's blog - CSDN blog
RGBD = RGB + Depth Map
RGB ：
The RGB color mode is a color standard in the industry. It obtains a variety of colors by changing the three color channels of red (R), green (G), and blue (B) and superimposing them with each other. Yes, RGB is the color that represents the three channels of red, green, and blue. This standard includes almost all the colors that human vision can perceive. It is one of the most widely used color systems at present.
Depth Map：
In 3D computer graphics, a Depth Map is an image or image channel that contains information about distances to surfaces of scene objects from viewpoints. Among them, the Depth Map is similar to a grayscale image, except that each pixel value of it is the actual distance from the sensor to the object. Usually the RGB image and the Depth image are registered, so there is a one-to-one correspondence between pixels.
Classification of RGB-D cameras
Divided into two categories:
one is the structured light method , such as Kinect v1, Iphone X;
the structured light method is produced to solve the problem of binocular matching, to solve the problem of sensitivity to ambient light, it is infrared light, which can also be used at night, and does not depend on light and texture.
Among them, Kinect v1 is the earliest structured light method. It is the PrimeSense company in Israel, which was later acquired by Apple and used in Iphone X. Face unlocking is very stable. The spatial resolution is 3mm and the depth resolution is 1cm. (2m distance)
One is the time-of-flight (ToF) method , such as Kinect v2, Phab 2 Pro.
The time-of-flight method is to emit a beam of light pulses (usually invisible light), and then reflect back through the object, and then receive the light pulses. Calculate the distance of the measured object from the camera by detecting the time of flight.
The measurement accuracy of TOF will not decrease with the increase of the measurement distance, and the anti-interference ability is relatively strong, which is suitable for occasions where the measurement distance is relatively long, such as unmanned driving, AR, etc. But his power consumption is relatively large, the resolution is low, and the quality of the depth map is poor.
Disadvantages of RGBD cameras:
(1) Not suitable for outdoor use
(2) Affected by dark objects, translucent objects, specular reflection objects, and parallax (
3) High power consumption
(4) Relatively low resolution
(5) Depth map quality is closely related to hardware
( 6) Higher cost, more expensive than binocular cameras
Dark black objects can absorb a lot of infrared light so it will cause inaccurate measurements , which makes the received infrared light very little.
The influence of smooth surface objects: Diffuse reflection is the basis for clear imaging from multiple angles
The influence of transparent objects: translucent objects will be reflected twice, if they are transparent objects, they will pass through directly, resulting in inaccurate depth measurement.
The impact of parallax: There is a certain distance between the infrared transmitter and receiver of the structured light depth camera, so there is obvious parallax at the edge of the object.
Advantages of RGBD cameras
3. What are the research directions of point cloud
1. Classification based on point cloud
- Classification of point clouds is often referred to as 3D shape classification. Similar to image classification models, 3D shape classification models usually first generate global embeddings by aggregating encoders, and then pass the embeddings through several fully connected layers to obtain the final result.
- Based on point cloud aggregation methods, classification models can be roughly divided into two categories: projection-based methods and point-based methods.
2. Segmentation based on point cloud
The purpose of 3D segmentation is to label each point, which requires the model to collect global context and detailed local information at each point. In 3D image segmentation, there are mainly two tasks: semantic segmentation and instance segmentation.
3. Target detection based on point cloud
4. Enhancement and completeness of point cloud data
Point clouds collected by LiDAR, especially those from outdoor scenes, suffer from different kinds of quality issues, such as noise, outliers, and missing points. So our goal is to complete the missing points and remove the abnormal points
5. Driverless
statement:
This article is a summary of the content of the cited articles, if there is any infringement, it will be deleted!