Recently, I was looking at the papers on 3D reconstruction and made a knowledge summary.
3D reconstruction technology
3D reconstruction technology is to truly reconstruct the 3D virtual model of the surface of the object in the computer, and build a complete 3D model of the object, which can be roughly divided into three steps:
- The point cloud data of the object is collected from various angles by image acquisition equipment such as cameras. A single camera can only shoot from one angle of the object. To obtain complete information on the surface of the object, it is necessary to shoot the object from multiple angles;
- Transform the point cloud data of each perspective obtained in the first step into the same coordinate system to complete the registration of multi-view point cloud data;
- The mesh surface of the model is constructed according to the registered point cloud data.
3D reconstruction method
At present, according to the different reconstruction methods, there are mainly the following reconstruction methods:
binocular stereo vision
The binocular stereo vision measurement method mainly simulates the stereo imaging principle of human eyes. Two cameras with appropriate angles on the left and right shoot an object in the scene at the same time. Through the triangular geometric relationship and the principle of parallax, the surface of the object under the same viewing angle of the left and right cameras is obtained. The coordinate information of the point is used to construct the position and shape of the object.
Photometric Stereo Vision Methods
According to color theory, the color of the surface of an object under different lighting strips is also different. Photometric stereo vision captures the same surface of an object under lighting conditions in various directions, and uses the obtained set of images to reconstruct the surface shape of the object. In theory, three light sources can calculate the normal vector of a certain point on the surface of the object, but usually in order to avoid noise interference and improve accuracy, light sources in more than one direction will be selected.
Trinocular Stereoscopic Vision and Polyocular Stereoscopic Vision
In binocular stereo vision, the matching result often depends on the amount of texture and color information on the surface of the object being photographed. When the texture information of the object is small, it is easy to produce wrong matching when the pixels of the left and right images are matched, and the result is unreliable. In order to reduce wrong matching and improve the accuracy of binocular vision matching, trinocular and multi-eye stereo vision systems are a solution. According to the optical triangle theory, the matching in multi-eye stereo vision adopts the method of "multiple overlapping points and multi-angle front intersection", so redundant data can be effectively used, the problem of mismatching can be solved to a certain extent, and the accuracy of 3D reconstruction can be improved, but this method It is more cumbersome, and the hardware equipment used at the same time is more complicated and expensive.
Motion-Based 3D Reconstruction
This method is mostly used in dynamic scene tracking and other fields. The basic principle of the algorithm is to reconstruct the three-dimensional information of the object by relying on the relative motion of the camera to output a series of images.
Structured light stereo vision
The structured light stereo vision method uses standard grating stripe structured light to project onto the surface of the object, and the projected light stripes change with the fluctuation of the surface shape of the object. Extract the three-dimensional information of the object. The structured light stereo vision method is based on the optical triangulation measurement theory, and the camera and the optical projector must be calibrated before measurement.
The next step is to look at the knowledge related to offline structured light stereo vision and binocular stereo vision.