1. Image feature point detection and matching
1. Image feature point detection
The process of determining affine transformation parameters such as the position, orientation, and scale of feature points. Currently popular feature point detection algorithms: Laplacian detection algorithm, Gaussian difference DOG algorithm, and * detection algorithms based on the gradient covariance matrix to detect the position of corner points in the graph, among which the DOG-based feature point detection method-scale invariant feature Transformation (sift) is currently the best performing feature point detection algorithm. SIFT process: DOG filtering is performed on the input image, and all the maximum and minimum values in the filtered image are searched. The pixel coordinates corresponding to these extreme values are the coordinates of the feature points. When the size of the feature area is roughly the same as the width of the DOG filter The extreme value appears when it is equivalent.
2. Image feature point description
SIFT extracts an area around the feature point and uses the feature vector to describe the area as a feature descriptor . The SIFT descriptor uses image gradients, calculates a histogram of local image gradient directions, and creates a 4*4 histogram grid around feature points, each histogram contains eight gradient directions, so SIFT contains 128-dimensional feature vectors.
PCA-SIFT also has SURF (to improve the speed of SIFT by calculating gradients and integrals by approximate methods)
3. Image feature point matching
One is rough matching, which is determined by comparing the Euclidean distance between descriptors and setting a threshold.
One is to search for the nearest neighbors in the feature space to determine the matching relationship. The threshold can be learned, and the distance ratio test can also be used to eliminate false matches.
Another index structure is multi-dimensional search tree. Lowe conducts comparative research on such methods and proposes a priority search strategy based on hierarchical K-means tree. Their work shows that in high-dimensional approximate nearest neighbor (ANN) search problems, multi-random KD-trees perform optimally. In order to further eliminate false matches, geometric verification is required. Usually, the random sampling agreement (RANSAC) algorithm is used to estimate the basic matrix between the two views, and then the false matching is eliminated by calculating the degree of fit between the image matching and the basic matrix. The optional basic matrix The estimation methods of the normalized eight-point method, the gold standard method
4. Camera calibration
Camera calibration in 3D reconstruction refers to the process of solving the camera projection matrix. Including the internal and external parameters of the camera, the internal parameters include the focal length and the principal point, and the external parameters specify the camera pose, which is composed of the camera's rotation matrix and translation vector. The self-calibration of the camera is mainly introduced here, and no prior knowledge of the calibration object and any scene is required.
5. Motion inference structure
SFM improvements: use camera pose estimation to initialize camera parameters, select new heuristics to select initial two views, introduce 3D reconstruction point screening process to eliminate poor quality reconstruction effects, and use focal length information extracted from image file EXIF tags .
6. Dense point cloud
7. Surface reconstruction (mesh)
8. Texture reconstruction (texture map)
The above content refers to "Multi-view 3D Reconstruction Based on Image Point Features" by Mr. Kang Lai from Science Press