1 What is point cloud registration
Point cloud registration refers to inputting two point clouds Ps (source) and Pt (target), and outputting a transformation matrix T (ie, rotation R and translation t) so that the degree of coincidence between T (Ps) and Pt is as high as possible. We can imagine the point cloud as a cloud composed of countless three-dimensional points, and the point cloud registration is to stitch these clouds together according to their actual positions and postures, just like stitching multiple puzzles together to form a Complete 3D model. **Coarse Registration** Coarse registration is performed when the transformation between the two point clouds is completely unknown. The purpose is to provide a better initial transformation value for fine registration. **Fine Registration**Fine registration is given an initial transformation, further optimization to obtain a more accurate transformation. The process of coarse registration and fine registration is shown in the figure below:
2 Common Registration Algorithms
2.1 4PCS Registration
1 principle
The coplanar four points a, b, c, d, which are not fully collinear, define two independent ratios r1 and r2 which are invariant and unique across affine variations . Now given a point set Q with n points, and two affine-invariant ratios r1, r2 obtained from points P, for each pair of points q1, q2⊂ Q, calculate their intermediate point:
If any two pairs of such points, one pair of intermediate points calculated by r1 and the other pair of intermediate points calculated by r2 are consistent within the allowable range, then it can be considered that these two pairs of points may be the affine of the basic points in P corresponding point. Apply the four-point transformation to the global point cloud transformation, calculate the matching overlap of the point cloud, and if it reaches the set threshold, complete the point cloud coarse registration.
2 core code
pcl::registration::FPCSInitialAlignment<pcl::PointXYZ, pcl::PointXYZ> fpcs;
fpcs.setInputSource(source_cloud); // 源点云
fpcs.setInputTarget(target_cloud); // 目标点云
fpcs.setApproxOverlap(0.7); // 设置源和目标之间的近似重叠度。
fpcs.setDelta(0.01); // 设置常数因子delta，用于对内部计算的参数进行加权。
fpcs.setNumberOfSamples(100); // 设置验证配准效果时要使用的采样点数量
2.2 K-4PCS Registration
1 step
The K-4PCS method is mainly divided into two steps:
(1) The point cloud Q is down-sampled using a VoxelGrid filter , followed by 3D keypoint detection using standard methods.
(2) The 4PCS algorithm uses the key point set instead of the original point cloud for data matching, which reduces the scale of the search point set and improves the operation efficiency .
2 core code
pcl::registration::KFPCSInitialAlignment<pcl::PointXYZ, pcl::PointXYZ> kfpcs;
kfpcs.setInputSource(source); // 源点云
kfpcs.setInputTarget(target); // 目标点云
kfpcs.setApproxOverlap(0.7); // 源和目标之间的近似重叠。
kfpcs.setLambda(0.5); // 平移矩阵的加权系数。
kfpcs.setDelta(0.002, false); // 配准后源点云和目标点云之间的距离
kfpcs.setNumberOfThreads(6); // OpenMP多线程加速的线程数
kfpcs.setNumberOfSamples(200); // 配准时要使用的随机采样点数量
pcl::PointCloud<pcl::PointXYZ>::Ptr kpcs(new pcl::PointCloud<pcl::PointXYZ>);
kfpcs.align(*kpcs);
2.3 SAC-IA Registration
1 Step SAC-IA registration implementation process:
① Calculate the FPFH feature descriptors of the source point cloud and the target point cloud respectively;
② Match the points in the two point clouds based on the FPFH feature descriptor;
③ Randomly select n (n >= 3) pairs of matching points;
④ Solve the rotation and translation matrix in the case of matching;
⑤ Calculate the corresponding error at this time; repeat steps 3-5 until the conditions are met, and take the rotation and displacement corresponding to the minimum error as the final result.
2 core code
pcl::SampleConsensusInitialAlignment<pcl::PointXYZ, pcl::PointXYZ, pcl::FPFHSignature33> sac_ia;
sac_ia.setInputSource(source);
sac_ia.setSourceFeatures(source_fpfh);
sac_ia.setInputTarget(target);
sac_ia.setTargetFeatures(target_fpfh);
sac_ia.setMinSampleDistance(0.1);//设置样本之间的最小距离
sac_ia.setCorrespondenceRandomness(6); //在选择随机特征对应时，设置要使用的邻居的数量;
pointcloud::Ptr align(new pointcloud);
sac_ia.align(*align);
2.4 Principal Component Analysis (PCA) Registration
1 principle
The main axis direction of the point cloud data is mainly used for registration. First calculate the covariance matrix of the two sets of point clouds, calculate the main characteristic components according to the covariance matrix, that is, the main axis direction of the point cloud data , and then calculate the rotation matrix through the main axis direction, and calculate the convenient shift of the center coordinates of the two sets of point clouds Find the translation vector.
2 core code
void ComputeEigenVectorPCA(const pcl::PointCloud<pcl::PointXYZ>::Ptr& cloud, Eigen::Vector4f& pcaCentroid, Eigen::Matrix3f& eigenVectorsPCA)
{
pcl::compute3DCentroid(*cloud, pcaCentroid);
Eigen::Matrix3f covariance;
pcl::computeCovarianceMatrixNormalized(*cloud, pcaCentroid, covariance);
Eigen::SelfAdjointEigenSolver<Eigen::Matrix3f> eigen_solver(covariance, Eigen::ComputeEigenvectors);
eigenVectorsPCA = eigen_solver.eigenvectors();
}
2.5 ICP registration
1 principle
The core of the ICP algorithm is to minimize an objective function, which is actually the sum of the squares of the Euclidean distances between all corresponding points .
2 steps
**①Looking for corresponding points: **We assume that we use the initial transformation matrix to transform the source cloud when there is an initial value, and compare the transformed point cloud with the target cloud, as long as the distance between the two point clouds is less than a certain Threshold, we think that these two points are the corresponding points.
**②R, T optimization: **After having the corresponding points, we can use the corresponding points to estimate the rotation R and translation T. Here there are only 6 degrees of freedom in R and T, and our number of corresponding points is huge. Therefore, we can use methods such as least squares to solve the optimal rotation-translation matrix, a numerical optimization problem.
**③Iteration:**We have optimized and obtained a new R and T, resulting in changes in the positions of some points after conversion, and corresponding changes in some corresponding points. Therefore, we return to the method of finding corresponding points in step ②. ②Steps are iteratively carried out until some iteration termination conditions are met, such as the change of R and T is less than a certain value, or the change of the above objective function is less than a certain value, or the corresponding point does not change anymore.
3 core code
icp.setInputSource(source); // 源点云
icp.setInputTarget(target); // 目标点云
icp.setTransformationEpsilon(1e-10); // 为终止条件设置最小转换差异
icp.setMaxCorrespondenceDistance(1); // 设置对应点对之间的最大距离（此值对配准结果影响较大）。
icp.setEuclideanFitnessEpsilon(0.05); // 设置收敛条件是均方误差和小于阈值， 停止迭代；
icp.setMaximumIterations(35); // 最大迭代次数
pcl::PointCloud<pcl::PointXYZ>::Ptr icp_cloud(new pcl::PointCloud<pcl::PointXYZ>);
icp.align(*icp_cloud);
3 Registration Algorithms for Deep Learning
①PointNetLK (Deep ICP) is an improved version of the ICP algorithm based on PointNet. PointNet is used to extract the global features of the point cloud, and then iteratively approximates the similarity transformation parameters using Newton's method, and uses the point-to-point mapping estimated in this process to update the weights.
②Deep Closest Point (DCP) is a point cloud registration algorithm based on deep neural network. It first extracts features through PointNet, then calculates the nearest neighbor point of each point in the target point cloud, and calculates the distance between these two points. After that, it passes this information to a shape encoder to learn to find the optimal registration relationship between the two point clouds, and outputs the transformation matrix to make the two point clouds coincide.
③PRNet PRNet is a point cloud registration algorithm based on PointNet++. Its main idea is to project two point clouds onto a sphere, and then calculate the convolution features on this sphere. After the convolution is completed, PRNet uses the coarse registration stage for initial registration, then uses RANSAC for fine registration, and finally outputs the registration matrix.
④PPFNet PPFNet is a point cloud registration algorithm based on local point pair features (PPF), which uses a neural network to learn the relative transformation between point pairs, and outputs a transformation matrix to align the two point clouds. This algorithm encodes point clouds using a convolutional neural network and learns the features of the PPF matching relationship, and uses the trained network to register new point cloud pairs.
4 References
https://blog.csdn.net/qq_36686437/article/details/114160640
https://www.zhihu.com/tardis/bd/art/506823350?source_id=1001