Cainiao College: Create your own SLAM system from start to finish
Love to be in a daze de cabbage headRobotSlamApplication project two: small SLAM system
Research background: Because I used to be more impetuous, I always like to study other people's libraries and then test to run through. If the effect is good, I will use it to modify, and then use it for testing. If the effect is not good, I will discard it. I didn't really concentrate on writing a system before. This time I wrote a SLAM system from start to finish. The writing process is really not easy, and I feel much more comfortable after writing. .
I. Introduction
Advantages and disadvantages of RGB-D camera (structured light depth camera)
advantage:
- Suitable for use in scenes with insufficient lighting and lack of texture;
- High measurement accuracy can be achieved within a certain range;
- The technology is mature, and the depth image can achieve higher resolution;
Disadvantages:
- Basically cannot be used in outdoor environment;
- The measurement distance is relatively close;
- It is easily affected by reflections from smooth surfaces.
Second, the pose estimation of the camera
Obtain the transformation between different frames of images based on the ORB feature point method (this method has been discussed by many experts, so I won't say more here), which mainly involves the following parts:
1. Feature point extraction and matching:
In the process of acquiring image data in real time, this link needs to pay attention to the acquisition of image frames. The frames here include the current frame and the reference frame (that is, the image frame at the previous moment). When we calculate the matching of the image, we need to distinguish between different The frame at the moment to calculate the image transformation between adjacent frames.
That is, at the initial moment, the current frame is assigned to the reference frame, and then in the camera motion tracking process, after each image matching and tracking is performed, the current frame must be converted and assigned to the reference frame to achieve "frame "'S update.
case INITIALIZING:
{
state_ = OK;
curr_frame = ref_frame = frame;
....
break;
}
case OK:
{
curr_frame = frame;
....
if ( CameraPoses is OK ... )
{
curr_frame->T_c_w_ = T * ref_frame->T_c_w_;
ref_frame = curr_frame;
....
}
else
{
.....
}
}
There are also many ways to estimate the camera pose: 2d-2d (calculated and solved by essential matrix and homography matrix), 3d-2d (solved by PnP), 3d-3d (solved by SVD+BA).
Among these methods, the PnP method is more commonly used, and the PnP solution is relatively simple, and you can directly call the OpenCV method.
bool solvePnPRansac( InputArray objectPoints, InputArray imagePoints,
InputArray cameraMatrix, InputArray distCoeffs,
OutputArray rvec, OutputArray tvec,
bool useExtrinsicGuess = false, int iterationsCount = 100,
float reprojectionError = 8.0, double confidence = 0.99,
OutputArray inliers = noArray(), int flags = SOLVEPNP_ITERATIVE );
bool solvePnP( InputArray objectPoints, InputArray imagePoints,
InputArray cameraMatrix, InputArray distCoeffs,
OutputArray rvec, OutputArray tvec,
bool useExtrinsicGuess = false, int flags = SOLVEPNP_ITERATIVE );
I tested the test set of tum to get the camera motion trajectory (visual odometer: VO) as shown below:
Three, map creation 1
After completing the visual odometry module, the next step is to create and update the map. Due to time constraints, I want to complete the entire SLAM system, so there is no in-depth debugging for the optimization part of the map. At present, only the creation and update of the preliminary map have been completed. The first step is to construct a point cloud map as follows:
Is it crazy to see a map like this? . .
Don't worry, after careful comparison and debugging, it is found that the transformation matrix between the current frame and the reference frame does not correspond well, resulting in an error in the projection of the map point. After the transformation matrix is modified, the following is obtained:
Isn’t that much more attractive~~~.
Fourth, map creation 2
After the above steps, the basic real-time map update function has been completed, but the map has not been optimized and the corresponding loop detection part has not been optimized. This will continue to be improved in the future, but before that, I still want to try it Real-time creation of octomap maps. . Well, I started to do it when I thought about it, so I got the following otomap map:
OK, successfully completed the writing test of the small SLAM.
Five, experience
After completing the writing and testing of the above system, you can clearly and accurately grasp each link in the SLAM process. This is very important for students who are just starting to get started with SLAM. Not only that, for myself, I can learn a lot from the overall process of SLAM. Next, I will focus on the back-end optimization part. Loop detection is very important.
Moreover, after going through the SLAM process, I began to enter the application and improvement of deep learning in SLAM from these important parts, keep on going~~
references
RobotSlamApplication project two: small SLAM system
Research background: Because I used to be more impetuous, I always like to study other people's libraries and then test to run through. If the effect is good, I will use it to modify, and then use it for testing. If the effect is not good, I will discard it. I didn't really concentrate on writing a system before. This time I wrote a SLAM system from start to finish. The writing process is really not easy, and I feel much more comfortable after writing. .
I. Introduction
Advantages and disadvantages of RGB-D camera (structured light depth camera)
advantage:
- Suitable for use in scenes with insufficient lighting and lack of texture;
- High measurement accuracy can be achieved within a certain range;
- The technology is mature, and the depth image can achieve higher resolution;
Disadvantages:
- Basically cannot be used in outdoor environment;
- The measurement distance is relatively close;
- It is easily affected by reflections from smooth surfaces.
Second, the pose estimation of the camera
Obtain the transformation between different frames of images based on the ORB feature point method (this method has been discussed by many experts, so I won't say more here), which mainly involves the following parts:
1. Feature point extraction and matching:
In the process of acquiring image data in real time, this link needs to pay attention to the acquisition of image frames. The frames here include the current frame and the reference frame (that is, the image frame at the previous moment). When we calculate the matching of the image, we need to distinguish between different The frame at the moment to calculate the image transformation between adjacent frames.
That is, at the initial moment, the current frame is assigned to the reference frame, and then in the camera motion tracking process, after each image matching and tracking is performed, the current frame must be converted and assigned to the reference frame to achieve "frame "'S update.
case INITIALIZING:
{
state_ = OK;
curr_frame = ref_frame = frame;
....
break;
}
case OK:
{
curr_frame = frame;
....
if ( CameraPoses is OK ... )
{
curr_frame->T_c_w_ = T * ref_frame->T_c_w_;
ref_frame = curr_frame;
....
}
else
{
.....
}
}
There are also many ways to estimate the camera pose: 2d-2d (calculated and solved by essential matrix and homography matrix), 3d-2d (solved by PnP), 3d-3d (solved by SVD+BA).
Among these methods, the PnP method is more commonly used, and the PnP solution is relatively simple, and you can directly call the OpenCV method.
bool solvePnPRansac( InputArray objectPoints, InputArray imagePoints,
InputArray cameraMatrix, InputArray distCoeffs,
OutputArray rvec, OutputArray tvec,
bool useExtrinsicGuess = false, int iterationsCount = 100,
float reprojectionError = 8.0, double confidence = 0.99,
OutputArray inliers = noArray(), int flags = SOLVEPNP_ITERATIVE );
bool solvePnP( InputArray objectPoints, InputArray imagePoints,
InputArray cameraMatrix, InputArray distCoeffs,
OutputArray rvec, OutputArray tvec,
bool useExtrinsicGuess = false, int flags = SOLVEPNP_ITERATIVE );
I tested the test set of tum to get the camera motion trajectory (visual odometer: VO) as shown below:
Three, map creation 1
After completing the visual odometry module, the next step is to create and update the map. Due to time constraints, I want to complete the entire SLAM system, so there is no in-depth debugging for the optimization part of the map. At present, only the creation and update of the preliminary map have been completed. The first step is to construct a point cloud map as follows:
Is it crazy to see a map like this? . .
Don't worry, after careful comparison and debugging, it is found that the transformation matrix between the current frame and the reference frame does not correspond well, resulting in an error in the projection of the map point. After the transformation matrix is modified, the following is obtained:
Isn’t that much more attractive~~~.
Fourth, map creation 2
After the above steps, the basic real-time map update function has been completed, but the map has not been optimized and the corresponding loop detection part has not been optimized. This will continue to be improved in the future, but before that, I still want to try it Real-time creation of octomap maps. . Well, I started to do it when I thought about it, so I got the following otomap map:
OK, successfully completed the writing test of the small SLAM.
Five, experience
After completing the writing and testing of the above system, you can clearly and accurately grasp each link in the SLAM process. This is very important for students who are just starting to get started with SLAM. Not only that, for myself, I can learn a lot from the overall process of SLAM. Next, I will focus on the back-end optimization part. Loop detection is very important.
Moreover, after going through the SLAM process, I began to enter the application and improvement of deep learning in SLAM from these important parts, keep on going~~