[Algorithm] Problems encountered in running ORB-SLAM3, solutions, and effect display (environment: Ubuntu18.04+ROS melodic)

1. ./build.shVarious problems in compiling ORB-SLAM3

1. Problem: OpenCV > 4.4 not found

Compiling build.sh reports an error: OpenCV > 4.4 not found

Solution
(1) Find the location in CMakeList.txt under the ORB_SLAM3 folder find_package(OpenCV 4.4)and modify it to:

LIST(APPEND CMAKE_MODULE_PATH ${PROJECT_SOURCE_DIR}/cmake_modules)
 
find_package(OpenCV 3 REQUIRED)
   if(NOT OpenCV_FOUND)
      message(FATAL_ERROR "OpenCV > 4.4 not found.")

2、问题：error: ‘slots_reference’ was not declared in this scope

The bug caused by the latest version of Pangolin, error error; if you run ORB-SLAM2, ORB-SLAM3, do not use the latest version of Pangolin to compile and install, it will cause error: 'slots_reference' was not declared in this scope or cannot find Eigen3.cmake, I I just installed the latest version and reported a lot of errors when I compiled the ORB-SLAM2 library. It took a day to change the environment and the code, but I didn’t get it right. Finally, I found out that this is the problem. I want to cry without tears www

Solution

1. Uninstall the latest version of Pangolin, enter the build of the Pangolin folder, and execute
sudo make uninstall

2. Install Pangolin v0.5.

Installation link: https://github.com/stevenlovegrove/Pangolin/tree/v0.5

3. Installation steps
cd Pangolin
mkdir build && cd build
cmake ..
cd ..
cmake --build .
sudo make install

Reference link: https://codingnote.cc/p/484790/

2. ORB-SLAM3 effect display

1. ./build.shSuccessful compilation

2. Found that there is no euroc_examples.sh file in the source code

Finally build.sh compiled to test the data, and then found that there is no euroc_examples.sh in the source code, but the README said:

directly split, empty examples, a search found that a colleague said that it is in the new version of the ORB-SLAM3 library No more, so I hurriedly looked for anyone who had downloaded the euroc_examples.sh file, and then found a github with this file in it. This github is still a detailed commented version of ORB-SLAM3. If you want to understand the code in the future, it is indispensable. It, put the link here first

https://github.com/electech6/ORB_SLAM3_detailed_comments

3. Test the effect on the EuRoC dataset

(1) The EuRoC dataset is recorded by two pinhole cameras (pinhole) and an inertial sensor (IMU). The following only shows some results of some video sequences tested

(2) Video sequence: MH_01_easy

MH_01_easy binocular effect

Eval command line example:

echo "Launching MH01 with Stereo sensor"
./Examples/Stereo/stereo_euroc ./Vocabulary/ORBvoc.txt ./Examples/Stereo/EuRoC.yaml "$pathDatasetEuroc"/MH01 ./Examples/Stereo/EuRoC_TimeStamps/MH01.txt dataset-MH01_stereo
echo "------------------------------------"
echo "Evaluation of MH01 trajectory with Stereo sensor"
python evaluation/evaluate_ate_scale.py evaluation/Ground_truth/EuRoC_left_cam/MH01_GT.txt f_dataset-MH01_stereo.txt --plot MH01_stereo.pdf

和GroundTruth轨迹对比的Eval结果：
absolute_translational_error.rmse 0.025333 m
absolute_translational_error.mean 0.022465 m
absolute_translational_error.median 0.022143 m
absolute_translational_error.std 0.011709 m
absolute_translational_error.min 0.000709 m
absolute_translational_error.max 0.072913 m

(3) Video sequence: MH_03_medium

MH_03_medium binocular effect

MH_03_medium monocular+IMU effect

(3) Video sequence: V1_01_easy

V1_01_easy binocular effect
V1_01_easy monocular + IMU effect

(4) Possible problems when using IMU data:

(4-a) Failed to track the local map, need to wait for the SLAM system to reinitialize

Fail to track local map!
IMU is not or recently initialized.
Reseting active map...
SYSTEM-> Reseting active map in monocular case`

(4-b) Insufficient IMU accelerationnot enough acceleration

(5) Monocular, binocular, monocular + IMU, binocular + IMU accuracy comparison

Taking the MH_01_easy video sequence as an example, binocular or monocular + IMU has the highest accuracy, and the monocular deviation is outrageous (because there is no scale factor, there is zero drift)

4. Test the effect on the TUM-VI dataset

(1) The TUM-VI data set is recorded by two fisheye cameras (fisheye) and an inertial sensor (IMU). The following only shows some results of some video sequences tested

(2) Video sequence: dataset-room1_512_16

dataset-room1_512_16 monocular + IMU effect

dataset-room1_512_16 binocular + IMU effect

(3) Video sequence: dataset-corridor1_512_16

dataset-corridor1_512_16 monocular + IMU effect

dataset-corridor1_512_16 binocular + IMU effect

(4) Video sequence: dataset-magistrale1_512_16

dataset-magistrale1_512_16 monocular + IMU effect

(5) Video sequence: dataset-outdoors1_512_16

dataset-outdoors1_512_16 Monocular + IMU effect
→ A. Do not ignore distant feature points 20 meters away. When running to an outdoor big sky scene, it can be seen that there are many sky points in the red frame tracked by the algorithm, which will cause the algorithm to diverge. For map points The positioning failed

→ B. After ignoring the distant feature points 20 meters away, the final map building effect, I don’t know why it always shuts down at the end, and I can’t see the effect after the closed-loop detection. I have to find out the reason later,

dataset -outdoors1_512_16 binocular + IMU effect
→ A. (1) Do not ignore distant feature points 20 meters away. When running to an outdoor big sky scene, it can be seen that there are many sky points in the red frame tracked by the algorithm, which will cause the algorithm to diverge. The location of the map point failed

(2) Running to the back is getting more and more stuck. This is because the map is too large and the cache is full


→ B. (1) Ignoring the distant feature points 20 meters away, you can see the outdoor big sky scene Many sky points are filtered out, the algorithm will not diverge, and the matching points are concentrated within 20 meters

(2) This is what I ran. When the closed loop is not done in the far and large scene, there may be cumulative drift in the end

(3) I ran to In the end, I don’t know why there will always be Shutdown, and I can’t see the effect of the closed-loop detection. I have to find out the reason later. The final mapping effect can refer to other people’s results. (4) After ignoring the

feature points 20 meters away, the memory will not explode.

(6) Running problem: Fail to track local map!

Failed to track the local map, just wait for the system to execute Relocalize or re-build a new map

3. Other issues and local camera operation reference

https://blog.csdn.net/hhz_999/article/details/120473659
https://blog.csdn.net/qq_36804414/article/details/109241264
https://blog.csdn.net/qq_36804414/article/details/109248693