ORB_SLAM3 adds fisheye camera support for the first time. If we want to use our own fisheye camera image sequence to test the corresponding effect, we need to do certain preparation and processing:
Table of contents
1. Download and compile ORB_SLAM3
1.2 Compilation: directly ./build.sh is stuck
4. Monocular fisheye sequence requirements
5. Run your own monocular sequence
1. Download and compile ORB_SLAM3
1.1 Installation
Official website address:
Find a suitable folder and run the terminal:
git clone https://github.com/UZ-SLAMLab/ORB_SLAM3.git ORB_SLAM3The downloaded folder name is ORB_SLAM3.
There are libraries that need to be installed on github. You can refer to my tutorial:
Ubuntu upgrade eigen to 3.4.0_eigen3.4_Qianmengyu 11's blog-CSDN blog
Installing Sophus on ubuntu_Qianmengyu 11's blog-CSDN blog
Installing the Pangolin library on ubuntu16.04_Qianmengyu 11's blog-CSDN blog
Ubuntu16.04 C++ installation opencv tutorial_Qianmengyu 11's blog-CSDN blog
Note: Eigen cannot install 3.4.0 , and an error of TensorXXX may be reported. Most of the errors on the Internet are that there is no file, but if 3.4.0 is installed, the error is: Eigen/CXX11/src/Tensor/TensorBlock.h:1303:7: error: the value of 'j' is not usable in a constant expression
In fact, it is because the version does not correspond to tensorflow. After searching for a long time, I found someone who said that it is OK to install version 3.3.7. No error is reported after installation.
1.2 Compilation: directly ./build.sh is stuck
Official website command:
cd ORB_SLAM3
chmod +x build.sh
./build.shBut this ordinary computer is prone to jamming. You can actually open the build.sh file in the main directory of the ORB_SLAM3 folder: This file is the compiled summary file, and its content is:
echo "Configuring and building Thirdparty/DBoW2 ..."
cd Thirdparty/DBoW2
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j
cd ../../g2o
echo "Configuring and building Thirdparty/g2o ..."
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j
cd ../../Sophus
echo "Configuring and building Thirdparty/Sophus ..."
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j
cd ../../../
echo "Uncompress vocabulary ..."
cd Vocabulary
tar -xf ORBvoc.txt.tar.gz
cd ..
echo "Configuring and building ORB_SLAM3 ..."
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j4In fact, the compilation of the first few libraries is very easy, but the make -j4 of ORBSLAM3 in the last line is easy to get stuck.
Just change the last line to make without options. Compilation should be slower, but it won't freeze.
2. Run the EuRoC data set
2.1 Data set download
Refer to my blog: ORB_SLAM2 download, compile and run the EuRoC data set_euroc data set download_Qianmengyu 11's blog-CSDN blog
The official website address of the data set is:
kmavvisualinertialdatasets – ASL Datasets
Just download an ASL Dataset Format. Just unzip a mav0 folder. For example, we download the Machine Hall01 data set and extract it to the MH01 folder.
2.2 Running the data set
To run these datasets you can use the examples in the Examples folder. Using the executable mono_euroc
The executable file mono_euroc has four input parameters: vocabulary, camera configuration file yaml, data set sequence address, and data set corresponding timestamp file (.txt). The code is:
./Examples/Monocular/mono_euroc 词库文件.txt 配置文件.yaml 数据集序列 数据集对应时间戳.txtAmong them, the vocabulary file can be the ORBvoc.txt file that comes with it and is located in the ORB_SLAM3/Vocabulary folder. When this file is compiled, there is a line of command to decompress the compressed file in this folder, and the decompressed file is the thesaurus file. Other subsequent data sets can also use this file.
Fill in the code as:
./Vocabulary/ORBvoc.txtThere are many camera configuration files yaml in the ORB_SLAM3/Examples/Monocalur/ folder, which will describe some types and parameters of the camera. Most of them are monocular cameras. The RealSense_T265.yaml file contains fisheye cameras (Camera. type: "KannalaBrandt8"). In the EuRoC data set, just use the EuRoC.yaml file, the content is:
The corresponding should be filled in:
./Examples/Monocular/EuRoC.yamlThe data set address can be written to the upper level of the mav0 folder. For example, if we decompress mav0 to the MH01 folder, the corresponding address that should be entered is
/home/ /xxxxx/MH01The timestamp file is a .txt file. There are two data sets (RuRoC and TUM) in the ORB_SLAM3/Examples/Monocalur/ folder, which are placed in the EuRoC_TimeStamps and TUM_TimeStamps folders respectively. This section uses the MH01 data set of the EuRoC data set, and the corresponding file is:
./Examples/Monocular/EuRoC_TimeStamps/MH01.txt2.3 Run command
Summarized, the total command is:
./Examples/Monocular/mono_euroc ./Vocabulary/ORBvoc.txt ./Examples/Monocular/EuRoC.yaml 数据集地址/MH01 ./Examples/Monocular/EuRoC_TimeStamps/MH01.txt3. Run the TUM data set
3.1 TUM data set download
Official website:
Computer Vision Group - Datasets - Visual-Inertial Dataset
The 512x512 of corridor4 I downloaded is the fourth file in the fourth line. Unzip to the corresponding folder and extract a dataset-corridor4_512_16 folder:
3.2 Dataset operation
The data set uses the executable file mono_tum_vi. Similar to the previous one, there are also four parameters.
./Examples/Monocular/mono_tum_vi 词库文件.txt 配置文件.yaml 数据集序列 数据集对应时间戳.txtThe first one remains unchanged, and the second configuration file is changed to the ./Examples/Monocular/TUM1.yaml file in the same folder.
The data set sequence needs to be specific to the data in cam1 (a folder in mav0). The corresponding data set timestamp file is in the ./Examples/Monocular/TUM_TimeStamps/ folder and is selected based on the 512 resolution version of the corresponding corridor4 data set. txt file is:
./Examples/Monocular/TUM_TimeStamps/dataset-corridor4_512.txt3.3 Run command
The final running command is:
./Examples/Monocular/mono_tum_vi ./Vocabulary/ORBvoc.txt ./Examples/Monocular/TUM1.yaml 数据集地址/dataset-corridor4_512_16/mav0/cam1/data ./Examples/Monocular/TUM_TimeStamps/dataset-corridor4_512.txtBecause the picture name corresponding to the timestamp is in the cam1 folder, not in cam0. The TUM data set is data from a binocular fisheye camera + IMU.
Just change the data set address.
4. Monocular fisheye sequence requirements
Through the previous operations, we can actually see that the changes are basically the data set address and the data set timestamp file. Of course, the camera file also needs to be changed, to the parameters corresponding to the camera.
The monocular fisheye camera uses the executable file mono_tum_vi. The required data set is a folder full of png images. The requirements are:
1. The file contains png images.
2. The png image is named with a 20-digit timestamp (the actual name is similar to 12345678912345678900.png)
Then the actual corresponding timestamp file (.txt) contains the timestamp, one 20-digit timestamp per line. The actual running timestamp + '.png' is the picture name.
Then we need to prepare a folder to put the stored pictures, and require PNG pictures with 20-digit names, similar to:
Then create a txt with the following content: (If the timestamp is actual, use the actual one, otherwise only the pose transformations of different pictures are accurate)
...
00000000123400000000
00000000123500000000
00000000123600000000
00000000123700000000
...Assume that the data set folder is: /home/xxx-username/slam/000
The timestamp file is placed in the newly created my_TimeStamps folder in the original ./Examples/Monocular/ folder.
The configuration file is my.yaml
5. Run your own monocular sequence
The corresponding running command is:
./Examples/Monocular/mono_tum_vi ./Vocabulary/ORBvoc.txt ./Examples/Monocular/配置文件.yaml 数据集地址 时间戳文件.txtThe position of reference four is:
./Examples/Monocular/mono_tum_vi ./Vocabulary/ORBvoc.txt ./Examples/Monocular/my.yaml /home/xxx-username/slam/000 ./Examples/Monocular/my_TimeStamps/my.txtThe running result is:
Finally, there is the average time and so on.
The generated trajectory files are: CameraTrajectory.txt and KeyFrameTrajectory.txt in the ORB_SLAM3 main folder. Among them, taking CameraTrajectory.txt as an example, there are 8 data in one line, represented in the code:
lT twc(0) twc(1) twc(2) q.x q.y q.z q.wAmong them, lT is the timestamp, q is the unit quaternion, and twc is the translation vector.