Table of contents
1. Install the evo tool
The Python environment supported by the latest evo tool Python 3.8+is the following two methods to install the latest version of evo:
1. The command line uses the pip tool to install directly
pip install evo --upgrade --no-binary evo
2. Source code installation
git clone https://github.com/MichaelGrupp/evo.git
cd evo
pip install --editable . --upgrade --no-binary evo
2. Run the RGB-D benchmark dataset
1. Download the TUM dataset
TUM dataset official website download address , there are depth images and RGB images in the TUM dataset. When using the monocular mode, only the pictures in the rgb folder are used; when using the RGB-D mode, the pictures in the depth and rgb folders are needed, and the associate.py script file is required to associate the RGB image with the depth image. Let's take rgbd_dataset_freiburg3_long_office_householda dataset as an example.
2. Associate the rgb image and the depth image
In this step, we need to use the association files of the RGB image and the depth image. The author of ORB-SLAM2 provides the association files of some sequences under the path of examples/RGB-D/associations/. We can also use the python script associate.py provided by TUM's official website to associate RGB images and depth images. Put the associate.py script in the downloaded and decompressed rgbd_dataset_freiburg3_long_office_household/ path, and execute the following command:
python associate.py rgb.txt depth.txt > associations.txt
The meaning of the above command is to align the time stamps of the color image and the depth image, and then write the correlation result to associations.txt.
3. Run the rgb-d dataset
Next, execute the run command, which is the most familiar:
./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUMX.yaml PATH_TO_SEQUENCE_FOLDER ASSOCIATIONS_FILE
Change TUMX.yaml to TUM1.yaml, TUM2.yaml or TUM3.yaml for freiburg1, freiburg2 and freiburg3 sequences respectively. Change PATH_TO_SEQUENCE_FOLDERto the unzipped dataset folder path. will ASSOCIATIONS_FILEbe changed to the path of the corresponding associated file.
In simple terms, the command is divided into five parts:
- TUM RGB-D executable file
- trained bag of words
- parameter file
- Decompressed dataset path
- Associated file path
After the rgbd_tum program finishes running, it will save the camera and key frame track files to the root directory of ORB_SLAM2.
3. Use evo to evaluate
1. Common commands
Draw a single trajectory, groundtruth.txt is the real trajectory of the dataset, -prepresenting the drawn image.
evo_traj tum groundtruth.txt -p
The real trajectory image of the camera in the data set in the three-dimensional space, there is a closed loop, and the unit is m.
The view of the xyz coordinate system, showing the x, y, z coordinates of the camera at each time, the unit of the vertical axis is m, and the unit of the horizontal axis is s.
The rpy view displays the r, p, and y coordinates of the camera at each time. The unit of the vertical axis is deg, and the unit of the horizontal axis is s.
Draw multiple trajectories: the real one and the camera one just run.
evo_traj tum groundtruth.txt CameraTrajectory.txt -p
The above trajectory is not aligned in rotation and translation, you can --refspecify the reference trajectory through parameters, and add the parameter -a to align the trajectory
evo_traj tum --ref=groundtruth.txt CameraTrajectory.txt -p -a
It can be seen that the running trajectory is almost the same as the real trajectory, and there are no two lines. ORB_SLAM2's powerful back-end optimization and closed-loop are reflected here.
The trajectory diagrams are so consistent, the xyz coordinates must be very accurate, and the z coordinate error is relatively large.
In rpy coordinates, y is very accurate, but r and p have large errors.
If it is a monocular program, you also need to add a parameter -sfor alignment on the scale
evo_traj tum --ref=groundtruth.txt CameraTrajectory.txt -p -a -s
Or use parameters directly -aswith rotation and translation and alignment on scale
evo_traj tum --ref=groundtruth.txt CameraTrajectory.txt -p -as
By observation, the above trajectory does not change much on the x and y axes, and parameters can be added --plot_mode=xzto compress the trajectory on the xz plane.
evo_traj tum --ref=groundtruth.txt CameraTrajectory.txt -p -as --plot_mode=xz
In most cases, you still choose to project the trajectory onto the xy plane, which is the top view.
Through the above practice, we summarize the commonly used command formats as follows:
evo_traj tum --ref=groundtruth.txt CameraTrajectory.txt -p -as
evo_traj tum --ref=groundtruth.txt CameraTrajectory.txt -p -as --plot_mode=xy
2. Absolute trajectory error
evo_ape tum groundtruth.txt CameraTrajectory.txt -p -as
max: Indicates the maximum error; mean: average error; median: median error; min: minimum error; rmse: root mean square error; sse: sum variance/sum of squares of error; std: standard deviation.
APE w.r.t. translation part (m)
(with Sim(3) Umeyama alignment)
max 0.025939
mean 0.009091
median 0.008388
min 0.000931
rmse 0.009895
sse 0.243401
std 0.003906
3. Relative pose error
The same as the absolute trajectory error parameter, the reason for using the camera trajectory is that it contains the keyframe trajectory, which reflects a globally consistent map.
evo_rpe tum groundtruth.txt CameraTrajectory.txt -p -as
RPE w.r.t. translation part (m)
for delta = 1 (frames) using consecutive pairs
(with Sim(3) Umeyama alignment)
max 0.030021
mean 0.004404
median 0.003749
min 0.000176
rmse 0.005210
sse 0.067444
std 0.002783
A photo is essentially a projection of the scene on the imaging plane of the camera when the photo is taken. It reflects the three-dimensional world in two-dimensional form. Obviously, this process loses one dimension of the scene, the so-called depth (or distance).