Introduction to ORB-SLAM series algorithms (comprehensive version)

1. Reference materials

ORB-SLAM2 Detailed Explanation (1) Introduction
(Open Class) Visual SLAM Principles and ORB-SLAM3 Series Algorithms

2. Related introduction

1. Introduction to ORB

ORB refers to a rotation invariant feature.

2. Evolution of ORB-SLAM series algorithms

Insert image description here

3. ORB-SLAM

Paper:ORB-SLAM: A Versatile and Accurate Monocular SLAM System
github code repository:ORB -SLAM
ORB-SLAM official website:ORB-SLAM

1. Introduction to ORB_SLAM

In 2015, Raul Mur-Artal et al proposed aSparse mapping system ORB_SLAM based on feature points, the entire system is implemented based on ORB features. ORB-SLAM tooReal-time monocular SLAM system, can operate in large-scale, small-scale, indoor and outdoor environments. The system is also robust to severe motion and supports wide-baseline closed-loop detection and relocalization, including fully automatic initialization.

The latest progress of ORB_SLAM is semi-dense scene reconstruction based on key frames of ORB_SLAM. New research results will continue to pay attentionORB-SLAM Project homepage.
Insert image description here

2. ORB-SLAM system architecture

It can be seen from the ORB-SLAM system architecture that ORB-SLAM is mainly divided into three threads, namelyTracking, < /span> File. and , . The ORB-SLAM2 project is very clear and beautiful. The three threads are stored in the corresponding three files, namely LoopClosing, LocalMappingTracking.cppLocalMapping.cppLoopClosing.cpp
Insert image description here

3. ORB-SLAM three threads

3.1 Tracking

The main work of this part is to extract ORB features from the image, perform pose estimation based on the previous frame, or initialize the pose through global relocation, then track the reconstructed local map, optimize the pose, and then determine the new pose according to some rules. Keyframe.

3.2 Mapping (LocalMapping)

This part mainly completes the construction of local maps. It includes inserting keyframes, verifying and filtering recently generated map points, then generating new map points, using local bundle adjustment (Local BA), and finally filtering the inserted keyframes to remove redundant keyframes.

3.3 LoopClosing (LoopClosing)

This part is mainly divided into two processes, namely loop detection and loop correction. Loop-closing detection first uses WOB for detection, and then calculates similarity transformation through Sim3算法; loop-closing correction is mainly loop-closing fusion and graph optimization of Essential Graph.

4. ORB-SLAM2

论文：ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras
github store： CSDN 专栏：ORB SLAM2 CSDN 专栏：ORB-SLAM2 CSDN专栏：视觉SLAM

1. Introduction to ORB-SLAM2

In 2017, based on ORB-SLAM, Raul Mur-Artal and others proposed an improved version of ORB-SLAM, which extended from monocular cameras to binocular and RGB-D cameras. ORB-SLAM2 utilizes far key points and near key points to improve the accuracy of the algorithm when using binocular cameras.

ORB-SLAM2 is a complete SLAM solution that can realize map reuse, loop detection and relocation. ORB-SLAM2 employs a monocular and binocular based approach on the backend that allows trajectory accuracy assessment on a metric scale. In addition, ORB-SLAM2 includes a lightweight positioning mode that can use visual odometry to track unmapped areas and match feature points while allowing zero-point drift. 光束法平差优化（BA）

2. ORB-SLAM2 system architecture

Insert image description here

3. Shortcomings of ORB-SLAM2

When encountering weak textures (for example: white walls), the features extracted by ORB-SLAM2 are insufficient, resulting in tracking loss.

5. ORB-SLAM-VI

1. Introduction to ORB-SLAM-VI

In 2017, based on ORB-SLAM, Raul Mur-Artal and others proposed a new tightly coupled visual-inertial ORB-SLAM-VI system, adding $unit\textcolor{Red}{Inertial measurement unit}$ .

Aiming at the lack of scale information in monocular SLAM, a novel IMU initialization method was proposed to Calculatescale, gravity direction, velocity, and gyroscope and accelerometer biases quickly and with high precision, and reuse maps to achieve zero-drift positioning in reconstructed areas.

2. ORB-SLAM-VI system architecture

Insert image description here

6. ORB-SLAM-Atlas

论文：ORBSLAM-Atlas: a robust and accurate multi-map system（IROS 2019）

1. Introduction to ORB-SLAM-Atlas

In 2019, based on ORB-SLAM2, Raul Mur-Artal and others proposeda system that can handle an unlimited number of non-connected sub-maps , including a robust map merging algorithm, does not stop updating the map when camera tracking is lost, and immediately builds a new sub-map.

Compared with ORB-SLAM2, ORB-SLAM-Atlas mainly works as follows:

Use Atlas to solve unlimited sub-map fusion problems. Atlas has an unlimited number of word bag databases of sub-map key frames, ensuring the efficiency of map scene re-identification;
Multi-map operation algorithm: new map generation, relocation in hybrid maps and map fusion;

2. ORB-SLAM-Atlas system architecture

Insert image description here

7. ORB-SLAM3

Paper:ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM
github code repository: ORB_SLAM3
Detailed explanation of the overall process of ORB-SLAM3
A detailed explanation of ORB-SLAM3

1. Introduction to ORB-SLAM3

ORB-SLAM3 is a support $\textcolor{Red}{Vision, vision + inertial navigation, hybrid map}$ can be used in $cameras\textcolor{Red}{ Monocular, binocular and RGB-D cameras}$ Usepinholeor a> runs. ORB-SLAM3 can run robustly in real-time in large scenes/small scenes, indoors/outdoors. Fisheye camera model

Compared with ORB-SLAM2, ORB-SLAM3 adds support forIMU fusion, is compatible with the fisheye camera model, and adds Atlas multi-map support; at the same time, in order to support the multi-map mode, loopback detection provides a method called Optimization method of Welding BA.

2. ORB-SLAM3 system architecture

Insert image description here

3. Advantages of ORB-SLAM3

In ORB-SLAM2, tracking loss is prone to occur in weak texture scenes, while ORB-SLAM3 can reconstruct the map and achieve automatic relocation.

4. Experience with ORB-SLAM3

For indoor scenes, ORB-SLAM3 has high positioning accuracy.

For rich outdoor scenes, ORB-SLAM3 can be used directly.

For UAV navigation, due to the small number of feature points in the sky, the effect of ORB-SLAM3 is slightly poor, and multi-modal information (GPS) needs to be integrated to improve positioning and navigation accuracy.

For driverless driving, lidar needs to be integrated to improve the accuracy of positioning and navigation.

8. Project experience

Turbot-VSLAM Getting Started Tutorial - Implementing ORB-SLAM V2
Turbot-VSLAM Getting Started Tutorial - Implementing RGBD-SLAM-V2
ORB_SLAM running detailed process (Ubuntu14.04 system and ROS Indigo environment construction, configuration and test operation)

EuRoC

EuRoC dataset

The EuRoc data set is a visual inertial navigation data set collected on a Micro Aerial Vehicle (MAV). The dataset contains: stereo images, synchronized IMU measurements, precise motion and ground-truth of structures.