[20 years of VIO review]

1. Introduction to open source code: DSM

Direct Sparse Mapping (DSM) is a monocular visual SLAM system using the direct method, which can be regarded as an enhanced version of DSO. The biggest difference between DSM and DSO is that DSM can handle re-observation of map points. To this end, DSM draws on concepts such as local windows and common view relationships in ORB-SLAM2. The biggest advantage of DSM is that the code is well modularized. Each module and class of the program is well encapsulated, which is much clearer and easier to understand than DSO code. Moreover, there are many things that can be learned from the DSM code. For example, DSM will count the distribution of photometric errors of each frame in real time (using the Student t distribution to fit). DSM also uses memory pools and thread pools internally to manage memory resources and computing resources. Abstract and encapsulate. DSM is very suitable for friends who are interested in direct method to learn.

Code link
Paper link
Video display
Related references:
Literature Review: Direct Sparse Mapping based on sparse direct method Direct Sparse Mapping
Direct Sparse Odometry

Paper: Direct Sparse Mapping
Algorithm introduction: Previous direct method odometry, such as DSO, used Schur Complement to marginalize points and poses outside the sliding window, which resulted in Unable to handle re-observation of map points. This paper proposes a complete monocular direct method SLAM system, which can handle re-observation of map points, and achieves the most accurate results among the direct methods on the EuRoC data set. The contribution of this article is that it can handle the re-observation of map points, use the common view relationship to select key frames, use the coarse-to-fine strategy to do photometric optimization to increase the convergence radius, and use the t distribution to fit the photometric error for external points. manage.

2. FMD Stereo SLAM: Fusion of MVG and direct methods to achieve accurate and fast stereo SLAM

Title: FMD Stereo SLAM: Fusing MVG and Direct Formulation Towards Accurate and Fast Stereo SLAM
Author: Fulin Tang, Heping Li, Yihong Wu
Source: 2019 IEEE International Conference on Robotics and Automation (ICRA), May 20-24, 2019, Montreal, Canada
(Chinese Academy of Sciences, combination of feature point method and direct method)
Bubble robot description:

Abstract description:
We propose a novel binocular vision SLAM framework that takes into account both accuracy and speed. This framework takes full advantage of key feature-based multi-view geometry (MVG) and direct methods. On the front end, our system implements direct formulation and constant motion models to predict robust initial poses, reprojects local maps to find 3D-2D correspondences, and finalizes poses by minimizing reprojection errors. This front-end process makes our system faster. On the backend, MVG is used to estimate the 3D structure. When inserting new keyframes, new map points are generated through triangulation. To improve the accuracy of the proposed system, bad map points are removed and the global map is maintained through bundle adjustment. In particular, stereoscopic constraints are implemented to optimize the map. This back-end process makes our system more accurate. Experimental evaluation on the EuRoC dataset shows that the proposed algorithm can run at over 100 frames per second on consumer computers while achieving highly competitive accuracy.

3. SPVIS developed based on VINS-Mono

ICRA 2020 Hierarchical Quadtree Feature Optical Flow Tracking Based Sparse Pose-Graph Visual-Inertial SLAM

SPVIS was developed based on VINS-Mono. The front-end proposed a hierarchical quadtree method to improve the performance of optical flow feature tracking and significantly reduce the number of features required. The backend uses a fixed-duration sliding window for global optimization, and only the pose graph remains outside the window. The entire system is highly efficient, supporting long-term operation.

4. Improvement: a dynamic environment mobile robot positioning solution based on optical flow

标题：AnAccurate Localization Scheme for Mobile Robots Using Optical Flow in DynamicEnvironments
作者：Jiyu Cheng, Yuxiang Sun, Wenzheng Chi, Chaoqun Wang, Hu Cheng and Max Q.-H. Meng
来源：2018 IEEE International Conference on Robotics and Biomimetics

bubble robot

Abstract description:
A new method using optical flow to identify and eliminate dynamic feature points from extracted feature points is proposed. Static feature points are added to the visual SLAM algorithm for pose estimation. We integrate our method with the ORB-SLAM system and validate the proposed method using dynamic sequences in the TUM dataset. The entire system can run in real time. Qualitative and quantitative evaluations show that our method significantly improves the performance of visual SLAM in dynamic scenes.

Comment: Use the feature point transformation of the previous and next frames to calculate the optical flow to remove dynamic points.

5. PVIO: Efficient VIO based on a priori plane constraints

Paper: Jinyu Li, Bangbang Yang, Kai Huang, Guofeng Zhang, and Hujun Bao*. Robust and Efficient Visual-Inertial Odometry with Multi-plane Priors. PRCV 2019, LNCS 11859, pp. 283–295, 2019 .
Paper download link
Introduction
Code warehouse
Summary description:
In artificial environments, there are usually planar structures. If these planar information can be effectively utilized, it will help improve the robustness of VIO. To this end, we propose an efficient and robust lightweight VIO system PVIO based on multi-plane prior information. In this work, we propose a novel visual-inertial-planar PnP algorithm to achieve fast localization while reliably extending the plane based on reprojection consistency, thereby enhancing tracking robustness under high depth uncertainty. . In addition, an unstructured plane distance constraint term and corresponding marginalization and sliding window strategies are proposed in the sliding window optimization module, which can effectively reduce the computational complexity while ensuring good accuracy. Experimental results on the EuRoC and TUM-VI data sets show that PVIO is comparable in tracking accuracy to the current cutting-edge VI-SLAM system VINS-Mono, but it only requires a single thread to run, and has obvious advantages in efficiency.

6. Real-time graph-based SLAM with occupancy normal distribution transformation

Title: Real-Time Graph-Based SLAM with Occupancy Normal Distributions Transforms
Author: Cornelia Schulz and Andreas Zell
Source: 2020 IEEE International Conference on Robotics and Automation (ICRA)
2D laser SLAM based on NDT map has better accuracy and speed than the occupation version of baseline (Cartographer).
Bubble Robot
Abstract description:
In this work, we reformulate the (O)NDT matching function formatted so that it is a least squares problem that can be solved using a variety of robust numerical and analytical nonlinear optimizers. Furthermore, we propose a novel global (O)NDT scan matcher for loop closure. In our evaluation, our NDT and ONDT methods are able to outperform the occupancy grid graph-based method we adopted from the Google Cartographer implementation.

7. PL-SLAM: Binocular SLAM combining point and line features

Title: PL-SLAM: A Stereo SLAM System Through the Combination of Points and Line Segments
Author: Ruben Gomez-Ojeda, Francisco-Angel Moreno
Source: TRO 2019
Bubble Robot

Abstract description:
This paper proposes PL-SLAM, a binocular vision SLAM system that combines points and line segments to stably perform operations in a wider range of scenes. work, especially when point features are lacking or unevenly distributed in the image. PL-SLAM utilizes points and line segments in all instances of the process: visual odometry, keyframe selection, bundle adjustment, etc. We also provide a loop closing process through a novel bag-of-words approach that exploits the combined descriptive power of these two features.
Main contributions:
1. The first real-time open source deep SLAM system using both point and line segmentation features, and traditional point feature methods in weak texture environments It has high operational robustness in case of failure.
2. A new method that simultaneously processes point and line features to adjust keyword pose.
3. An extended BoW method that simultaneously considers point and line feature segmentation to improve closed-loop detection
Note: The entire method is based on ORB-SLAM.
C++ open source code

8. Long-term place recognition based on graph matching of landmark appearance and spatial relative position

标题：Long-term Place Recognition through Worst-case Graph Matching to Integrate Landmark Appearances and Spatial Relationships
作者：Peng Gao and Hao Zhang
来源：ICRA2020

bubble robot

Abstract description:
Location recognition is an important part of SLAM technology in many robot applications. Recently, some methods that use landmark information to solve the problem of scene changes in long-term place recognition have achieved good results. However, previous methods did not consider changes in landmarks, such as the disappearance of old landmarks and the appearance of new landmarks. Moreover, these methods usually only use the visual information or geometric information of the landmark to describe the landmark. This paper designs a graph to represent a place by encoding distance and angular spatial relationships as well as the visual appearance of landmarks. The place recognition problem can then be transformed into a worst-case graph matching problem. This algorithmic method measures the similarity of places by calculating the distance and angular spatial relationship between landmarks with minimal similarity. If, in the worst case, the visual similarity of the landmarks is low, then even if their geometric positions are familiar, they will be regarded as different places.

Main contributions:
1. Proposed a place coding method that integrates landmark visual information and relative position geometric information, improving the landmark-based place coding method;
2. Treat the location recognition problem as a worst-case graph matching problem, thus effectively solving the problem of landmark changes in long-term location recognition

9. Semi-direct monocular vision and visual-inertial SLAM and its closed-loop detection

标题：Semi-direct monocular visual and visual-inertial SLAM with loop closuredetection
作者：Shao-peng Li, Tao Zhang, Xiang Gao, Duo Wang, Yong Xian
来源：Robotics and Autonomous Systems 2018

bubble robot

Abstract description:
The author proposes a new semi-direct monocular vision simultaneous localization and mapping (SLAM) system. It can not only maintain the rapidity of the direct method, but also have the high accuracy and closed-loop detection capabilities of the feature-based method. The system extracts and matches ORB features in each key frame and tracks each non-key frame using a direct method that does not require feature extraction and matching. Keyframes are used for global or local optimization and loop closure, while non-keyframes are used for fast tracking and positioning. Therefore, the system combines the advantages of the direct method and the feature method. In addition, a monocular visual inertial SLAM system is also proposed to fuse inertial measurement data based on visual SLAM. The system is able to successfully recover scale information.
Main contributions:

1. A new semi-direct monocular visual SLAM system - SVL (semi-direct, visual and loop closure detection) is proposed;
2. At the same time, a semi-direct monocular visual inertial SLAM system is proposed, which is to fuse inertial measurement data based on the above system - SVL-VI (visual-inertial);
3. It can not only maintain the rapidity of the direct method, but also have the high precision and closed-loop detection capabilities of the feature-based method.
4. Competitive test results of a semi-direct SLAM system are given.

10. Loosely coupled semi-direct monocular SLAM

标题：Loosely-Coupled Semi-Direct Monocular SLAM
作者：Seong Hun Lee and Javier Civera
来源：International Conference on Robotics and Automation (ICRA)，2019

bubble robot

Abstract description:
This paper proposes a novel monocular simultaneous localization and mapping (SLAM) method, which combines the advantages of the direct method and the feature point method. The proposed method performs three levels of parallel optimization on direct method visual odometry and loosely coupled SLAM based on feature point method: (1) photometric bundle adjustment (BA) that combines local features and camera motion; (2) improves key frame bits Geometric BA of pose and associated map points; (3) Pose graph optimization for global map consistency when there is closed-loop detection. This is achieved by limiting the operation of feature-based marginalized keyframes to real-time using the direct method odometry module. Detailed evaluation on both datasets shows that our system outperforms state-of-the-art monocular odometry and SLAM systems in terms of overall accuracy and robustness.

Contribution:
(1) Local: For local accuracy, the direct method can be used to track the camera pose robustly, short-term, semi-dense and quickly,
(2) Global: Use the feature point-based method to improve the pose of the key frame and perform loop closure detection at the same time. A reusable global, long-term, sparse feature point map is established.

Comment: You can pay attention to how the author handles the loose coupling of the feature point method and the direct method on the front end.

Algorithm open source

11. VI-DSO based on dynamic marginalization

Title: Direct Sparse Visual-Inertial Odometry using Dynamic Marginalization
Author: Lukas von Stumberg, Vladyslav Usenko, Daniel Cremers
Source: IEEE International Conference on Robotics and Automation (ICRA), 2018, Brisbane
Bubble Robot
Abstract Description:
This paper proposes a new Visual Inertial Odometry (VIO) VI-DSO, which estimates the camera pose and recovers sparse scenes through photometric minimization and IMU measurements.
In the visual part, VI-DSO also uses beam method adjustment for optimization, but unlike the traditional system based on feature points, the object of optimization in this system is photometric error; this means, This system is not only able to track corner points, but also pixels with sufficiently large gradients.
For the IMU part, this article uses pre-integration to obtain the relative relationship between video frames and adds it as a constraint to the optimization process; in this system, the scale and gravity direction also need to be estimated The parameters are estimated and optimized together with the pose parameters and so on. At the same time, in response to the problem that IMU generally takes a period of time to obtain scale information, the VI-DSO in this article can first be initialized at any scale without waiting for the IMU to obtain sufficient observation values.
In addition, in order to improve calculation efficiency, this article performs partial marginalization on the "old" parameters, that is, the dynamic marginalization (Dynamic Marginalization) proposed in this article; this marginalization method even Ability to perform partial marginalization when scale is unknown. Finally, this article tests on EuRoC data, and the experimental results show that VI-DSO is currently the optimal algorithm.

12. Indoor navigation test of unmanned underwater vehicle based on pose graph

Bubble Robot
Abstract:
This paper presents preliminary experimental results of indoor navigation test based on pose graph, aiming to apply to unmanned Underwater Vehicles (UUV). In order to verify its effectiveness and estimate the performance of pose graph technology, we conducted indoor tests using the ground robot kobuki. In this experiment, the robot moved indoors in offices and corridors to continuously create a pose map of the robot's path. The LIDAR sensor located on the front of the robot was also used to build the map.

Comment: The author used lasers and surface codes on the robot

13. Comparison of open source VIO experiments based on state estimation in the underwater field

题:Experimental Comparison of Open Source Visual-Inertial-Based State Estimation Algorithms in the Underwater Domain
Literature: Bharat Joshi, Sharmin Rahman, Michail Kalaitzakis, Brennan Cain, James Johnson, Marios Xanthidis, Nare Karapetyan, Alan Hernandez, Alberto Quattrini Li, Nikolaos Vitzilaios, and Ioannis Rekleitis

bubble robot

Abstract description:
In this article, 10 recent VIO open source packages are compared in order to provide insights into their performance and guidelines for solving current challenges. Specifically, direct and indirect methods were chosen to fuse camera and inertial measurement unit data. Experiments are conducted by testing all packages on the underwater robotics dataset collected over the years in our laboratory. All datasets are available online.

14. DISCOMAN: Indoor scene dataset for indoor odometry, cartography and navigation

Pavel Kirsanov, Airat Gaskarov, Filipp Konokhov, Konstantin Sofiuk, Anna Vorontsova, Igor Slinko, Dmitry Zhukov , Sergey Bykov, Olga Barinova, Anton Konushin Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

bubble robot

Abstract description:
We present a novel dataset for training and benchmarking semantic SLAM methods, consisting of 200 long sequences, each containing 3000 -5000 data frames. , we sample trajectories that simulate the motion of a simple home robot and then render frames along these trajectories. Each data frame contains: a) an RGB image generated using physically based rendering, b) a simulated depth measurement, c) a simulated IMU indexed to an existing dataset, our dataset is of wider use and was developed for The first large-scale benchmark of SLAM mapping components. The dataset is divided into benchmarks ranging from classical geometry [1], [2] and recent learning [3] based SLAM algorithms, baseline mapping methods [4], semantic segmentation [5] and panoramic segmentation [6]. Test Results. The dataset and source code used to replicate our experiments will be publicly available upon release.

Comment: Can be used for slam simulation in indoor environments

15. Vision-assisted positioning of ground robots (vins on wheel)

Title: Vision-Aided Localization For Ground Robots
Author: Mingming Zhang, Yiming Chen and Mingyang Li
Source: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019
Bubble Robot

Abstract introduction:
By fusing the measurement results of the camera, IMU and wheel odometer, a novel vision-based positioning algorithm specially designed for ground robots is proposed. The first contribution of this paper is that we propose a novel algorithm that approximates the motion manifold of a ground robot through a parametric representation and performs attitude integration through IMU and wheel odometry measurements. Secondly, a complete localization algorithm is proposed by using a sliding window based estimator. The estimator is designed based on iterative optimization and can fuse measurements from multiple sensors with a proposed manifold representation. We show, based on various real-world experiments, that the proposed algorithm substantially outperforms state-of-the-art vision-based localization algorithms, especially when deployed in large-scale complex environments.

Comment: You can see how the author integrated the wheel speed meter into vio

16. UAVS automatic parking combined with UWB-Vision method in GPS-denied environment

Title: Integrated UWB-Vision Approach for Autonomous Docking of UAVs in GPS-denied Environments
Author: Thien-Minh Nguyen; Thien Hoang Nguyen; Muqing Cao; et. al a>
Source: ICRA 2019
Bubble Robot

Main contributions:

Combining UWB ranging and visual movement measurement, a relative positioning scheme based on recursive least squares optimization is proposed, and it is proved that the estimation error converges in an exponential form.
2) Compared with the previous solution of landing on a fixed platform, we have achieved collaborative docking with a mobile landing platform, which makes the docking task more flexible.
3) During the landing stage, a method combining relative position estimation and ranging was proposed to achieve a more accurate landing, and a large number of experiments verified the effectiveness of the solution up to 50 meters long.

Abstract description:
This paper proposes an effective and robust GPS-assisted visual inertial odometry (GPS-VIO) system that uses low-frequency GPS measurement data fusion Camera IMU data. In order to perform sensor fusion, spatiotemporal sensor calibration and conversion initialization between sensor reference frames are required. This paper proposes an online calibration method for the external parameters and time offset between GPS-IMU and the reference frame initialization step that is robust to GPS sensor noise. Furthermore, when estimating in the VIO reference frame, it is proved that there are four unobservable directions in the GPS-VIO system, and a state transition of the GPS reference frame is proposed to achieve full observability. We evaluate the proposed method extensively in a Monte Carlo simulation environment, where we study the robustness of the system to different levels of noise and GPS signal loss and additionally study the hyperparameters used during the initialization process. Finally, the proposed system is verified in large-scale real experiments.

17. Open source vision-inertial navigation motion capture system based on reference points

Title: An Open Source, Fiducial Based, Visual-Inertial Motion Capture System
Author: Michael Neunert, Michael Bloesch, Jonas Buchli
Source: FUSION 2016
Bubble Robot
Main contributions:
This paper proposes a monocular-inertial navigation EKF-SLAM A lightweight motion estimation system that uses artificial landmarks. This work works by tightly coupling SLAM and fiducial-based localization using corner point observations of landmarks at 6 DoF and adapting the corresponding Kalman filter. This allows for smaller map sizes and leaner estimates. Compared with existing methods that rely on 6DoF fiducials, the proposed framework processes visual measurement and inertial navigation data in a single estimator, which guarantees consistent data and avoids pre-calibration and re-calibration.

18. Improvement: Add hint binary feature descriptor for visual location re-identification

Title: Adding Cues to Binary Feature Descriptors for Visual Place Recognition
Author: Dominik Schlegel and Giorgio Grisetti
Source: The International Conference in Robotics and Automation(ICRA) 2019
Bubble Robot
Abstract description:
This paper proposes a continuous or selective prompt A method for embedding binary descriptors for visual place re-identification. This embedding method combines the original descriptor with a binary string encoding the hint and supports the use of Hamming distance as a metric. Applying descriptors augmented in this way to original applications does not require much improvement.

19. Efficient visual context classifier for robust localization (new description method of feature points)

Title: LandmarkBoost: Efficient Visual Context Classifiers for Robust Localization
Author: Marcin Dymczyk, Igor Gilitschenski, Juan Nieto, Simon Lynen, Bernhard Zeis, and Roland Siegwart< a i=2> Source: International Conference on Intelligent Robots and Systems (IROS 2018)Bubble Robot

Abstract description:
The increasing expansion of automated systems creates a need for reliable and efficient scale location retrieval algorithms. Currently used methods rely on nearest neighbor search relying on binary code to perform 2D3D matching and guarantee real-time capabilities on mobile platforms. However, these methods are inconsistent with the increase in map size, changes in viewpoint or appearance, and the presence of visual aliasing in the environment.

We propose the LandmarkBoost method to classify with traditional 2D-3D matching methods, which treats the search problem as a landmark classification task.

20. Hong Kong University of Science and Technology: Multi-sensor global pose estimation framework (vins-fusion) based on general optimization

Title: A General Optimization-based Framework for GlobalPose Estimation with Multiple Sensors
Author: Tong Qin, Shaozu Cao, Jie Pan, and Shaojie Shen
Source: arXiv 2019
Bubble Robot
Open Source Address

21. OmniSLAM: SLAM system with multiple fisheye cameras

文章：OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems
作者：Changhee Won, Hochang Seok , Zhaopeng Cui , Marc Pollefeys , and Jongwoo Lim
介绍链接

Main contributions of the article:
The main contributions are summarized as follows:

(i) A lightweight and improved depth estimation for networked fisheye cameras is proposed. The accuracy of the network, the number of parameters, and the running time have all been greatly improved compared to the previous version, making this system more practical.

(ii) By integrating the depth map into ROVO (robust visual odometry) and adding a loop closure module, a robust omnidirectional visual SLAM system is constructed. The accuracy of estimated trajectories is improved over previous versions in challenging indoor and large outdoor environments.

(iii) A complete omnidirectional positioning and dense mapping system is proposed, and extensive experiments are conducted in synthetic environments and real indoor and outdoor environments. The results show that our system is able to generate well-reconstructed 3D dense maps for various scenarios.

Brief comment: The author uses a lightweight model for depth estimation and integrates the generated depth map into ROVO.

22. Extension scheme for sustainable map preservation based on ORB-SLAM2

Article: Persistent Map Saving for Visual Localization for Autonomous Vehicles: An ORB-SLAM 2 Extension
Author: Felix Nobis∗, Odysseas Papanikolaou, Johannes Betz and Markus Lienkamp
Added map saving and a priori map positioning functions
Work introduction:
This project supports ROS, and a map needs to be provided in positioning mode. File containing objects of the ORB SLAM classes: map, MapPoint, Keyframe, and KeyframeDatabase, as well as DBoW2 BoWVector and FeatureVector for each keyframe. Therefore, we created a SaveMap method that saves this information in a binary file at the end of the SLAM process. For continuous running, the user can specify whether to load the map file through the settings file. If the user decides to use the saved map, the LoadMap method is called at system startup, which loads the saved map and KeyframeDatabase and sets the system to positioning mode. Positioning mode is done automatically and users do not need to manually change the system mode online. Both SaveMap and LoadMap methods are integrated in ORB-SLAM 2’s system class, which handles all the main functionality.

Open source address

23. ORB-SLAM2 / ORB-SLAM3 related improvement code

Introduction link
This article summarizes the currently best methods in feature point method SLAM: ORB-SLAM2 / ORB-SLAM3 related improvement code summary, including acceleration, multi-sensor fusion, Dense mapping, line features, point-line fusion, navigation, dynamic environment, multi-platform transplantation, etc. See details below.
Summary of ORB-SLAM2 related improvement codes:
The paper was published in IEEE Transactions on Robotics in 2017, the paper title is "ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras》

ORB-SLAM2 features:

The first (when released in 2017) complete open source SLAM solution supporting monocular, binocular and RGB-D cameras, with loop detection and repositioning capabilities.

It can perform real-time work on the CPU and can be used in mobile terminals such as mobile robots, mobile phones, drones, and cars.

The pinnacle of the feature point method, the positioning accuracy is extremely high, reaching centimeter level.

It can calculate the camera pose in real time and generate a sparse 3D reconstruction map of the scene.

The code is very neat, contains many practical application techniques, and is very practical.

Supports positioning-only mode, which is suitable for lightweight and long-term running when the map is known, and does not use local mapping and loop detection threads

[Super detailed Chinese annotated version]

ways to improve

ORBSLAM2_with_pointcloud_map,
Adding dense point cloud map implemented by Gao Xiang

[ORB-SLAM2_RGBD_DENSE_MAP]
(https://github.com/tiantiandabaojian/ORB-SLAM2_RGBD_DENSE_MAP), adds a dense closed-loop map based on Gao Xiang

[ORB-YGZ-SLAM]
(https://github.com/gaoxiang12/ORB-YGZ-SLAM),
using SVO The direct method is used to track instead of time-consuming feature point extraction and matching. While maintaining the same accuracy, it is 3 times faster than the original ORB-SLAM2

[YGZ-stereo-inertial SLAM]
(https://github.com/gaoxiang12/ygz-stereo-inertial),
Double The current VIO version adds LK optical flow and sliding window BA optimization

[VI-ORB] https://github.com/jingpang/LearnVIORB
VI-ORB-SLAM2 implemented by Jingpang

[Fisheye-ORB-SLAM]
(https://github.com/lsyads/fisheye-ORB-SLAM), added fisheye support

[Save and load orb-slam2 maps]
(https://github.com/AlejandroSilvestri/osmap), add the function of saving and importing maps

[ORB_SLAM2 with map load/save function]
(https://github.com/Jiankai-Sun/ORB_SLAM2_Enhanced), add save and import map function

[Viewer for maps from ORB-SLAM2 Osmap]
(https://github.com/AlejandroSilvestri/Osmap-viewer),
added map visualization

[Add line feature based ORB-SLAM2]
(https://github.com/atlas-jj/ORB_Line_SLAM), added line feature

[RGBD-SLAM with Point and Line Features, developed based on ORB_SLAM2]
(https://github.com/maxee1900/RGBD-PL-SLAM), added points Line Fusion

[Good Feature Selection for Least Squares Pose Optimization in VO/VSLAM]
(https://github.com/ivalab/gf_orb_slam2), uses a better feature Select method

ORB_SLAM2_SSD_Semantic,
Dynamic semantic SLAM target detection+VSLAM+optical flow/multi-view geometry dynamic object detection+octomap map+target database

[Tracking Enhanced ORB-SLAM2]
(https://github.com/Eralien/TE-ORB_SLAM2),
using YOLO v3 Semantic information to increase tracking performance

YOLO Dynamic ORB_SLAM, use YOLO to detect dynamic environments

Multi-platform porting code
[Windows version ORBSLAM2,Easy built by visual studio]
(https://github.com/phdsky/ORBSLAM24Windows ),
Windows platform version, can be easily compiled with visual studio

[ORB-SLAM-Android, test on Sony Xperia Z]
(https://github.com/castoryan/ORB-SLAM-Android),< a i=2> Android transplant, tested on Sony Xperia

[ORBSLAM2 on Mac OSX]
(https://github.com/meiroo/ORBSLAM2-OSX), Mac OSX version book

[ROS interface for ORBSLAM2]
(https://github.com/ethz-asl/orb_slam_2_ros), add ROS interface

ORB-SLAM3 related code

Open sourced in July 2020, the paper "ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM"

ORB-SLAM3 features

The SLAM system supports vision, vision plus inertial navigation, and hybrid maps, and can operate using pinhole or fisheye models on monocular, binocular, and RGB-D cameras.

The feature-based tightly coupled VIO system relies only on the maximum posterior estimate (including during IMU initialization). Therefore, whether it is in a large scene or a small scene, indoors or outdoors, it can run robustly and in real time, and the accuracy is improved by 2 to 5 times compared to the previous version.

The multi-map system built based on the new relocation module allows the system to run for a long time in scenes with poor characteristics.

The first system that can reuse all previous information in all algorithm stages, making it possible to use widely separated common view keyframes during BA.
https://arxiv.org/pdf/2007.11898.pdf
https://github.com/UZ-SLAMLab/ORB_SLAM3

ORB_SLAM3-RGBD-Inertial

The operating mode and ROS interface of RGBD-IMU are added, the ROS interface of monocular IMU and binocular IMU is added, the dictionary is replaced with binary format, and the loading speed is faster. The ROS interface of RGBD-IMU was rewritten based on ORB_SLAM3 to avoid queue congestion, and the parameter file of Kinect for Azure was provided.

————————————————

24. <Experimental Evaluation> Comprehensive comparison of several monocular inertial mileage calculation methods suitable for UAVs

标题：A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots
作者：Jeffrey Delmerico and Davide Scaramuzza
来源：2018 IEEE International Conference on Robotics and Automation (ICRA), May 21-25, 2018, Brisbane, Australia

bubble robot

This paper evaluates a series of open source VIO algorithms (MSCKF, OKVIS, ROVIO, VINS-Mono, SVO+MSF, SVO+GTSAM) under different hardware conditions (including the most common case of a single computing platform for drones) ). We tested the pose estimation accuracy, time spent processing each frame, and CPU and memory usage of each algorithm on the EuRoC data set, which contains various typical 6-degree-of-freedom flight trajectories of UAVs. We present data for all comparisons, which can provide a baseline for comparison for other related studies.
The video of the article has been published on YouTube

25. <Review>A review of visual inertial odometry based on filtering and optimization

标题：A review of visual inertial odometry from filtering and optimisation perspectives
作者：Jianjun Gui∗, Dongbing Gu, Sen Wang and Huosheng Hu
来源：Advanced Robotics, 2015

Bubble Bot DescriptionAdd link description

26. Indoor slam data set: Lifelong SLAM data set and challenge

Paper addressAdd link description
Dataset addressAdd link description
Competition and data set introductionAdd link description
2019 competition status introduction

27. <Various Datasets>The most powerful SLAM Datasets collection

Bubble RobotAdd link description
Dataset introduction linkAdd link description

28. <Calibration> Automatically calibrate the external parameters between the camera and the three-dimensional lidar using the correspondence between straight lines and planes

标题：Automatic Extrinsic Calibration of a Camera and a 3D LiDAR using Line and Plane Correspondences
作者：Lipu Zhou, Zimo Li, and Michael Kaess
来源：2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Bubble RobotAdd link description
Abstract:
In this article the camera is processed using a checkerboard calibration plate External parameter calibration with three-dimensional lidar. Unlike the previous requirement of at least three checkerboards to calibrate board positions, our algorithm reduces the minimum required checkerboard positions to 1 by combining three-dimensional line and plane correspondences.

29. Intermittent GPS-Assisted VIO: Online Initialization and Calibration

Title: Intermittent GPS-aided VIO: Online Initialization and Calibration
Author: Woosik Lee, Kevin Eckenhoff, Patrick Geneva, and Guoquan Huang
Source ：2020 IEEE International Conference on Robotics and Automation (ICRA)
Bubble Robot DescriptionAdd link description

Abstract description:
This paper proposes an effective and robust GPS-assisted visual inertial odometry (GPS-VIO) system that uses low-frequency GPS measurement data fusion Camera IMU data. In order to perform sensor fusion, spatiotemporal sensor calibration and conversion initialization between sensor reference frames are required. This paper proposes an online calibration method for the external parameters and time offset between GPS-IMU and the reference frame initialization step that is robust to GPS sensor noise. Furthermore, when estimating in the VIO reference frame, it is proved that there are four unobservable directions in the GPS-VIO system, and a state transition of the GPS reference frame is proposed to achieve full observability. We evaluate the proposed method extensively in a Monte Carlo simulation environment, where we study the robustness of the system to different levels of noise and GPS signal loss and additionally study the hyperparameters used during the initialization process. Finally, the proposed system is verified in large-scale real experiments.

30. Multi-sensor fusion slam data set for indoor environment

http://www.rawseeds.org/rs/datasets/view//6

31. SlamBench2: A multi-objective head-to-head benchmark for visual SLAM

Abstract:
SLAM is becoming a key component of robotics and augmented reality (AR) systems. Although a large number of SLAM algorithms have been proposed, little effort has been spent on unifying the interfaces of these algorithms or making an overall comparison of their capabilities. This is a problem because different SLAM applications can have different functional and non-functional requirements. For example, mobile phone-based AR applications have tight energy budgets, while drone navigation systems often require high accuracy. SlamBench2 is a benchmark framework for evaluating existing and future SLAM systems, both open and closed source, against an extensible list of datasets while using a comparable and clearly specified list of performance metrics. Various existing SLAM algorithms and datasets are supported, such as ElasticFusion, InfiniTAM, ORB-SLAM2, OKVIS, and the integration of new algorithms and datasets is directly and explicitly specified by the framework. SlamBench2 is a publicly available software framework that represents the starting point for quantitative, comparable and verifiable experimental studies to study trade-offs across SLAM systems.

32. Hong Kong University of Science and Technology - Binocular vision depth estimation and 3D map construction work

For how to judge that vins-mon fails to run and the trajectory has drifted, you can refer to the work of the Hong Kong University of Science and Technology:
VINS-Fusion with Cerebro

Binocular large viewing angle fisheye vins-fusion
Open source project
VINS-Fisheye development
In the article It introduces the part about binocular depth calculation and the development process, which is of great reference significance, and this part is also separated in the code

Binocular depth calculation based on deep learning:
HKUST-Aerial-Robotics——MVDepthNet
Usage example, Gao Fei of Hong Kong University of Science and Technology uses vins -mono and MVDepthNet for fine reconstruction of three-dimensional environmental obstacle maps
Open source address

More detailed content consolidation resources, HKUST Teaching Repetition Plan (Drone Competition)

Very powerful probabilistic map generation for dronesAdd link description
This project contains powerful and computationally efficient tools for fast quadcopter flight A collection of algorithms: