DXSLAM es un sistema SLAM visual basado en una extracción profunda de características de CNN. La dirección en papel y los resultados de la medición real no son buenos. TUM perdió la pista al cambiar el conjunto de datos.
(1) Instalación de biblioteca dependiente de C ++
C++11 or C++0x Compiler
Pangolin
OpenCV
Eigen3
Dbow、Fbow and g2o (Included in Thirdparty folder)
tensorflow(1.12)
(2) Descargar el código fuente
git clone https://github.com/raulmur/DXSLAM.git DXSLAM
(3) compilar código
cd dxslam
chmod +x build.sh
./build.sh
(4) Descargue el conjunto de datos de prueba, el autor utiliza el conjunto de datos TUM
Dirección de descarga del conjunto de datos TUM , donde el prefijo fr1 es el conjunto de datos TUM1, el prefijo fr2 es el conjunto de datos TUM2 y el prefijo fr3 es el conjunto de datos TUM3. Seleccione descargar aquí (muchos fr1/xyzconjuntos de datos se pierden durante la ejecución, pero esto el conjunto de datos se puede ejecutar correctamente)
Después de descargar el conjunto de datos, descomprímalo en DXSLAMel proyecto y deberá sincronizar la marca de tiempo:
Cree un script en la carpeta del conjunto de datos descomprimidos associate.pypara sincronizar marcas de tiempo, escrito en base a Python2:
#!/usr/bin/python
# Software License Agreement (BSD License)
#
# Copyright (c) 2013, Juergen Sturm, TUM
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
# * Neither the name of TUM nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
# Requirements:
# sudo apt-get install python-argparse
"""
The Kinect provides the color and depth images in an un-synchronized way. This means that the set of time stamps from the color images do not intersect with those of the depth images. Therefore, we need some way of associating color images to depth images.
For this purpose, you can use the ''associate.py'' script. It reads the time stamps from the rgb.txt file and the depth.txt file, and joins them by finding the best matches.
"""
import argparse
import sys
import os
import numpy
def read_file_list(filename):
"""
Reads a trajectory from a text file.
File format:
The file format is "stamp d1 d2 d3 ...", where stamp denotes the time stamp (to be matched)
and "d1 d2 d3.." is arbitary data (e.g., a 3D position and 3D orientation) associated to this timestamp.
Input:
filename -- File name
Output:
dict -- dictionary of (stamp,data) tuples
"""
file = open(filename)
data = file.read()
lines = data.replace(","," ").replace("\t"," ").split("\n")
list = [[v.strip() for v in line.split(" ") if v.strip()!=""] for line in lines if len(line)>0 and line[0]!="#"]
list = [(float(l[0]),l[1:]) for l in list if len(l)>1]
return dict(list)
def associate(first_list, second_list,offset,max_difference):
"""
Associate two dictionaries of (stamp,data). As the time stamps never match exactly, we aim
to find the closest match for every input tuple.
Input:
first_list -- first dictionary of (stamp,data) tuples
second_list -- second dictionary of (stamp,data) tuples
offset -- time offset between both dictionaries (e.g., to model the delay between the sensors)
max_difference -- search radius for candidate generation
Output:
matches -- list of matched tuples ((stamp1,data1),(stamp2,data2))
"""
first_keys = first_list.keys()
second_keys = second_list.keys()
potential_matches = [(abs(a - (b + offset)), a, b)
for a in first_keys
for b in second_keys
if abs(a - (b + offset)) < max_difference]
potential_matches.sort()
matches = []
for diff, a, b in potential_matches:
if a in first_keys and b in second_keys:
first_keys.remove(a)
second_keys.remove(b)
matches.append((a, b))
matches.sort()
return matches
if __name__ == '__main__':
# parse command line
parser = argparse.ArgumentParser(description='''
This script takes two data files with timestamps and associates them
''')
parser.add_argument('first_file', help='first text file (format: timestamp data)')
parser.add_argument('second_file', help='second text file (format: timestamp data)')
parser.add_argument('--first_only', help='only output associated lines from first file', action='store_true')
parser.add_argument('--offset', help='time offset added to the timestamps of the second file (default: 0.0)',default=0.0)
parser.add_argument('--max_difference', help='maximally allowed time difference for matching entries (default: 0.02)',default=0.02)
args = parser.parse_args()
first_list = read_file_list(args.first_file)
second_list = read_file_list(args.second_file)
matches = associate(first_list, second_list,float(args.offset),float(args.max_difference))
if args.first_only:
for a,b in matches:
print("%f %s"%(a," ".join(first_list[a])))
else:
for a,b in matches:
print("%f %s %f %s"%(a," ".join(first_list[a]),b-float(args.offset)," ".join(second_list[b])))
Ejecute el script:
python associate.py PATH_TO_SEQUENCE/rgb.txt PATH_TO_SEQUENCE/depth.txt > associations.txt
# 例如
cd rgbd_dataset_freiburg1_xyz
python associate.py rgb.txt depth.txt > associations.txt # 需要Python2，python3会报错
(5) TUMArchivos de configuración preparados
Debido a DXSLAMque se basa en ORB2la modificación, puede copiar directamente el archivo de configuración ORB2dentro TUM1( TUM1a qué fr1conjuntos de datos con prefijo corresponde el archivo de configuración; si descarga otras series de conjuntos de datos, copie el archivo de configuración correspondiente)
Aquí está TUM1el contenido del archivo de configuración:
%YAML:1.0
#--------------------------------------------------------------------------------------------
# Camera Parameters. Adjust them!
#--------------------------------------------------------------------------------------------
File.version: "1.0"
Camera.type: "PinHole"
# Camera calibration and distortion parameters (OpenCV)
Camera.fx: 517.306408
Camera.fy: 516.469215
Camera.cx: 318.643040
Camera.cy: 255.313989
Camera.k1: 0.262383
Camera.k2: -0.953104
Camera.p1: -0.005358
Camera.p2: 0.002628
Camera.k3: 1.163314
Camera.width: 640
Camera.height: 480
# Camera frames per second
Camera.fps: 30
# Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)
Camera.RGB: 1
# Close/Far threshold. Baseline times.
Stereo.ThDepth: 40.0
Stereo.b: 0.07732
# Depth map values factor
RGBD.DepthMapFactor: 5000.0 # 1.0 for ROS_bag
#--------------------------------------------------------------------------------------------
# ORB Parameters
#--------------------------------------------------------------------------------------------
# ORB Extractor: Number of features per image
ORBextractor.nFeatures: 1000
# ORB Extractor: Scale factor between levels in the scale pyramid
ORBextractor.scaleFactor: 1.2
# ORB Extractor: Number of levels in the scale pyramid
ORBextractor.nLevels: 8
# ORB Extractor: Fast threshold
# Image is divided in a grid. At each cell FAST are extracted imposing a minimum response.
# Firstly we impose iniThFAST. If no corners are detected we impose a lower value minThFAST
# You can lower these values if your images have low contrast
ORBextractor.iniThFAST: 20
ORBextractor.minThFAST: 7
#--------------------------------------------------------------------------------------------
# Viewer Parameters
#--------------------------------------------------------------------------------------------
Viewer.KeyFrameSize: 0.05
Viewer.KeyFrameLineWidth: 1.0
Viewer.GraphLineWidth: 0.9
Viewer.PointSize: 2.0
Viewer.CameraSize: 0.08
Viewer.CameraLineWidth: 3.0
Viewer.ViewpointX: 0.0
Viewer.ViewpointY: -0.7
Viewer.ViewpointZ: -1.8
Viewer.ViewpointF: 500.0
(6) El modelo de aprendizaje profundo extrae puntos característicos
Este proyecto se ejecuta sin conexión y necesita extraer puntos característicos del conjunto de datos con anticipación. El entorno de aprendizaje profundo requiere TensorFlow 1.12. Aquí puedo instalar 1.14 y aprobar:
# 创建一个conda虚拟环境，安装TensorFlow1.14
conda create -n tensorflow114 python=3.7
pip install tensorflow_gpu==1.14.0
pip install keras==2.2.5
pip install protobuf==3.20.0
pip install numpy==1.16.5
pip install pandas==1.0.0
pip install sklearn
pip install matplotlib==3.0.0
pip install python-opencv
Después de configurar el entorno, puede ejecutar el modelo de aprendizaje profundo y extraer puntos característicos:
cd hf-net
python3 getFeature.py image/path/to/rgb output/feature/path
# 例如
python getFeature.py ../rgbd_dataset_freiburg1_xyz/rgb ../feature
(7) Todo está listo, ejecuta el programa principal de SLAM:
Primero echemos un vistazo a la composición del archivo del proyecto en este momento: (solo se muestran los archivos principales)
├── Ejemplos
│ └── RGB-D
│ ├── rgbd_tum (programa principal)
│ ├── rgbd_tum.cc
│ ├── TUM1.yaml (archivo de configuración)
├── característica (puntos de característica extraídos por el modelo de aprendizaje profundo)
├── rgbd_dataset_freiburg1_xyz (conjunto de datos TUM)
└── Vocabulario
├── DXSLAM.fbow (modelo de bolsa de palabras)
└── DXSLAM.tar.xz
./Examples/RGB-D/rgbd_tum Vocabulary/DXSLAM.fbow Examples/RGB-D/TUMX.yaml PATH_TO_SEQUENCE_FOLDER ASSOCIATIONS_FILE OUTPUT/FEATURE/PATH
# 例如
./Examples/RGB-D/rgbd_tum Vocabulary/DXSLAM.fbow ./Examples/RGB-D/TUM1.yaml ./rgbd_dataset_freiburg1_xyz ./rgbd_dataset_freiburg1_xyz/associations.txt ./feature
