[VSLAM] HF-Net framework learning (1) - configure the environment and run the Demo program

HF-NetIt is a convolutional neural network that can extract image descriptors ( global_descriptors) and feature points in the image ( keypoints) and their descriptors ( local_descriptors). The former is used for image retrieval, and the latter SuperGlue/NNcan be used for camera pose calculations with other feature matching algorithms. Therefore HF-Net, the application scenario is SLAMmap positioning and pose recovery.

HF-NetPaper address

HF-Netcode address

Here are the installation steps:

(1) Download source code

git clone https://github.com/ethz-asl/hfnet.git

(2) Environment initialization

method one:

The author wrote an initialization script

make install

Method Two:

But I don’t plan to use this script here, for fear of Ubuntumessing up the environment, I’d better follow the script content step by step.

#1.安装jupyter notebook
conda activate base
pip install jupyter
#2.创建一个虚拟环境用于HF-Net，虽然作者指定tf版本为1.12，但测试1.14也行，这里就安装1.14版本
conda create -n tf114 python=3.7
#3.把tf114虚拟环境添加到jupyter notebook中
conda activate tf114
conda install ipykernel
pip install --upgrade jupyter_client
python3 -m ipykernel install --user --name tf114  --display-name "tf114"
#4.开始配置tf114虚拟环境
conda activate tf114
pip install tensorflow_gpu==1.14.0
pip install keras==2.2.5
pip install protobuf==3.20.0
pip install pandas==1.0.0
pip install sklearn
pip install matplotlib==3.0.0
pip install numpy==1.19
pip install opencv-python==4.2.0.32
pip install scipy
pip install tqdm
pip install pyyaml
pip install flake8
pip install matplotlib
pip install protobuf
pip install sklearn
pip install pillow
pip install deepdish
#5.创建项目路径文件settings.py
cd hfnet
sh setup/setup.sh
# 运行后会让你在终端中输入两个路径：DATA_PATH和EXPER_PATH，前者是存放训练图像和预训练模型权重，后者存放 hfnet的训练输出，路径要为绝对路径，不要带~，例如我的是：
# DATA_PATH：/home/xxx/Project/python/tensorflow/hfnet/data/input
# EXPER_PATH：/home/xxx/Project/python/tensorflow/hfnet/data/output
# 路径会定义在./hfnet/settings.py文件中，后续也可以自己修改两个路径

(3) Download the trained model weights

download link

After downloading, unzip and saved_modelscopy the folder to EXPER_PATHthe path

(4) Run `Demo`the program

The author provides a Demoprogram demo.ipynbthat can quickly feel the matching results of the feature points extracted by HF-Net

cd hfnet
# 打开jupyter notebook
conda activate base
jupyter notebook

jupyterAfter running, open demo.ipynbthe file and modify the running environment to your own configured virtual environment.tf114

Then run it step by step, which step to load the model. If it is the following picture, the loading is successful:

Finally, let’s take a look at the matching effect. It can be successfully matched even when the lighting changes drastically.