《视觉SLAM十四讲 第二版》笔记及课后习题（第十一讲）

读书笔记：回环检测

回环检测与词袋

回环检测的意义:

• VO和后端都存在误差
• SLAM的建图与定位是耦合的——误差将会累计

Loop Closing步骤:

• 检测到回环的发生
• 计算回环修选帧与当前帧的运动
• 验证回环是否成立
• 闭环
  

Word的形成:

• 聚类
• 经典的k-means聚类 N个特征点->k个类
• 随机选取k 个中心点： $c_1,···,c_k;$
• 对每一个样本，计算与每个中心点之间的距离，取最小的作为它的归类；
• 重新计算每个类的中心点。
• 如果每个中心点都变化很小，则算法收敛，退出；否则返回1。
  

SLAM中的回环检测:

• 使用相对的评分而非绝对的评分
• 思路：相邻关键帧是相似的，而回环的相似度应该约等于相邻关键帧

检测成功以后：

• 根据PnP等算法计算运动关系
• 根据重投影关系验证回环是否成立
• 利用全局BA或Pose Graph进行优化

与机器学习的关系:
从前边的论述中可以看出，回环检测与机器学习有着千丝万缕的关联。回环检测本身非常像是一个分类问题。与传统模式识别的区别在于，回环中的类别数量很大，而每类的样本很少——极端情况下，当机器人发生运动后，图像发生变化，就产生了新的类别，我们甚至可以把类别当成连续变量而非离散变量；而回环检测，相当于两个图像落入同一类，则是很少出现的。从另一个角度，回环检测也相当于对“图像间相似性”概念的一个学习。既然人类能够掌握图像是否相似的判断，让机器学习到这样的概念也是非常有可能的。

从词袋模型来说，它本身是一个非监督的机器学习过程——构建词典相当于对特征描述子进行聚类，而树只是对所聚的类的一个快速查找的数据结构而已。既然是聚类，结合机器学习里的知识，我们至少可以问：

1. 是否能对机器学习的图像特征进行聚类，而不是SURF、ORB 这样的人工设计特征进行聚类？
2. 是否有更好的方式进行聚类，而不是用树结构加上K-means 这些较朴素的方式？

结合目前机器学习的发展，二进制描述子的学习和无监督的聚类，都是很有望在深度学习框架中得以解决的问题。我们也陆续看到利用机器学习进行回环检测的工作。尽管目前词袋方法仍是主流，但我个人是相信未来深度学习方法很有希望打败这些人工设计特征的，“传统”的机器学习方法。毕竟词袋方法在物体识别问题上已经明显不如神经网络了，而回环检测又是非常相似的一个问题。

实践部分

本章需要用到BoW库，这里选用的是DBoW3，因为它对OpenCV的兼容性较好 。由于它也是个cmake的工程，所以安装方法如之前所述，按照cmake的方法安装即可。

feature_training：

编译出现错误：

====================[ Build | feature_training | Debug ]========================
/opt/clion-2019.2.5/bin/cmake/linux/bin/cmake --build /home/wh/shenlan/slambook2/ch11/cmake-build-debug --target feature_training -- -j 6
make[3]: *** No rule to make target '/usr/local/lib/libDBoW3.a', needed by 'feature_training'。 停止。
CMakeFiles/Makefile2:106: recipe for target 'CMakeFiles/feature_training.dir/all' failed
make[2]: *** [CMakeFiles/feature_training.dir/all] Error 2
CMakeFiles/Makefile2:113: recipe for target 'CMakeFiles/feature_training.dir/rule' failed
make[1]: *** [CMakeFiles/feature_training.dir/rule] Error 2
Makefile:131: recipe for target 'feature_training' failed
make: *** [feature_training] Error 2

看错误是缺少libDBoW3.a静态库，重新安装了一下，发现果然没有，看了对应的文件夹，也是果然没有。安装的文件如下：

Install the project...
-- Install configuration: "Release"
-- Installing: /usr/local/lib/cmake/FindDBoW3.cmake
-- Installing: /usr/local/lib/cmake/DBoW3/DBoW3Config.cmake
-- Installing: /usr/local/lib/libDBoW3.so.0.0.1
-- Up-to-date: /usr/local/lib/libDBoW3.so.0.0
-- Up-to-date: /usr/local/lib/libDBoW3.so
-- Set runtime path of "/usr/local/lib/libDBoW3.so.0.0.1" to ""
-- Up-to-date: /usr/local/include/DBoW3/DescManip.h
-- Up-to-date: /usr/local/include/DBoW3/timers.h
-- Up-to-date: /usr/local/include/DBoW3/DBoW3.h
-- Up-to-date: /usr/local/include/DBoW3/exports.h
-- Up-to-date: /usr/local/include/DBoW3/QueryResults.h
-- Up-to-date: /usr/local/include/DBoW3/ScoringObject.h
-- Up-to-date: /usr/local/include/DBoW3/Database.h
-- Up-to-date: /usr/local/include/DBoW3/Vocabulary.h
-- Up-to-date: /usr/local/include/DBoW3/quicklz.h
-- Up-to-date: /usr/local/include/DBoW3/FeatureVector.h
-- Up-to-date: /usr/local/include/DBoW3/BowVector.h
-- Installing: /usr/local/bin/demo_general
-- Set runtime path of "/usr/local/bin/demo_general" to ""
-- Installing: /usr/local/bin/create_voc_step0
-- Set runtime path of "/usr/local/bin/create_voc_step0" to ""
-- Installing: /usr/local/bin/create_voc_step1
-- Set runtime path of "/usr/local/bin/create_voc_step1" to ""

可以看到，的确是没有.a的静态库。而CmakeList.txt里有：

set( DBoW3_LIBS "/usr/local/lib/libDBoW3.a" )

这不报错才奇怪（逃

于是我怀疑是安装的库有问题，所以又去github重新下载安装(话说一到晚上，github下载就慢的使人抓狂，到了第二天白天又瞬间满速了，这是为了让我们不要晚上工作吗2333)。
安装之后发现，还是那些东西：

Install the project...
-- Install configuration: "Release"
-- Installing: /usr/local/lib/cmake/FindDBoW3.cmake
-- Installing: /usr/local/lib/cmake/DBoW3/DBoW3Config.cmake
-- Installing: /usr/local/lib/libDBoW3.so.0.0.1
-- Up-to-date: /usr/local/lib/libDBoW3.so.0.0
-- Up-to-date: /usr/local/lib/libDBoW3.so
-- Set runtime path of "/usr/local/lib/libDBoW3.so.0.0.1" to ""
-- Installing: /usr/local/include/DBoW3/DescManip.h
-- Installing: /usr/local/include/DBoW3/timers.h
-- Installing: /usr/local/include/DBoW3/DBoW3.h
-- Installing: /usr/local/include/DBoW3/exports.h
-- Installing: /usr/local/include/DBoW3/QueryResults.h
-- Installing: /usr/local/include/DBoW3/ScoringObject.h
-- Installing: /usr/local/include/DBoW3/Database.h
-- Installing: /usr/local/include/DBoW3/Vocabulary.h
-- Installing: /usr/local/include/DBoW3/quicklz.h
-- Installing: /usr/local/include/DBoW3/FeatureVector.h
-- Installing: /usr/local/include/DBoW3/BowVector.h
-- Installing: /usr/local/bin/demo_general
-- Set runtime path of "/usr/local/bin/demo_general" to ""
-- Installing: /usr/local/bin/create_voc_step0
-- Set runtime path of "/usr/local/bin/create_voc_step0" to ""
-- Installing: /usr/local/bin/create_voc_step1
-- Set runtime path of "/usr/local/bin/create_voc_step1" to "

也果然编译错误。虽说是路径报错，但是路径链接的应该没问题啊？！
于是我强行改成了动态链接库.so，这下编译完成，但是调试又出了问题：

error while loading shared libraries: libopencv_core3.so.3.3: cannot open shared object file: No such file or directory

最后，我又卸载了3rdparty的版本，去安装了slambok第一版的DBoW3的库，然后，成功了！果然是版本问题，可以看见，这个版本的是有静态库的：

Install the project...
-- Install configuration: "Release"
-- Installing: /usr/local/lib/cmake/FindDBoW3.cmake
-- Installing: /usr/local/lib/libDBoW3.a
-- Installing: /usr/local/include/DBoW3/DescManip.h
-- Installing: /usr/local/include/DBoW3/DBoW3.h
-- Installing: /usr/local/include/DBoW3/exports.h
-- Installing: /usr/local/include/DBoW3/QueryResults.h
-- Installing: /usr/local/include/DBoW3/ScoringObject.h
-- Installing: /usr/local/include/DBoW3/Database.h
-- Installing: /usr/local/include/DBoW3/Vocabulary.h
-- Installing: /usr/local/include/DBoW3/FeatureVector.h
-- Installing: /usr/local/include/DBoW3/BowVector.h
-- Installing: /usr/local/bin/demo_general
-- Set runtime path of "/usr/local/bin/demo_general" to ""
-- Installing: /usr/local/bin/create_voc_step0
-- Set runtime path of "/usr/local/bin/create_voc_step0" to ""
-- Installing: /usr/local/bin/create_voc_step1
-- Set runtime path of "/usr/local/bin/create_voc_step1" to ""

最终，编译执行成功，结果如下：

/home/wh/shenlan/slambook2/ch11/cmake-build-debug/feature_training /home/wh/shenlan/slambook2/ch11/data
reading images... 
detecting ORB features ... 
creating vocabulary ... 
vocabulary info: Vocabulary: k = 10, L = 5, Weighting = tf-idf, Scoring = L1-norm, Number of words = 4995
done

Process finished with exit code 0

loop_closure ：

配置好文件环境：

运行结果如下：

/home/wh/shenlan/slambook2/ch11/cmake-build-debug/loop_closure /home/wh/shenlan/slambook2/ch11/vocabulary.yml.gz /home/wh/shenlan/slambook2/ch11/data
reading database
reading images... 
detecting ORB features ... 
comparing images with images 
image 0 vs image 0 : 1
image 0 vs image 1 : 0.0273297
image 0 vs image 2 : 0.0302259
image 0 vs image 3 : 0.0189142
image 0 vs image 4 : 0.0268474
image 0 vs image 5 : 0.0209544
image 0 vs image 6 : 0.0258485
image 0 vs image 7 : 0.024856
image 0 vs image 8 : 0.0364348
image 0 vs image 9 : 0.0526796

image 1 vs image 1 : 1
image 1 vs image 2 : 0.0445484
image 1 vs image 3 : 0.0397254
image 1 vs image 4 : 0.0322338
image 1 vs image 5 : 0.0270872
image 1 vs image 6 : 0.0240194
image 1 vs image 7 : 0.0304614
image 1 vs image 8 : 0.0275719
image 1 vs image 9 : 0.0308597

image 2 vs image 2 : 1
image 2 vs image 3 : 0.0342784
image 2 vs image 4 : 0.0285314
image 2 vs image 5 : 0.024943
image 2 vs image 6 : 0.0421186
image 2 vs image 7 : 0.0365022
image 2 vs image 8 : 0.0352966
image 2 vs image 9 : 0.0391034

image 3 vs image 3 : 1
image 3 vs image 4 : 0.0258569
image 3 vs image 5 : 0.0308412
image 3 vs image 6 : 0.0310181
image 3 vs image 7 : 0.0412206
image 3 vs image 8 : 0.0258239
image 3 vs image 9 : 0.0430648

image 4 vs image 4 : 1
image 4 vs image 5 : 0.0584299
image 4 vs image 6 : 0.0162427
image 4 vs image 7 : 0.0317469
image 4 vs image 8 : 0.0290156
image 4 vs image 9 : 0.023683

image 5 vs image 5 : 1
image 5 vs image 6 : 0.0252686
image 5 vs image 7 : 0.0425041
image 5 vs image 8 : 0.0265156
image 5 vs image 9 : 0.0342134

image 6 vs image 6 : 1
image 6 vs image 7 : 0.0377101
image 6 vs image 8 : 0.0395201
image 6 vs image 9 : 0.0272016

image 7 vs image 7 : 1
image 7 vs image 8 : 0.0426041
image 7 vs image 9 : 0.0358428

image 8 vs image 8 : 1
image 8 vs image 9 : 0.031152

image 9 vs image 9 : 1

comparing images with database 
database info: Database: Entries = 10, Using direct index = no. Vocabulary: k = 10, L = 5, Weighting = tf-idf, Scoring = L1-norm, Number of words = 4995
searching for image 0 returns 4 results:
<EntryId: 0, Score: 1>
<EntryId: 9, Score: 0.0526796>
<EntryId: 8, Score: 0.0364348>
<EntryId: 2, Score: 0.0302259>

searching for image 1 returns 4 results:
<EntryId: 1, Score: 1>
<EntryId: 2, Score: 0.0445484>
<EntryId: 3, Score: 0.0397254>
<EntryId: 4, Score: 0.0322338>

searching for image 2 returns 4 results:
<EntryId: 2, Score: 1>
<EntryId: 1, Score: 0.0445484>
<EntryId: 6, Score: 0.0421186>
<EntryId: 9, Score: 0.0391034>

searching for image 3 returns 4 results:
<EntryId: 3, Score: 1>
<EntryId: 9, Score: 0.0430648>
<EntryId: 7, Score: 0.0412206>
<EntryId: 1, Score: 0.0397254>

searching for image 4 returns 4 results:
<EntryId: 4, Score: 1>
<EntryId: 5, Score: 0.0584299>
<EntryId: 1, Score: 0.0322338>
<EntryId: 7, Score: 0.0317469>

searching for image 5 returns 4 results:
<EntryId: 5, Score: 1>
<EntryId: 4, Score: 0.0584299>
<EntryId: 7, Score: 0.0425041>
<EntryId: 9, Score: 0.0342134>

searching for image 6 returns 4 results:
<EntryId: 6, Score: 1>
<EntryId: 2, Score: 0.0421186>
<EntryId: 8, Score: 0.0395201>
<EntryId: 7, Score: 0.0377101>

searching for image 7 returns 4 results:
<EntryId: 7, Score: 1>
<EntryId: 8, Score: 0.0426041>
<EntryId: 5, Score: 0.0425041>
<EntryId: 3, Score: 0.0412206>

searching for image 8 returns 4 results:
<EntryId: 8, Score: 1>
<EntryId: 7, Score: 0.0426041>
<EntryId: 6, Score: 0.0395201>
<EntryId: 0, Score: 0.0364348>

searching for image 9 returns 4 results:
<EntryId: 9, Score: 1>
<EntryId: 0, Score: 0.0526796>
<EntryId: 3, Score: 0.0430648>
<EntryId: 2, Score: 0.0391034>

done.

Process finished with exit code 0

课后习题

1. 请书写计算PR 曲线的小程序。用MATLAB 或Python 可能更加简便一些，因为它们擅长作图。

2. 验证回环检测算法，需要有人工标记回环的数据集，例如[94]。然而人工标记回环是很不方便的，我们会考虑根据标准轨迹计算回环。即，如果轨迹中有两个帧的位姿非常相近，就认为它们是回环。请你根据TUM 数据集给出的标准轨迹，计算出一个数据集中的回环。这些回环的图像真的相似吗？

3. 学习DBoW3 或DBoW2 库，自己寻找几张图片，看能否从中正确检测出回环。

4. 调研相似性评分的常用度量方式，哪些比较常用？

参考这篇博客

5. Chow-Liu 树是什么原理？它是如何被用于构建字典和回环检测的？

参考知乎的回答

6. 阅读[106]，除了词袋模型，还有哪些用于回环检测的方法？

相关方法