K-Nearest Neighbors (KNN for short) and ratio test (Ratio Test) are common techniques for feature matching in computer vision. They are usually used together with feature descriptors (such as SIFT, SURF, ORB, etc.) to find similar feature points in images.
Here are the general steps for feature matching using K-Nearest Neighbors and Ratio Test:
-
Extract feature descriptors:
Use appropriate feature extraction algorithms (such as SIFT, SURF, ORB, etc.) to extract feature points from the image, and calculate the descriptor of each feature point. -
Feature point matching:
For two images, calculate the descriptor of each feature point separately, and use the KNN algorithm to find the K feature points closest to each feature point in the second image. -
Ratio test:
For each feature point, calculate the distance ratio of the two closest feature points (often called a ratio test). A match is reliable if the nearest neighbor distance is farther from the next nearest neighbor distance. You can use a threshold to filter matches. -
Filter matches:
Apply a ratio test to filter out those matching points whose distance ratio is below a threshold. This can help weed out less reliable matches. -
Draw matching results:
Draw the remaining matching points on the two images for observation and analysis.
Imagine a large group of well-known philosophers inviting you to judge their
debate on a question important to life, the universe, and everything. You listen carefully as each philosopher takes his turn to speak. Finally, when all the philosophers have published all their arguments, you scan the notes and notice two things:
- Every philosopher disagrees with the views of the other philosophers.
- No philosopher is more persuasive than other philosophers.
From your initial observations, you infer that at most one philosopher is right,
but in fact, it is also possible that all philosophers are wrong. Then, on a second observation, even if one of the philosophers is right, you start to worry about possibly picking a philosopher who is wrong. No matter how you look at it, these people are going to make you late for dinner. You called it a tie and said that the most important issues in the debate remained unresolved. We can compare the hypothetical problem of judging philosophers' debates with the real problem of filtering out poor keypoint matches.
First, it is assumed that each keypoint in the query image has at most one correct match in the scene . That is, if the query image is a NASA logo, then another image, (scene) is assumed to contain at most one NASA logo. Assuming that there is at most one correct or good match for a query keypoint, we mainly observe bad matches when considering all possible matches.
Thus, a brute force matcher that computes a distance score for each possible match can provide a large number of observations about the distance scores of bad matches. I would expect a good match to have a significantly better (lower) distance score than an infinite number of bad matches, so the bad match score can help us choose a threshold for good matches. Such a threshold may not generalize well to different query key points or different scenarios, but at least it will help in specific cases.
Now, let's consider the implementation of a modified brute force matching algorithm that
adaptively chooses a distance threshold in the manner described above. In the sample code in the previous section, we used
the match method of the cv2.BFMatcher class to obtain a list containing the single best match (minimum distance) for each query keypoint. Such an implementation discards information about the distance scores of all possible bad matches, which is needed by adaptive methods. Fortunately, cv2.BFMatcher还提供了knnMatch方法, the method accepts a parameter k that specifies the maximum number of best (shortest distance) matches that we want to keep for each query keypoint. (In some cases, the number of matches obtained may be less than the specified maximum number.) KNN stands for K-Nearest Neighbor.
We will use the knnMatch method to request a list of the two best matches for each query keypoint
. Based on the assumption that there is at most one correct match for each query keypoint, we are convinced that suboptimal matches are false. The threshold is obtained by multiplying the distance score of the suboptimal match by a value less than 1.
The best match is then considered a good match only if the distance score is less than a threshold
.
This method is called the ratio test (ratio test), first proposed by David Lowe (the author of the SIFT algorithm). He describes ratio tests in the paper "Distinctive Image
Features from Scale-Invariant Keypoints" (available at https://www.cs.ubc.cs/~lowe/papers/ijcv04.pdf) . Specifically, in the "Application to object recognition" section, he states the following:
The probability of a correct match can be determined from the ratio of the distance from the nearest neighbor to the 2nd nearest neighbor
.
We can load the image, detect keypoints,
and compute the ORB descriptor in the same way as the previous code example. Then, perform brute force KNN matching with the following two lines of code:
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck= False)
knn_matches = bf.knnMatch(descriptors1, descriptors2, k=2)
knnMatch returns a list of lists, each inner list contains at least one match,
and no more than k matches, sorted from best (shortest distance) to worst.
The following line of code sorts the outer list according to the distance score of the best match:
# 根据匹配的质量，对匹配进行排序，显示前n个排序
knn_matches = sorted(knn_matches, key=lambda x: x[0].distance)
The overall code is as follows:
import cv2
def orb_test():
# 加载图片 灰色
img1 = cv2.imread('images\\quexiao\\2-1.png')
gray1 = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)
img2 = cv2.imread('images\\quexiao\\2.png')
gray2 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)
image1 = gray1.copy()
image2 = gray2.copy()
'''
1.使用ORB算法检测特征点、描述符
'''
orb = cv2.ORB_create(128)
keypoints1, descriptors1 = orb.detectAndCompute(image1, None)
keypoints2, descriptors2 = orb.detectAndCompute(image2, None)
# BFMatcher解决匹配
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck= False)
knn_matches = bf.knnMatch(descriptors1, descriptors2, k=2)
# matches = bf.match(descriptors1, descriptors2)
# 根据匹配的质量，对匹配进行排序，显示前n个排序
knn_matches = sorted(knn_matches, key=lambda x: x[0].distance)
# 绘制前10个最佳匹配
img_matches = cv2.drawMatchesKnn(img1, keypoints1, img2, keypoints2, knn_matches[:10], img2,flags=cv2.DRAW_MATCHES_FLAGS_NOT_DRAW_SINGLE_POINTS)
cv2.imshow('matche', img_matches)
cv2.waitKey(0)
cv2.destroyAllWindows()
if __name__ == '__main__':
orb_test()
running result:
Comparing this output image with the output image from the previous section, we can see that using KNN
and the ratio test can filter out a lot of bad matches. The remaining matches are not perfect, but almost all of them point to the correct area.
The original image of the experiment is in the previous section, and the previous section opencv advanced 16-ORB (image matching) based on FAST features and BRIEF descriptors