[300 OpenCV routines] 245. BRISK operator for feature detection

"300 OpenCV routines of youcans - general catalog"

[300 OpenCV routines of youcans] 245. BRISK operator for feature detection

6.8.1 Introduction to Algorithms

The scale-invariant binary feature description BRISK (Binary Robust Invariant Scalable Kepoints) is an improved BRIEF algorithm and a binary feature descriptor. It has high computational efficiency, rotational invariance, scale invariance, and robustness to noise.

1. Establish a scale space

To achieve scale invariance, the BRISK algorithm obtains the scale of each keypoint by constructing an image pyramid. The image pyramid of the BRISK algorithm consists of n-level octave and n-level intra-octave. The upper-level image of the octave is obtained by down-sampling the lower-level image, and the inner octave is located between adjacent octaves.

img	c0	d0	c1	d1	c2	d2	c3	d3
high	h	2/3*h	1/2*h	1/3*h	1/4*h	1/6*h	1/8*h	1/12*h
width	in	2/3 * in	1/2 * in	1/3 * in	1/4 * in	1/6 * in	1/8 * in	1/12 * in

2. Key point detection

Similar to the SIFT algorithm, keypoint detection includes three steps of scale-space corner detection, non-maximum suppression and sub-pixel interpolation.

(1) Use the FAST9-16 and FAST5-8 detectors of the AGAST method for corner detection. Use FAST5-8 to detect the corners of the original image img as the d(-1) layer; use FAST9-16 to detect the corners of the images c0~c3 and d0~d3. A total of 9 corner point detection images are obtained.

(2) Non-maximum suppression is performed in the 3*3*3 neighborhood to locate local feature points in spatial location and scale space. The pixel point is compared with the 26 points in the neighborhood (8 adjacent points of the same scale and 9*2 points corresponding to the upper and lower adjacent scales), and if it is the maximum/minimum value, it is reserved as a key point.

(3) Use Sub-pixel interpolation for precise positioning. The extreme points obtained by non-maximum suppression are discrete in scale space and pixel space, and the accuracy of the obtained extreme point position and scale size is low.

First, perform two-dimensional quadratic function interpolation in the pixel space according to the FAST score value to obtain the precise coordinate position of the key point; then perform one-dimensional interpolation on the scale direction to obtain the precise scale of the key point.

3. Sampling mode and main direction

The key to the BRISK descriptor is the sampling pattern of the keypoint neighborhood. As shown in the figure, N points are sampled in the S*S area centered on the feature point (key point) to form $N\ast (N-1)/2$ pairs of sample points$(p_i,p_j)$ .

In order to avoid image aliasing, the variance of sampling point P is $Gaussian\; filtering\; of\; \sigma$ , variance$\sigma$ is proportional to the distance r from the sampling point to the center.

Then calculate the local gradient value:
$$g(p_i,p_j)=(p_j-p_i)\ast \frac{I(p_j,p_j)-I(p_i,p_i}{||p_j-p_i|{|}^2}$$
Among them, $g(p_i,p_j)$ represents the local gradient value,$I(p_j,p_j)$ represents the pixel value,$p$ is the key point position,$\sigma$ is the keypoint scale.

Considering all sample point pairs, the short distance threshold $d_{max}=9.75\ast t$ and long distance threshold$d_{min}=13.67\ast t$ (t is the scale k of the feature points), a subset S of short-distance point pairs and a subset L of long-distance point pairs are obtained.

The principal directions of keypoints can be calculated from the local gradient values:
$$g=\left(\genfrac{}{}{0ex}{}{g_x}{g_{}}\right)=\frac{}{L}\ast \underset{(pi,pj)\in L}{}g(p_i,p_j)$$
Rotate the sampling area around the feature point to the main direction, the rotation angle is$a=arctan2(g_{},g_x)$ , thus achieving rotational invariance.

4. Binary feature descriptor

The BRISK descriptor is a binary feature, and each bit of the descriptor corresponds to the comparison result of short-distance point pairs:
$$b=\{\begin{array}{ll}1& ,I(p_j^{},p_j)>I(p_i^{},p_i)\\ & ,else\end{array}$$
Taking N as 60, there are 1770 point pairs in total. Consider the 512 short-distance point-to-point pairs, which constitute a 512-bit binary code, that is, 64 bytes.

The BRISK algorithm directly generates the feature description method of binary strings, which speeds up the establishment of feature descriptors, and greatly reduces the memory footprint of feature descriptors and the time for feature matching.

4. Feature matching

Feature point matching using Hamming distance.

Speed ​​comparison in image registration test: SIFT<SURF<BRISK<FREAK<ORB. The BRISK algorithm performs best for image registration with larger blurriness.

6.8.2 BRISK class in OpenCV

OpenCV provides a wealth of feature detection algorithms, and inherits the cv::Feature2D class, using a unified definition and encapsulation.

OpenCV provides the cv::BRISK class to implement the BRISK method, inherits the cv::Feature2D class, and creates it through the create static method.

The constructor of the BriefDescriptorExtractor class has three forms, namely:

BRISK::BRISK()
static Ptr< BRISK > create (int thresh=30, int octaves=3, float patternScale=1.0f)
static Ptr< BRISK > create (const std::vector< float > &radiusList, const std::vector< int > &numberList, float dMax=5.85f, float dMin=8.2f, const std::vector< int > &indexChange=std::vector< int >()))
static Ptr< BRISK > create (int thresh, int octaves, const std::vector< float > &radiusList, const std::vector< int > &numberList, float dMax=5.85f, float dMin=8.2f, const std::vector< int > &indexChange=std::vector< int >())

In the Python language, OpenCV provides the interface function cv.BRISK.create() to instantiate the BRISK class.

cv.BRISK.create([, thresh=30, octaves=3, patternScale=1.0f]) → retval
cv.BRISK.create(radiusList, numberList[, dMax=5.85f, dMin=8.2f, indexChange=std::vector<int>()]) → retval

cv.BRISK.create(thresh, octaves, radiusList, numberList[, dMax=5.85f, dMin=8.2f, indexChange=std::vector<int>()]) → retval
cv.BRISK_create([, thresh=30, octaves=3, patternScale=1.0f]	) → retval

cv.BRISK_create(radiusList, numberList[, dMax=5.85f, dMin=8.2f, indexChange=std::vector<int>()]) → retval
cv.BRISK_create(thresh, octaves, radiusList, numberList[, dMax=5.85f, dMin=8.2f, indexChange=std::vector<int>()]) → retval

brisk.compute(image, keypoints[, descriptors=None]) → keypoints, descriptors

Parameter Description:

• image: input image, single channel

• thresh: AGGST detection threshold

• octaves: octaves, 0 means a single scale

• patternScale: pattern scale, used to sample the neighborhood of keypoints

• radiusList: The sampling radius around the keypoint (units are pixels, scale is 1)

• numberList: defines the number of sampling points on the circumference, the length is the same as radiusList

• dMax: short pairing threshold (in pixels, scale 1)

• dMin: long pairing threshold (unit is pixel, scale is 1)

• indexChange: The index of the remapping of the bits

• keypoints : The detected keypoints, a special data structure

• descriptors: descriptors of key points, Numpy array, shape (n, bytes)

Routine 14.27: BRISK operator for feature detection

    # 14.27 特征检测之 BRISK 算子
    # 读取基准图像
    imgRef = cv.imread("../images/Circuit04.png", flags=1)  # (480, 600, 3)
    refer = cv.cvtColor(imgRef, cv.COLOR_BGR2GRAY)  # 基准图像
    height, width = imgRef.shape[:2]  # 图片的高度和宽度
    print("shape of image: ", height, width)
    # 读取或构造目标图像
    top, left = int(0.1*height), int(0.1*width)
    border = cv.copyMakeBorder(imgRef, top, top, left, top, borderType=cv.BORDER_CONSTANT, value=(32,32,32))
    zoom = cv.resize(border, (width, height), interpolation=cv.INTER_AREA)
    theta= 15  # 顺时针旋转角度，单位为角度
    x0, y0 = width//2, height//2  # 以图像中心作为旋转中心
    MAR = cv.getRotationMatrix2D((x0,y0), theta, 1.0)
    imgObj = cv.warpAffine(zoom, MAR, (width, height))  # 旋转变换，默认为黑色填充
    # imgObj = cv.imread("../images/Circuit04B.png", flags=1)  # (480, 600, 3)
    object = cv.cvtColor(imgObj, cv.COLOR_BGR2GRAY)  # 目标图像
    print("shape of image: ", imgObj.shape)

    # 构造 BRISK 对象，检测关键点，计算特征描述向量
    brisk = cv.BRISK_create()  # 创建 BRISK 检测器
    kpRef, desRef = brisk.detectAndCompute(refer, None)  # 基准图像关键点检测
    kpObj, desObj = brisk.detectAndCompute(object, None)  # 目标图像关键点检测
    imgRefBrisk = cv.drawKeypoints(imgRef, kpRef, None, flags=cv.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)  # 绘制关键点大小和方向
    imgObjBrisk = cv.drawKeypoints(imgObj, kpObj, None, flags=cv.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)  # 绘制关键点大小和方向
    print(desRef.shape, desObj.shape)

    # 特征点匹配，Brute-force matcher
    matcher = cv.BFMatcher()  # 构造 BFmatcher 对象
    matches = matcher.match(desRef, desObj)  # 对描述子 des1, des2 进行匹配
    matches = sorted(matches, key=lambda x: x.distance)
    matches1 = cv.drawMatches(imgRef, kpRef, imgObj, kpObj, matches[:100], None, flags=2)
    print('queryIdx=%d' % matches[0].queryIdx)
    print('trainIdx=%d' % matches[0].trainIdx)
    print('distance=%d' % matches[0].distance)
    print("bf.match:{}".format(len(matches)))

    plt.figure(figsize=(9, 6))
    ax1 = plt.subplot(212)
    ax1.set_title("BRISK detector & BFMatcher")
    plt.imshow(cv.cvtColor(matches1, cv.COLOR_BGR2RGB)), plt.axis('off')
    ax2 = plt.subplot(221)
    plt.axis('off'), plt.imshow(cv.cvtColor(imgRefBrisk, cv.COLOR_BGR2RGB))
    ax2.set_title("BRISK keypoints (Ref)")
    ax3 = plt.subplot(222)
    plt.axis('off'), plt.imshow(cv.cvtColor(imgObjBrisk, cv.COLOR_BGR2RGB))
    ax3.set_title("BRISK keypoints (Obj)")
    plt.tight_layout()
    plt.show()

---

参考文献：Stefan Leutenegger, Margarita Chli, and Roland Yves Siegwart. Brisk: Binary robust invariant scalable keypoints. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 2548–2555. IEEE, 2011.

【End of this section】

Copyright statement:
Youcans@xupt original works, reprints must be marked with the original link: (https://blog.csdn.net/youcans/article/details/127415464)
Copyright 2022 youcans, XUPT Crated
: 2022-10-20

241. Scale-Invariant Feature Transform (SIFT)
242. Accelerated Robust Feature Detection Algorithm (SURF)
243. FAST Algorithm for Feature Detection