Feature point detection: The feature points of an image can be simply understood as the more significant points in the image, such as contour points, bright spots in darker areas, dark spots in brighter areas, etc. ORB uses the FAST (features from accelerated segment test) algorithm to detect feature points . This definition is based on the gray value of the image around the feature point, and detects the pixel value of a circle around the candidate feature point. If there are enough pixels in the area around the candidate point and the gray value of the candidate point is sufficiently different, it is considered that the The candidate point is a feature point.
Among them, I(x) is the grayscale of any point on the circumference, I(p) is the grayscale of the center of the circle, and Ed is the threshold value of the grayscale value difference. If N is greater than the given threshold, it is generally three-quarters of the surrounding circle points. , then p is considered to be a feature point. Additional speedups are used for faster results. If 4 points every 90 degrees around the candidate point are tested, there should be at least 3 points whose gray value difference is large enough with the candidate point. The selection radius of the circle around the candidate point is a very important parameter. Here, for simplicity and efficiency, the radius is 3, and a total of 16 surrounding pixels need to be compared. In order to improve the efficiency of comparison, only N surrounding pixels are usually used for comparison, which is often referred to as FAST-N.
Calculation of feature descriptors: ORB uses the Brief algorithm to calculate the descriptor of a feature point. The core idea of the BRIEF algorithm is to select N point pairs in a certain pattern around the key point P, and combine the comparison results of these N point pairs as a descriptor.
(1) Make a circle O with the key point P as the center and d as the radius.
(2) Select N point pairs in a certain pattern in circle O. For the convenience of explanation here, N=4, and N can be 512 in practical applications. Assume that the currently selected 4 point pairs are marked as shown in the figure above:
(3) Define the operation T: 
(4) Perform T operations on the selected point pairs respectively, and combine the obtained results
, the final descriptor is: 1011
The coordinate system established by ORB when calculating the Brief descriptor takes the key point as the center of the circle, and establishes a 2-dimensional coordinate system with the line connecting the key point and the centroid of the point-taking area as the X-axis. P is the key point. The inside of the circle is the point-taking area, and each small grid represents a pixel. Now we regard this central area as a board, and the quality of each point on the board is equal to its corresponding pixel value. According to the knowledge of integral calculus, we can find the mass center Q of this uneven density plank. Calculated as follows. where R is the radius of the circle.
We know that the center of the circle is fixed and rotates as the object rotates. When we use PQ as the coordinate axis, under different rotation angles, the points we take out in the same point taking mode are consistent. This solves the problem of rotational consistency.
In BRIEF, a 9x9 Gaussian operator is used for filtering, which can solve the noise-sensitive problem to a certain extent, but one filtering is obviously not enough. It is proposed in ORB to use the integral image to solve: in a 31x31 window, after generating a pair of random points, take the random point as the center, take a 5x5 sub-window, compare the size of the sum of the pixels in the two sub-windows for binary encoding, and The binary encoding is not determined by just two random points. (This step can be done with integral image)
Matching of feature points :
The biggest feature of the ORB algorithm is the fast calculation speed. This is first thanks to the use of FAST to detect feature points. FAST's detection speed is as famous as its name.
quick. Again, use the BRIEF algorithm to calculate the descriptor. The unique binary string representation of the descriptor not only saves storage space, but also greatly shortens the matching time.
between. For example, the descriptors of feature points A and B are as follows. A: 10101011B: 10101010 We set a threshold, such as 80%. When the similarity between the descriptors of A and B is 90%, we judge that A and B are the same feature points, that is, the two points are successfully matched. In this example, A and B are only different in the last digit, and the similarity is 87.5%, which is greater than 80%. Then A and B are matched.
We can easily calculate the similarity between A and B by XORing A and B. The XOR operation can be completed by group hardware, which has high efficiency and speeds up the matching speed.
img=cv2.imread('images/ktxleft2.jpg')
orb=cv2.ORB_create()
kp,des=orb.detectAndCompute(img,None)
img3=cv2.drawKeypoints(img,kp,None,color=(0,255,0),flags=0)