"python+opencv practice" four, image feature extraction and description - 30Harris corner detection

Goals
• Understand the concept of Harris Corner Detection

• Learning functions: cv2.cornerHarris(), cv2.cornerSubPix()

principle

In the previous section we already knew a property of corners: moving in any direction changes a lot. Chris_Harris and Mike_Stephens have proposed a method for corner detection as early as 1988 in the article "A Combined Corner and Edge Detector", which is called Harris corner detection. He turned this simple idea into mathematical form. Move the window in all directions (u, v) and calculate the sum of all differences. The expression is as follows:

The window function can be a normal rectangular window or a Gaussian window with different weights for each pixel.

The value of E (u; v) should be maximized in corner detection. This means that the value of the second term on the right side of the equation must be maximized. Expansion of the above equation by Taylor series and then through a few mathematical conversions (you can refer to other standard textbooks), we get the following equation:

in:

Here Ix and Iy are the derivatives of the image in the x and y directions. (It can be calculated using the function cv2.Sobel()).
Then there is the main part. They scored based on an equation used to determine whether a window contained a corner.

in:

So according to these features, we can judge whether an area is a corner, a boundary or a plane.

We can use the following figure to express our conclusion:

So the result of Harris corner detection is a grayscale image composed of corner scores. By binarizing the resulting image with an appropriate threshold, we have detected the corners in the image. We will demonstrate this with a simple picture.

30.1 Harris Corner Detection in OpenCV

Functions in Opencv2.cornerHarris(src, blockSize, ksize, k[, dst[, borderType]]) → dst can be used for corner detection. The parameters are as follows:

Parameters:

• src – Input single-channel 8-bit or floating-point image.
• dst – Image to store the Harris detector responses. It has the typeCV_32FC1 and the same size assrc .
• blockSize – Neighborhood size (see the details oncornerEigenValsAndVecs() ).
• ksize – Aperture parameter for theSobel() operator.
• k – Harris detector free parameter. See the formula below.
• borderType – Pixel extrapolation method. SeeborderInterpolate() .

      • img - an input image of data type float32.
      • blockSize - the size of the area to be considered in corner detection.
      • ksize - window size used in Sobel's derivation
      • k - free parameter in Harris corner detection equation, the value parameter is [0, 04, 0.06].
Examples are as follows:

[python] view plain copy

1. <span style="font-size:12px;"># -*- coding: utf-8 -*-  
2. """ 
3. Created on Mon Jan 20 18:53:24 2014 
4. @author: duan 
5. """  
6. import  cv2  
7. import numpy as np  
8. filename = 'chessboard.jpg'  
9. img = cv2.imread(filename)  
10. gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)  
11. gray = np.float32(gray)  
12. # The input image must be float32, the last parameter is between 0.04 and 0.05  
13. dst = cv2.cornerHarris(gray,2,3,0.04)  
14. #result is dilated for marking the corners, not important  
15. dst = cv2.dilate(dst,None)  
16. # Threshold for an optimal value, it may vary depending on the image.  
17. img[dst>0.01*dst.max()]=[0,0,255]  
18. cv2.imshow('dst',img)  
19. if cv2.waitKey(0) & 0xff == 27:  
20.     cv2.destroyAllWindows()</span>  

                                   

30.2 Corners with sub-pixel accuracy
Sometimes we need corner detection with maximum accuracy. OpenCV provides us with the function cv2.cornerSubPix(), which can provide sub-pixel level corner detection. Below is an example. First we need to find the Harris corner, and then pass the center of gravity of the corner to this function for correction. Harris corners are marked with red pixels and green pixels are corrected pixels. When using this function we want to define an iteration stop condition. The iteration stops when the number of iterations is reached or the accuracy condition is met. We also need to define the size of the neighborhood
for

[python] view plain copy

1. <span style="font-size:12px;"># -*- coding: utf-8 -*-  
2. """ 
3. Created on Mon Jan 20 18:55:47 2014 
4. @author: duan 
5. """  
6. import cv2  
7. import numpy as np  
8. filename = 'chessboard2.jpg'  
9. img = cv2.imread(filename)  
10. gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)  
11. # find Harris corners  
12. gray = np.float32(gray)  
13. dst = cv2.cornerHarris(gray,2,3,0.04)  
14. dst = cv2.dilate(dst,None)  
15. ret, dst = cv2.threshold(dst,0.01*dst.max(),255,0)  
16. dst = np.uint8(dst)  
17. # find centroids  
18. #connectedComponentsWithStats(InputArray image, OutputArray labels, OutputArray stats,  
19. #OutputArray centroids, int connectivity=8, int ltype=CV_32S)  
20. ret, labels, stats, centroids = cv2.connectedComponentsWithStats(dst)  
21. # define the criteria to stop and refine the corners  
22. criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 0.001)  
23. #Python: cv2.cornerSubPix(image, corners, winSize, zeroZone, criteria)  
24. #zeroZone – Half of the size of the dead region in the middle of the search zone  
25. #over which the summation in the formula below is not done. It is used sometimes  
26. # to avoid possible singularities of the autocorrelation matrix. The value of (-1,-1)  
27. # indicates that there is no such a size.  
28. # 返回值由角点坐标组成的一个数组（而非图像）  
29. corners = cv2.cornerSubPix(gray,np.float32(centroids),(5,5),(-1,-1),criteria)  
30. # Now draw them  
31. res = np.hstack((centroids,corners))  
32. #np.int0 可以用来省略小数点后面的数字（非四五入）。  
33. res = np.int0(res)  
34. img[res[:,1],res[:,0]]=[0,0,255]  
35. img[res[:,3],res[:,2]] = [0,255,0]  
36. cv2.imwrite('subpixel5.png',img)</span>  

The results are as follows, for the convenience of viewing we zoomed in on the corners: