OpenCV: 10 Feature Detection

Basic concepts of feature detection

Feature detection is a concept in computer vision and image processing . It refers to the use of computers to extract image information and determine whether each image point belongs to an image feature. The result of feature detection is to divide the points on the image into different subsets , which often belong to isolated points, continuous curves or continuous areas.

Feature detection includes edge detection, corner detection, region detection, and ridge detection

Feature detection application scenarios:

• Image search, such as image search
• jigsaw puzzle
• image stitching
• …

Take the jigsaw puzzle as an example to illustrate the application process of feature detection

• look for features
• The feature is unique (if it exists everywhere, then we can't know where to start)
• Traits are traceable
• characteristics are comparable

Therefore:
A, B (blue box) are not features --> not unique
C, D (black box) are not specific --> not unique
E, F (red box) are features --> unique, traceable, comparable

we discover:

• The flat part (A, B) is difficult to find its position in the original image
• The edge part (C, D) is easier to find than the flat part, but it cannot be uniquely determined
• Corner points (E, F) can find their position in the original image at once --> corner points are not edges! but the corner

Image features are regions with meaningful values ​​that are unique and easy to identify

Harris corner detection

The most important thing in the image features is the corners, which ones are the corners?

• The pixel corresponding to the maximum value of the grayscale gradient (different colors nearby! For example, the corner of a tall building is the color of the wall and the color of the sky)
• Intersection of two lines (two edges are two lines)
• The extreme point of the gray value in the image (the first derivative is the largest and the second derivative is 0)

---

The three pictures correspond to three situations:
Figure 1 moves around in a flat position, and the gray value changes little; Figure 2 moves up and down, the gray value changes little, and moves left and right, and the gray value changes drastically; Figure 3, no matter which direction it moves, the gray value changes drastically

---

principle

Image I(x,y), self-similarity when (x,y)translated at points:(△x,△y)

自相似性：移动后与原来点相似度有多少，其实就是差异；具体是上一个位置框的像素值 - 下一个框的像素值

Adding the square is because we only change the size of the variable , and don't care about the gray value variable or darkening

Our frame is equivalent to a convolution kernel, and the convolution kernel has a weight corresponding to each pixel of the position, so w(u,v)it is the weight of the pixel frame (Gaussian weighting: the closer to the center position, the greater the weighting coefficient)

We need to use Taylor's formula to expand and make a first-order approximation I(u + △x,v + △y)of the image image I(x,y)after (x,y)translation : where is the partial derivative of the image(△x,△y)

Ix、IyI(x,y)

---

Supplement Taylor's formula:


Generally, we can use the second order

---

Approximately available:

The essence of the quadratic function is an elliptic function, and the equation is:

that is λ, the larger the value, the greater the change in the gray value of the image (these λ1、λ2are respectively 矩阵M: λ1 = 1 / A²、λ2 = 1 / B²)

It's impossible for us to compare manually again λ1、λ2, that's too stupid

opencvThe API is provided, which is also the return value of our call : Harris角点检测
corner response:

among them detM = λ1 × λ2, traceM = λ1 + λ2,α = 0.04 ~ 0.06（自己选择）

corner response result: return value

• 0 < R < 1, is the plane part: for example λ1 = 0.01 ；λ2 = 0.02（λ1、λ2都很小）, there is
  

• R < 0, is the edge part: for example λ1 = 100 ； λ2 = 0.01（λ1、λ2中一个远大于另一个）, there are
  

• R >> 0, is the corner part: for example λ1 = 100 ； λ2 = 101（λ1、λ2都很大且数值相当）, there is
  

---

application

The detection window moves in the image, and the above picture corresponds to three situations

• Moving in a flat area : No matter which direction you move, the measurement system does not change much
• Moving in the edge area : When moving in the vertical edge direction (going in or out), the measurement system changes drastically
• Moving in the corner area : No matter which direction you move, the measurement system changes drastically

Key API: cv2.cornerHarris(src, blockSize, ksize, k[, dst[, borderType]])
Among them:

• src: Image to operate on (must be a grayscale image)
• blockSize: The size of the detection window (there is no requirement that it must be an odd number, we are not doing convolution operations), the smaller the value set, the better the corner points can be detected, if the value is too large, a large block will be scanned, which is not easy to detect
• ksize: Sobel算子size (the partial derivative is used in the principle, Sobel算子which is used to calculate the partial derivative )
• k: Equivalent to in the corner response formula α, the value is0.04 ~ 0.06（默认值为0.04 ）

import cv2
import numpy as np

img = cv2.imread('./chess.png')
# img = cv2.imread('./hand.png')


# 变成灰度图片
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

##----------------------------------------------------------
# 角点检测 ——> 返回角点响应R，每一个像素都可以计算出一个角点响应
R = cv2.cornerHarris(gray,blockSize = 2,ksize = 3,k = 0.04)

# 显示角点
# 设定阈值来判断角点 怎么设置？
# 返回值R中有一个最大的R值R.max()，即最大的角点响应，但我们不可能只用最大的角点，因为我们还有很多个角点，所以我们找一个关系来设置阈值
# R > ( 0.01 * R.max()) # 用ndarray和单个数字相比，返回值为bool类型：该点的角点响应大于最大角点响应的0.01倍，我们认为它就是角点 

# 我们把它作为一个条件，直接对图片进行筛选
img[R > ( 0.01 * R.max() )] = [0,0,255] # 颜色改成红色

##----------------------------------------------------------

cv2.imshow('img',img)

cv2.waitKey(0)
cv2.destroyAllWindows()

The corner response of the picture

is compared with the corner response threshold, if it is greater than it is True, if it is less than it is False:R > ( 0.01 * R.max() )

The image will be directly screened R > ( 0.01 * R.max() )as a condition img[R > ( 0.01 * R.max() )], and the pixels that meet the corner point requirements will be screened out. There are:

assign these points to red(0,0,255)

result: