Lesson3-5: OpenCV image processing---template matching and Hough transform

learning target

• Master the principles of template matching and be able to complete the application of template matching
• Understand the principle of Hough line transformation and Hough circle detection
• Know how to detect lines and circles using OpenCV

---

1 template matching

1.1 Principle

The so-called template matching is to find the area most similar to the template in a given picture. The input of this algorithm includes templates and pictures. The idea of ​​the whole task is to continuously move the template picture according to the sliding window idea, and calculate its relationship with the image. According to the matching degree of the corresponding area, the area with the highest matching degree is finally selected as the final result.

Implementation process:

• Prepare two images:

  1. Original image (I): In this image, find the area that matches the template

  2. Template (T): image block compared with the original image

• Compare the sliding template image with the original image:
  
  move the template block one pixel at a time (from left to right, top to bottom), and at each position, calculate the similarity with the template image.

• For each position, the calculated similarity results are stored in the result matrix ( $R$ ) in. If the size of the input image is ($WxH$ ) and the size of the template image ($wxh$ ), then the output matrix$The\; size\; of\; R$ is ($W-w+1,H-h+1$ ) Will$R$ is displayed as an image as shown below:
  

• After obtaining the above image, find the location of the maximum value, and then the area corresponding to that location is considered the best match. The corresponding area is a matrix with this point as the vertex and the length and width of the template image.

1.2 Implementation

We implement template matching using methods in OpenCV.
API:

res = cv.matchTemplate(img,template,method)

parameter:

• img: Image to be template matched
• Template:template
• method: Algorithm to implement template matching, mainly including:
  1. Squared difference matching ( CV_TM_SQDIFF): Use the squared difference between the template and the image to match. The best match is 0. The worse the match, the greater the matching value.
  2. Correlation matching ( CV_TM_CCORR): Matching is performed using the multiplication between the template and the image. The larger the value, the higher the degree of matching, and the smaller the value, the lower the matching effect.
  3. Use correlation coefficient matching ( CV_TM_CCOEFF): Use the correlation coefficient matching between the template and the image, $1$ means a perfect match,$-1$ represents the worst match.

After completing the matching, cv.minMaxLoc()just use the method to find the location of the maximum value. If squared difference is used as the comparison method, the minimum value position is the best matching position.

Example:

In this case, load the image and template to be searched. The image is as follows: The

template is as follows:

By matchTemplateimplementing template matching, use minMaxLoclocate the most matching area and mark the most matching area with a rectangle.

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
# 1 图像和模板读取
img = cv.imread('./image/wulin2.jpeg')
template = cv.imread('./image/wulin.jpeg')
h,w,l = template.shape
# 2 模板匹配
# 2.1 模板匹配
res = cv.matchTemplate(img, template, cv.TM_CCORR)
# 2.2 返回图像中最匹配的位置，确定左上角的坐标，并将匹配位置绘制在图像上
min_val, max_val, min_loc, max_loc = cv.minMaxLoc(res)
# 使用平方差时最小值为最佳匹配位置
# top_left = min_loc
top_left = max_loc
bottom_right = (top_left[0] + w, top_left[1] + h)
cv.rectangle(img, top_left, bottom_right, (0,255,0), 2)
# 3 图像显示
plt.imshow(img[:,:,::-1])
plt.title('匹配结果'), plt.xticks([]), plt.yticks([])
plt.show()

Expansion: Template matching is not suitable for images after scale transformation and perspective transformation. In this case, we must use key point matching algorithms. The more classic key point detection algorithms include SIFT and SURF. The main idea is to first pass the key point detection algorithm Obtain the key points in the template and test pictures; then use the key point matching algorithm to process them. These key points can handle scale changes, perspective changes, rotation changes, lighting changes, etc. very well, and have good invariance.

2 Hough transform

Hough transform is often used to extract geometric shapes such as straight lines and circles in images, as shown in the following figure:

2.1 Principle

1. Principle
   In the Cartesian coordinate system, a straight line consists of two points $A=(x_1,y_1)$ sum$B=(x_2,y_2)$ , as shown in the figure below:
   
   put the straight line$y=kx+q$ can be written as about$(k,Function\; expression\; of\; q)$ :

$$\{\begin{array}{l}q=-k{x}_1+y_1\\ q=-k{x}_2+y_2\end{array}$$
The corresponding transformation is intuitively represented through graphics:

The transformed space is called Hough space. That is: a straight line in the Cartesian coordinate system corresponds to a point in the Hough space . In turn, the same holds true, a line in the Hough space corresponds to a point in the Cartesian coordinate system, as follows:

We Let’s look at the two points A and B, corresponding to the Hough space:

Let’s look at the situation where the three points are collinear:

It can be seen that if the points in the Cartesian coordinate system are collinear, then these points are in the Hough space. The corresponding straight lines intersect at a point .

If there is more than one straight line, it is as follows:

We select the points where as many straight lines converge as possible. The two points A and B formed by the three straight lines in the above figure correspond to the straight lines in the Cartesian coordinate system:

to Here we seem to have completed the solution of the Hough transform. But if it is like the picture below:

the straight line in the picture above is $x=2,\; that(k,q)$ How to determine?

To solve this problem, we consider converting the Cartesian coordinate system to polar coordinates.

It is the same in polar coordinates. The points in polar coordinates correspond to the lines of the Hough space. At this time, the Hough space is no longer the parameter $(k,q)$ space, but$(p,\theta )$ space,$\rho$ is the vertical distance from the origin to the straight line,$\theta$ represents the angle between the vertical line of the straight line and the clockwise direction of the horizontal axis. The angle of the vertical line is$0$ degrees, the angle of the horizontal line is$180$ degrees.

We only need to find the position of the intersection point in Hough space to get the straight line in the original coordinate system.

Implementation process

Suppose there is a size of $100\ast 100$ pictures, use Hough transform to detect straight lines in the picture, the steps are as follows:

• Straight lines can be used $(p,\theta )$ means, first create a$2D$ array, we call it accumulator, initialize all values ​​to$0$ , the row represents$\rho$ , the columns represent$\theta$ .
  
  The size of the array determines the accuracy of the result. If you want the accuracy of the angle to be$1$ degree, then$180$ columns. For$\rho$ , the maximum value is the distance between the diagonal lines of the image. If you want the accuracy to reach the pixel level, the number of rows should be equal to the distance between the diagonal lines of the image.

• Take the first point (x, y) on the line (x,y)$(x,y)$ , bring that into the formula for the straight line in polar coordinates, and then traverseθ θValues ​​of$\theta$$0,1,2,...,180$ , find the corresponding ρ value respectively. If this value has a corresponding position in the above accumulator, add$1$.

• Take the second point on the line and repeat the above steps to update the value in the accumulator. For each point on the line in the image, take steps above the line, updating the value in the accumulator each time.

• Search the maximum value in the accumulator and find its corresponding $(p,\theta )$ , the straight line in the image can be expressed.
  

2.2 Hough line detection

The API used for Hough line detection in OpenCV is:

cv.HoughLines(img, rho, theta, threshold)

parameter:

• img: The detected image is required to be a binary image, so before calling the Hough transform, it must first be binarized, or $Cannyedge$ detection

• rho、theta: $r$和th iAccuracy of$\theta$

• threshold: Threshold above which only values ​​in the accumulator are considered a straight line.
  The entire process of Hough line detection is shown in the figure below. This is an explanation of Hough line transformation on stackflow:
  
  Example:

Detect straight lines in the following image:

import numpy as np
import random
import cv2 as cv
import matplotlib.pyplot as plt
# 1.加载图片，转为二值图
img = cv.imread('./image/rili.jpg')

gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
edges = cv.Canny(gray, 50, 150)

# 2.霍夫直线变换
lines = cv.HoughLines(edges, 0.8, np.pi / 180, 150)
# 3.将检测的线绘制在图像上（注意是极坐标噢）
for line in lines:
    rho, theta = line[0]
    a = np.cos(theta)
    b = np.sin(theta)
    x0 = a * rho
    y0 = b * rho
    x1 = int(x0 + 1000 * (-b))
    y1 = int(y0 + 1000 * (a))
    x2 = int(x0 - 1000 * (-b))
    y2 = int(y0 - 1000 * (a))
    cv.line(img, (x1, y1), (x2, y2), (0, 255, 0))
# 4. 图像显示
plt.figure(figsize=(10,8),dpi=100)
plt.imshow(img[:,:,::-1]),plt.title('霍夫变换线检测')
plt.xticks([]), plt.yticks([])
plt.show()

2.3 Hough Circle Detection [Understanding]

1. Principle
   The expression of a circle is:
   $(x-a{)}^2+(y-b{)}^2=r$

Among them, a and b represent the coordinates of the center of the circle, and r represents the radius of the circle. Therefore, the standard Hough circle detection is to perform circle detection on the three-dimensional space accumulator composed of these three parameters. At this time, the efficiency will be very low, so OpenCV uses The Hough gradient method is used to detect circles.

The Hough gradient method divides the Hough circle detection range into two stages. The first stage detects the circle center, and the second stage uses the circle center to derive the circle radius.

• The principle of circle center detection: the center of a circle is the intersection of the circumferential normals. Set a threshold. If the number of intersecting straight lines at a certain point is greater than this threshold, the intersection is considered to be the center of the circle.
• Principle of determining circle radius: The distance (radius) from the center of the circle to the circumference is the same. Determine a threshold. As long as the number of the same distance is greater than the threshold, the distance is considered to be the radius of the center of the circle.

In principle, the Hough transform can detect any shape, but complex shapes require more parameters and the Hough space has more dimensions. Therefore, the memory space required for program implementation and operating efficiency are not conducive to converting the standard Hough transform into Transformation is applied in the detection of actual complex graphics. The Hough gradient method is an improvement of the Hough transform. Its purpose is to reduce the dimension of the Hough space and improve efficiency.

2. API

The API used to detect rings in images in OpenCV is:

circles = cv.HoughCircles(image, method, dp, minDist, param1=100, param2=100, minRadius=0,maxRadius=0 )

parameter:

• image: Input image, grayscale image should be input

• method: Algorithm for circle detection using Hough transform, its parameters areCV_HOUGH_GRADIENT

• dp: The resolution of Hough space, dp=1when it means that the Hough space is consistent with the size of the input image space, dp=2when the Hough space is half of the input image space, and so on

• minDistis the minimum distance between circle centers. If the distance between two detected circle centers is less than this value, they are considered to be the same circle center.

• param1: Use Canny Canny for edge detection$Canny\; operator\text{'}s$ high threshold, the low threshold is half of the high threshold.

• param2: A common threshold for detecting the center of a circle and determining the radius.

• minRadiusand are maxRadiusthe minimum and maximum values ​​of the detected circle radius

return:

• circles: Output circle vector, including three floating point elements - the abscissa coordinate of the center of the circle, the ordinate of the center of the circle and the circle radius.

3. accomplish

Since Hough circle detection is sensitive to noise, median filtering is first performed on the image.

import cv2 as cv
import numpy as np
import matplotlib.pyplot as plt
# 1 读取图像，并转换为灰度图
planets = cv.imread("./image/star.jpeg")
gay_img = cv.cvtColor(planets, cv.COLOR_BGRA2GRAY)
# 2 进行中值模糊，去噪点
img = cv.medianBlur(gay_img, 7)  
# 3 霍夫圆检测
circles = cv.HoughCircles(img, cv.HOUGH_GRADIENT, 1, 200, param1=100, param2=30, minRadius=0, maxRadius=100)
# 4 将检测结果绘制在图像上
for i in circles[0, :]:  # 遍历矩阵每一行的数据
    # 绘制圆形
    cv.circle(planets, (i[0], i[1]), i[2], (0, 255, 0), 2)
    # 绘制圆心
    cv.circle(planets, (i[0], i[1]), 2, (0, 0, 255), 3)
# 5 图像显示
plt.figure(figsize=(10,8),dpi=100)
plt.imshow(planets[:,:,::-1]),plt.title('霍夫变换圆检测')
plt.xticks([]), plt.yticks([])
plt.show()

---

Summarize:

1. Template matching
   principle: Find the area most similar to the template in a given picture.
   API: use cv.matchTemplate()template matching, and then
   use cv.minMaxLoc()to search for the most matching position.

2. Hough line detection
   principle: Convert the content to be detected into Hough space, use the accumulator to count the optimal solution, and represent the detection results with the processing
   API: cv2.HoughLines()
   Note: The input of this method is a binary image, which must be transformed before detection. Image binarization

3. Hough circle detection
   method: Hough gradient method
   API:cv.HoughCircles()