OpenCV provides the function of template matching (Template Matching), which allows you to find the matching position of a specific template (small image) in the target image in the image. Template matching is used in computer vision for object detection, image recognition, feature extraction and other fields.
The following are the basic steps to use template matching in OpenCV:
-
Load images : First, load the target image and the template image to match.
-
Select the matching method : Select the appropriate matching method, such as cv2.TM_CCOEFF, cv2.TM_CCOEFF_NORMED, cv2.TM_CCORR, cv2.TM_CCORR_NORMED, cv2.TM_SQDIFF, or cv2.TM_SQDIFF_NORMED. Each method corresponds to a different matching calculation method.
-
Applying template matching : Use the cv2.matchTemplate() function for template matching, which returns a matrix of matching results.
-
Find the best matching position : In the matching result matrix, find the best matching position by analyzing the pixel values, that is, the matching position in the target image.
Template matching refers to finding the part most similar to image B in the current image A. Generally, image A is called the input image, and image B is called the template image. The operation method of the template matching is to slide the template image B on the image A, and traverse all pixels to complete the matching.
For example, in Figure 15-1, you want to find the "eye" image in the upper left corner within the large image "lena" in the figure. At this time, the large image "lena" is the input image, and the "eye" image is the template image. The way to search is to slide the template image from the upper left corner in the input image, and traverse the entire input image pixel by pixel to find the part that best matches it.
Template matching function description
In OpenCV, template matching is implemented using the function cv2.matchTemplate(). The syntax of this function is:
result = cv2.matchTemplate(image, templ, method[, mask ] )
in:
- image is the original image, which must be an 8-bit or 32-bit floating-point image.
- templ is the template image. It must be smaller or equal in size and of the same type as the original image
. - method is the matching method. This parameter is implemented by TemplateMatchModes, and there are 6 possible values, as shown in Table 15-1.
- mask is the template image mask. It must be the same type and size as the template image templ. Usually this value can use the default value. Currently, only two values TM_SQDIFF and TM_CCORR_NORMED are supported for this parameter.
The return value result of the function cv2.matchTemplate() is a result set formed by the combination of the comparison results of each position, and the type is a single-channel 32-bit floating-point type. If the size of the input image (original image) is W H and the size of the template is w h,
the size of the returned value is (W-w+1)*(H-h+1).
When doing template matching, templates are traversed within the original image. In the horizontal direction:
I represents the input image, T represents the template, R represents the output result image, and x and y represent the position information.
- The starting coordinate of the traversal is the first pixel value from the left of the original image (the serial number starts from 1).
- The last comparison is when the template image is located at the far right of the original image, at this time the position of the pixel in the upper left corner is W-w+1.
Therefore, the size of the return value result in the horizontal direction is W-w+1 (the number of comparisons in the horizontal direction).
In the vertical direction:
- The starting coordinates of the traversal start from the first pixel at the top of the original image.
- The last comparison is when the template image is at the bottom of the original image, and the pixel at the upper left corner is at H-h+1.
Therefore, the size of the return value result in the vertical direction is H-h+1 (the number of comparisons in the vertical direction).
If the original image size is W H and the template size is w h, the size of the returned value is (W-w+1) (H-h+1). That is, the template image is compared (W-w+1) (H-h+1) times within the input image .
It’s too abstract , then take a look at the description below to see if you can understand it better
For example, in Figure 15-2, the small 2×2 square in the upper left is the template image, and the 10×10 image in the lower right is the input image
(the original image). When doing template matching:
- The template image is first placed in the upper left corner of the input image.
- When the template image moves to the right, it can only be located at the farthest right border of the input image. At this time, the pixel in the upper left corner of the template image corresponds to the ninth column of the input image (input image width - template image width + 1 = 10 -2+1 = 9).
- When moving down, the template image can only be located at the bottom boundary of the input image. At this time, the pixel in the upper left corner of the template image corresponds to the 9th row of the input image (height of the input image - height of the template image + 1 = 10-2 + 1 = 9).
According to the above analysis, it can be seen that the size of the result of the comparison satisfies (W-w+1)*(H-h+1), in the above example it is (10-2+1)×(10-2+1), that is, 9 ×9. That is to say, the template image needs to be compared 9×9 = 81 times in total in the input image, and these comparison results will form a 9×9 two-dimensional array.
This feels a little understood.
What is required here 注意is that cv2.matchTemplate()通过参数 method 来决定使用不同的查找方法。the return value result of the function has different meanings for different search methods.
For example:
- When the value of method is cv2.TM_SQDIFF and cv2.TM_SQDIFF_NORMED, the result value is 0, which indicates
the best matching degree, and the larger the value, the worse matching degree. - When the value of method is cv2.TM_CCORR, cv2.TM_CCORR_NORMED, cv2.TM_CCOEFF and
cv2.TM_CCOEFF_NORMED, the smaller the result value, the worse the matching degree, and the larger the result value, the better the matching degree.
It can be seen from the above analysis that different search methods lead to different judgment methods of results. When finding the best match, first determine which method is used, and then determine whether to find the maximum value or the minimum value.
Code example: Use the function cv2.matchTemplate() for template matching.
It is required that the value of the parameter method be set to cv2.TM_SQDIFF to display the return result and matching result of the function.
First go to the Internet to find a picture, and then cut a part from the picture and save it as a picture.
code show as below:
import cv2
from matplotlib import pyplot as plt
img1 = cv2.imread('toukui.png')
#将图片转换为灰度图
img = cv2.cvtColor(img1,cv2.COLOR_BGR2GRAY)
template1 = cv2.imread('toukui2.png')
template = cv2.cvtColor(template1,cv2.COLOR_BGR2GRAY)
th, tw = template.shape[::]
rv = cv2.matchTemplate(img,template,cv2.TM_SQDIFF)
minVal, maxVal, minLoc, maxLoc = cv2.minMaxLoc(rv)
topLeft = minLoc
bottomRight = (topLeft[0] + tw, topLeft[1] + th)
cv2.rectangle(img,topLeft, bottomRight, 255, 2)
plt.subplot(121),plt.imshow(rv,cmap = 'gray')
plt.title('Matching Result'), plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(img,cmap = 'gray')
plt.title('Detected Point'), plt.xticks([]), plt.yticks([])
plt.show()
running result:
The simple target detection effect is here.
multiple template matching
That is to say, multiple targets appear during target detection. In the previous example, we searched for the helmet in the input image, and this sub-image only appeared once in the entire input image. However, in some cases, if there are now 3 people wearing helmets standing under the camera, then multiple matching results need to be found. The function cv2.minMaxLoc() can only find the maximum value, but cannot give the location information of all matching areas. Therefore, if you want to match multiple results, it is impossible to use the function cv2.minMaxLoc(), and you need to use a threshold for processing.
The following step by step describes how to obtain the result of multi-template matching.
1. Get the set of matching locations
The function where() can get a collection of template matching positions. For different inputs, the returned value is different.
- When the input (parameter) is a one-dimensional array, the return value is a one-dimensional index, and there is only one set of index arrays.
- When the input is a two-dimensional array, the return is the position index of the matching value, so there will be two sets of index arrays indicating the position of the return value.
The following code finds the index of the element whose value is greater than 5 in the one-dimensional array a (that is, the position of the element, and the index of the array starts from 0):
import numpy as np
a=np.array([3,6,8,1,2,88])
b=np.where(a>5)
print(b)
The result returned by this code is:
(array([1, 2, 5], dtype=int64),)
说明索引值为 1、2、5 的数组元素，它们的值是大于 5 的。
The above describes the situation when the input value is a one-dimensional array.
When the input value is a two-dimensional array, the function where() returns the index of the value in the two-dimensional array that satisfies the condition.
For example, the following code finds the index of an element in the two-dimensional array am that has a value greater than 5:
import numpy as np
am=np.array([[3,6,8,77,66],[1,2,88,3,98],[11,2,67,5,2]])
b=np.where(am>5)
print(b)
The result returned by this code is:
(array([0, 0, 0, 0, 1, 1, 2, 2], dtype=int64),
array([1, 2, 3, 4, 2, 4, 0, 2], dtype=int64))
The above results show that there is a two-dimensional array am, and its value is:
[[ 3 6 8 77 66]
[ 1 2 88 3 98]
[11 2 67 5 2]]
Among them, [0, 1]、[0, 2]、[0, 3]、[0, 4]、[1, 2]、[1, 4]、[2, 0]、[2, 2]the value of the element at the position is greater than 5.
To sum up, the function np.where() can find out 函数 cv2.matchTemplate()which positions of the return value are greater than the threshold threshold.
In the specific implementation, the statement that can be used is:
loc = np.where( res >= threshold)
In the formula:
- res is the return value of the function cv2.matchTemplate() after template matching.
- threshold is the preset threshold
- loc is an index set of pixels satisfying "res >= threshold". For example, in the above two-dimensional array am, the returned element index set greater than 5 is (array([0, 0, 0, 0, 1, 1, 2, 2], dtype=int64), array([1 , 2, 3, 4,2, 4, 0, 2], dtype=int64)). The two elements in the return value loc represent the row index and column index of the matching value.
2. Cyclic coordinate value
This is nothing to watch, as long as you know what a cycle is
For example, to process multiple values, you usually need to use a loop.
For example, if there is a list with values 71, 23, and 16, and you want to output these values one by one, you can write the code like this:
value = [71,23,16]
for i in value:
print('value 内的值:', i)
Running the above code, the output obtained is:
value 内的值: 71
value 内的值: 23
value 内的值: 16
Therefore, after obtaining the index set of matching values, the following statement can be used to traverse all matching positions and mark these positions: for
i in matching position set: mark matching positions.
3. Using the function zip() in a loop
The function zip() takes an iterable object as a parameter, packs the corresponding elements in the object into tuples, and then returns a list composed of these tuples.
For example, the following code uses the function zip() to pack the corresponding elements in t into tuples, and prints a list composed of these tuples:
x = [1,2,3]
y = [4,5,6]
z = [7,8,9]
t = (x,y,z)
print(t)
for i in zip(*t):
print(i)
In the above code, the statement print(t) outputs the elements in t, and the result is:
([1, 2, 3], [4, 5, 6], [7, 8, 9])
The loop statement for i in zip(*t) packs the elements in t into tuples and outputs them. The result is:
(1, 4, 7)
(2, 5, 8)
(3, 6, 9)
Therefore, if you want to loop through the template matching index set returned by np.where(), you can use the following statement:
for i in zip(*template matching index collection): tag processing
For example, for the array am mentioned above, use the function zip() to loop, and you can get the
set of element indices greater than 5:
import numpy as np
am=np.array([[3,6,8,77,66],[1,2,88,3,98],[11,2,67,5,2]])
print(am)
b=np.where(am>5)
for i in zip(*b):
print(i)
The output of the above code is:
[[ 3 6 8 77 66]
[ 1 2 88 3 98]
[11 2 67 5 2]]
(0, 1)
(0, 2)
(0, 3)
(0, 4)
(1, 2)
(1, 4)
(2, 0)
(2, 2)
4. Adjust coordinates
The function numpy.where() can obtain the set of template matching positions that meet the conditions, and then the function can be used cv2.rectangle()在上述匹配位置绘制矩形来标注匹配位置.
Use the function numpy.where() to find the specified value in the output value of the function cv2.matchTemplate(), and the obtained position index is in the form of "(row number, column number)".
However, the parameter used to specify the vertex in the function cv2.rectangle() uses the
position index in the form of "(column number, row number)". Therefore, before using the function cv2.rectangle() to draw a rectangle, the position index obtained by the function numpy.where() must be "swapped". The following statement can be used to exchange the positions of rows and columns in loc: loc[::-1]
The following statement exchanges the positions of two elements in loc:
import numpy as np
loc = ([1,2,3,4],[11,12,13,14])
print(loc)
print(loc[::-1])
Among them, the output corresponding to the statement print(loc) is:
([1, 2, 3, 4], [11, 12, 13, 14])
The output corresponding to the statement print(loc[::-1]) is:
([11, 12, 13, 14], [1, 2, 3, 4])
5. Mark the location of the matching image
The last step is to use the function cv2.rectangle() to mark the specific position of the matching image, and specify the original image to be marked, the diagonal vertices, the color, and the width of the rectangle edge.
With respect to the diagonal vertices of a rectangle:
- One of the diagonal vertices A can be obtained from the determined "matching position collection" that satisfies the conditions through the for loop statement.
- Another diagonal vertex can be obtained by calculating the position of vertex A and the width (w) and height (h) of the template.
Thus, the statement that marks each matching location is:
for i in 匹配位置集合:
cv2.rectangle(输入图像,i, (i[0] + w, i[1] + h), 255, 2)
Code example: Use template matching to mark multiple sub-images within an input image that match a template image.
code show as below:
import cv2
import numpy as np
img = cv2.imread('lena4.bmp',0)
template = cv2.imread('lena4Temp.bmp',0)
w, h = template.shape[::-1]
res = cv2.matchTemplate(img,template,cv2.TM_CCOEFF_NORMED)
threshold = 0.9
loc = np.where( res >= threshold)
for pt in zip(*loc[::-1]):
cv2.rectangle(img, pt, (pt[0] + w, pt[1] + h), 255, 1)
cv2.imshow("template",template)
cv2.imshow("result1",img)
cv2.waitKey()
cv2.destroyAllWindows()
The result of the operation is as follows:
It can be seen that multiple subgraphs matching the template image in the input image are marked