008 OpenCV matchTemplate template matching

Table of contents

2. Principle of template matching algorithm

1. Environment

The usage environment of this article is:

Windows10
Python 3.9.17
opencv-python 4.8.0.74

2. Principle of template matching algorithm

cv.matchTemplateis a function in the OpenCV library that is used to find features in an image that match a template. Its main application scenarios are in the fields of image processing, computer vision and pattern recognition.

Algorithm principle: cv.matchTemplateThe function finds the best matching position by calculating the similarity between the input image and the template image. It uses a sliding window method to move the template image over the input image and calculates the difference in pixel values within each window. Then, according to the selected matching method (such as squared difference, normalized squared difference, correlation coefficient, etc.), the differences are weighted and summed to obtain a matching score matrix. Finally, the function returns the maximum value and its position in the score matrix as the best matching position.

Function API:

retval, result = cv2.matchTemplate(image, templ, method, mask)

Parameter Description:

image: Input image, usually a grayscale image.
templ: Template image, which can be a grayscale image or a color image.
method: Matching method, commonly used ones are:
- cv2.TM_SQDIFF: Squared difference matching method, the smaller the calculation result is, the higher the degree of matching is.
- cv2.TM_SQDIFF_NORMED: Normalized squared difference matching method. The closer the calculation result is to 0, the higher the degree of matching.
- cv2.TM_CCORR: Correlation coefficient matching method. The closer the calculation result is to 1, the higher the degree of matching.
- cv2.TM_CCORR_NORMED: Normalized correlation coefficient matching method. The closer the calculation result is to 1, the higher the degree of matching.
- cv2.TM_CCOEFF: Correlation coefficient matching method. The closer the calculation result is to 1, the higher the degree of matching.
- cv2.TM_CCOEFF_NORMED: Normalized correlation coefficient matching method. The closer the calculation result is to 1, the higher the degree of matching.
Mask: Optional parameter. Mask is a binary image that acts on parameter temp1. The effective area participates in template matching calculations.

Return value description:

result: Matching result matrix, each element represents the matching degree of the corresponding position.
minVal, maxVal, minLoc, maxLoc: The minimum value, maximum value, minimum value position and maximum value position of the best matching position.

The following is a simple explanation of the cv.matchTemplate() algorithm:

Preprocessing: First, convert the input image and template image into grayscale images. This is because the cv.matchTemplate() algorithm only supports grayscale images.
Sliding Template: The algorithm then slides the template over the input image. For each position (left to right, top to bottom), a matching score is calculated. This score depends on the similarity between the current location image and the template.
Calculate matching score: Calculate the matching score of the current position according to the selected matching method. For example, if the normalized correlation coefficient method is selected, the algorithm calculates the correlation coefficient between the input image and the template at the current location. Other methods, such as normalized correlation, squared difference, etc., have different calculation methods.
Find the best matching position: Among all positions, the algorithm finds the position with the highest matching score. This position is the best matching position.
Return result: Finally, the function will return a structure containing all matching area information. This structure contains the coordinates of each matching area, matching score and other information.

It should be noted that due to the limitation of the sliding window, if some areas of the template and image do not match, these areas will not be counted in the matching score. This is accomplished using the mask parameter. The mask is a two-dimensional array of the same size as the template, and the values can be 0 or 1. Positions in the mask that are 1 will count toward the match score, positions that are 0 will not.

Overall, cv.matchTemplate() is a powerful tool for finding areas in an image that best match a template. It is widely used in fields such as image processing, computer vision and pattern recognition.

3. Code demonstration

The three pictures required in the code are given here, lena_tmpl.jpg, and the following are the original pictures:

tmpl.png, the following is a template image, which means: find a similar area in the original image and draw it with a rectangle.

mask.png, which must be the same size as tmp1.png, is a binary image used to indicate which areas in the template image are used for template matching during the template calculation process.

from __future__ import print_function
import sys
import cv2 as cv

# 全局变量
use_mask = False
img = None
templ = None
mask = None
image_window = "Source Image"
result_window = "Result window"

match_method = 0
max_Trackbar = 5

def main():
    # 读取三张图
    global img
    global templ
    img = cv.imread('data/lena_tmpl.jpg', cv.IMREAD_COLOR) # 图片1
    templ = cv.imread('data/tmpl.png', cv.IMREAD_COLOR) # 图片2

    global use_mask
    use_mask = True
    global mask
    mask = cv.imread('data/mask.png', cv.IMREAD_COLOR ) # 图片3

    if ((img is None) or (templ is None) or (use_mask and (mask is None))):
        print('Can\'t read one of the images')
        return -1
    cv.namedWindow( image_window, cv.WINDOW_AUTOSIZE )
    cv.namedWindow( result_window, cv.WINDOW_AUTOSIZE )
    # 创建滑条
    trackbar_label = 'Method: \n 0: SQDIFF \n 1: SQDIFF NORMED \n 2: TM CCORR \n 3: TM CCORR NORMED \n 4: TM COEFF \n 5: TM COEFF NORMED'
    cv.createTrackbar( trackbar_label, image_window, match_method, max_Trackbar, MatchingMethod )
    # 执行模板匹配
    MatchingMethod(match_method)
    cv.waitKey(0)
    return 0

def MatchingMethod(param):
    global match_method
    match_method = param
    img_display = img.copy()
    # 模板匹配
    method_accepts_mask = (cv.TM_SQDIFF == match_method or match_method == cv.TM_CCORR_NORMED)
    if (use_mask and method_accepts_mask):
        result = cv.matchTemplate(img, templ, match_method, None, mask)
    else:
        result = cv.matchTemplate(img, templ, match_method)
    # 将结果进行归一化到[0, 1]
    cv.normalize( result, result, 0, 1, cv.NORM_MINMAX, -1 )
    # 找到最佳匹配
    _minVal, _maxVal, minLoc, maxLoc = cv.minMaxLoc(result, None)
    # 得到的匹配位置，即：一个矩形框
    if (match_method == cv.TM_SQDIFF or match_method == cv.TM_SQDIFF_NORMED):
        matchLoc = minLoc
    else:
        matchLoc = maxLoc
    # 可视化
    cv.rectangle(img_display, matchLoc, (matchLoc[0] + templ.shape[0], matchLoc[1] + templ.shape[1]), (0,255,0), 2, 8, 0 )
    cv.rectangle(result, matchLoc, (matchLoc[0] + templ.shape[0], matchLoc[1] + templ.shape[1]), (0,0,255), 2, 8, 0 )
    cv.imshow(image_window, img_display)
    cv.imshow(result_window, result)
    pass
if __name__ == "__main__":
    main()

The following are the operating effects of different algorithms. Which algorithms are bad can be seen at a glance. You can play with them yourself.

\n 0: SQDIFF

\n 1: SQDIFF

\n 2: TM CCORR

\n 3: TM CCORR

\n 4: TM COEFF

\n 5: TM COEFF