Yongxing notes -OpenCV-11 template matching (Python)

First, what is the template matching?

• Description: Template matching is one of the most original, the most basic pattern recognition methods to study what pattern of a particular object of the image, thereby identifying the object, which is a matching problem. It is the most basic image processing, the most commonly used matching method.
• Limitations: template match has its own limitations, mainly in that it can only move in parallel, if the rotation or change the size of the original image to match the target occurs, the algorithm is not valid.
• Definition: a small image template is a known target and template matching is to search in a large image, it is known that the figure has to find a target, and that target the same template with the same size, orientation and image elements , through a certain algorithm to find a target in the figures, to determine its position coordinates.

Second, the realization of the principle of template matching (to understand):

• Correlation
  to 8-bit grayscale image, for example, template T (m, n) is stacked on the search graph S (W, H) translation, the region covered by the template is a piece called the search of FIG subgraph, i, j is sub-coordinates on the bottom left of FIG S is searched. Can be measured by the following formula similarity T and is:
  
  its normalized to give correlation coefficient template matching:
  
  when the template and the sub-diagrams, correlation coefficient R (i, j) = 1 , completed in the searched FIG S, after the search, to find a maximum value R, which is the corresponding sub-match the target FIG. Obviously, do image matching operation of the largest in this formula, the slowest. [1]
• Error method
  error method and an error that is a measure of T, the formula is:
  
  where E (i, j) is the minimum value of the matching target. To improve the calculation speed, taking an error threshold, when E (i, j)> stops the calculation point, continues to calculate the next point.
  The larger the template, the slower the rate matching; smaller template matching faster.
• The secondary matching error algorithm
  secondary matching error matching algorithm carried out twice.
  
• When image matching application template matching method, should note the following:
  • 归一化互相关注进行匹配，模板应在（M-P+1）×（N-Q+1）个参考位置上进行相关计算，计算量非常大，必要时可以采用序贯相似检验算法、幅度排序相关算法、FFT相关算法、分层搜索序贯判决等方法对其进行改进，以提高运算速度。
  • 模板匹配具有自身的局限性，主要表现在它只能进行平行移动，若原图像中的匹配目标发生旋转或大小变化，该算法无效。
  • 若原图像中要匹配的目标只有部分可见，也无法完全完成匹配。

三、OpenCV实现：
1、基本模版匹配实现：

result = cv2.matchTemplate(image, templ, method, result=None, mask=None)

• result是一个结果矩阵，假设待匹配图像为 I，宽高为(W,H)，模板图像为 T，宽高为(w,h)。那么result的大小就为(W-w+1,H-h+1) 。 原因是因为，在匹配时，以模板大小的搜索框依次遍历整张图片时，每行需要遍历(W-w+1)次，每列需要遍历(H-h+1)。
  以下列矩阵为例：
  待匹配图像 I.Size(5,5)，模板图像 T.Size(3,3)。以Size(3,3)的搜索框遍历图像I时，x方向需要遍历3次，y方向遍历3次。
  
  多观察发现，归纳出result.Size(W-w+1,H-h+1)。
  T 代表模板图像，I 代表待匹配图像。
  x ,y 代表当前搜索框在 I 矩阵中左上角元素的坐标。
  x’ ,y’ 代表T和搜索框框出来的 I 的矩阵的元素坐标。
  如下图：以归一化相关系数匹配方法为例。
  此时搜索框左上角坐标(x,y)=(0,0)。
  
  
• image: --运行搜索的图像。它必须是8位或32位浮点。
• templ: –搜索的模板。(它必须不大于源图像并且具有相同的数据类型。)
• result: –比较结果图。它必须是单通道32位浮点(If image is W \times H and templ is w \times h , then result is (W-w+1) \times (H-h+1) .)
• method –指定比较方法的参数

method	描述
CV_TM_SQDIFF	平方差匹配法：该方法采用平方差来进行匹配；最好的匹配值为0；匹配越差，匹配值越大。
CV_TM_SQDIFF_NORMED	相关匹配法：该方法采用乘法操作.。完全匹配会得到很大值，不匹配会得到一个很小值或0。
CV_TM_CCORR	相关系数匹配法：1表示完美的匹配；-1表示最差的匹配，该方法使用源图像与模板图像的卷积结果进行匹配，因此，最佳匹配位置在值最大处，值越小匹配结果越差。
CV_TM_CCORR_NORMED	归一化平方差匹配法
CV_TM_CCOEFF	归一化相关匹配法，与相关性匹配方法类似，最佳匹配位置也是在值最大处。
CV_TM_CCOEFF_NORMED	归一化相关系数匹配法 ， 该方法使用源图像与其均值的差、模板与其均值的差二者之间的相关性进行匹配，最佳匹配结果在值等于1处，最差匹配结果在值等于-1处，值等于0直接表示二者不相关。

数学原理（了解即可）：

查找最大最小值：
min_val,max_val,min_indx,max_indx = cv2.minMaxLoc(src, mask=None)
参数：

• src: 查找的图像
• mask : 图像掩膜

返回值：

• min_val :最小值
• max_val :最大值
• min_index: 最小值的索引
• max_index: 最大值的索引

在给定的矩阵中寻找最大和最小值，并给出它们的位置。 该功能不适用于多通道阵列。 如果您需要在所有通道中查找最小或最大元素，要先将阵列重新解释为单通道。

import cv2 as cv

def template_demo():
    tpl = cv.imread("a.jpg") #匹配的模版
    target = cv.imread("first.jpg")  #原图像
    cv.imshow("template image",tpl)
    cv.imshow("target image",target)
    methods = [cv.TM_SQDIFF_NORMED,cv.TM_CCORR_NORMED,cv.TM_CCOEFF_NORMED]#各种匹配算法
    th,tw = tpl.shape[:2]#获取模板图像的高宽
    for md in methods:
        result = cv.matchTemplate(target,tpl,md)
        # result是我们各种算法下匹配后的图像
        
        #获取的是每种公式中计算出来的值，每个像素点都对应一个值
        min_val,max_val,min_loc,max_loc = cv.minMaxLoc(result)
        if md == cv.TM_SQDIFF_NORMED: #相关匹配法的左上角是最小值
            tl = min_loc    #tl是左上角点
        else:
            tl = max_loc
        br = (tl[0]+tw,tl[1]+th)    #右下点
        cv.rectangle(target,tl,br,(0,0,255),2)#画矩形
        cv.imshow("match-%s"%md,target)
if __name__ == "__main__":
    template_demo()
    cv.waitKey(0)   #等待用户操作，里面等待参数是毫秒，我们填写0，代表是永远，等待用户操作
    cv.destroyAllWindows()  #销毁所有窗口

Exercise 11:
use template matching, the interception of a book cover (preferably with bright colors), using the camera to find books and play video and display frame box. (You can try to optimize before the match)

Comment out your answer.