Organizing when learning template matching algorithms

reference

[1] Lin Guangdong, Huang Guanghong. Commonly used target detection methods in industry and their prospects [J]. Integrated Circuit Applications, 2022, 39(12):54-57.DOI:10.19339/j.issn.1674- 2583.2022.12.020.
[1] Feng Bodi, Yang Haitao, Li Gaoyuan, Wang Jinyu, Zhang Changgong. Research review of neural networks in SAR image target recognition [J]. Journal of Ordnance and Equipment Engineering, 2021, 42( 10):15-22.
https://blog.csdn.net/weixin_57440207/article/details/122647050
https://blog.csdn. net/qq_46418503/article/details/119675943
[1] Zhang Weixing. Application of template matching technology in image recognition [J]. China High-tech, 2022(08):18-20.DOI :10.13535/j.cnki.10-1507/n.2022.08.07.

Overview of template matching algorithms

Template matching is a relatively primitive target detection method. The idea is to apply the target image template on all windows of the image to be detected one by one Match is performed and the window with the best matching result is found. If the matching degree at this position is greater than a certain threshold, it is considered that the target is detected at this position.
The recognition process of template matching method is divided into two stages: template construction and classification recognition. First, the labeled training images are constructed to form a series of template sets, and then in the classification stage, the data to be tested is registered with the template set according to a certain similarity registration rule to obtain the recognition result.

• The simplest template matching method isdirect template matching method (similarity measurement method), that is, the training image is directly used as a template, Usually in order to improve the recognition accuracy, related operations such as rotation, displacement, and denoising are performed. The problem isWhen there are many target categories, there are problems such as large template library storage and high computational complexity, which greatly limits the application of this method in actual scenarios. . (Multi-target template matching and Adaptive target matching)

• In order to suppress the clutter interference of the image, improve the robustness of the algorithm, and obtain a more robust recognition effect, researchers proposed first perform relevant transformation operations on the original image, and then The correlation filter matching method is used to perform template matching in the transform domain. Typical methods include the synthetic decision function method proposed by Casasent. In addition, there are also methods such as the minimum average energy correlation filter and the minimum noise and correlation energy filter, which have also achieved good recognition results.

• High-speed template matching method
  The basic search strategy of the traditional template matching algorithm is ergodic. In order to find the optimal matching point, the traditional method must search every Regional correlation matching calculations are performed on pixels. Image correlation matching requires a large amount of data and calculations, and the matching speed is slow. The usual solution is to perform parallel calculations to improve the matching speed. However, due to the limitations of computer hardware, the matching speed will also be affected. varying degrees of impact. The detection speed of template matching mainly depends on the search strategy of the algorithm.

  • 1. Sequential Similarity Detection Algorithm (SSDA)

    Sequential Similarity Detection Algorithm (SSDA) is an efficient image matching algorithm proposed for the traditional template matching algorithm. The idea is to artificially set a fixed threshold to terminate the calculation at the mismatched position early, thereby reducing the amount of calculation and improving the computing speed.
    The steps are as follows:

    • 1. Select an error criterion as the standard for terminating the calculation of mismatch points, usually the absolute error can be selected;
    • 2. Set an invariant threshold;
    • 3. Randomly select a point in the sub-image, calculate the absolute error value between it and the corresponding point in the template, and accumulate the errors of each random point pair. If the error exceeds the set threshold at the rth time of accumulation, stop accumulating. , record the accumulated number r at this time;
    • 4. Calculate the error e for the entire image, and you can get a curved surface composed of r values. The position corresponding to the maximum value of the surface is the best matching position of the template. This is because the point requires multiple accumulated errors to exceed the threshold, so it is the most likely matching location relative to other points.

  • 2. Hierarchical search sequential decision algorithm

    The hierarchical algorithm first performs hierarchical processing on the target image, reducing the dimension of the image to obtain a small low-resolution image, and then hierarchically processes the small image again to obtain a smaller image. After n layers of processing, it is compared with The original images are combined to construct n layered images with different dimensions and different resolutions. These images are sorted from large to small according to the number of layers. Then, known templates can be used to perform hierarchical search and hierarchical matching, thereby achieving The purpose is to improve the matching speed and matching accuracy.

  • 3. Amplitude sorting related algorithm

  • 4. FFT related algorithms

  • 5. There is also a way to divide the template movement on the image into coarse search and fine searchTwo-stage matching method.

    • Principle: General images have strong autocorrelation. Therefore, the similarity calculated by template matching forms a gentle pattern centered on the place where the object exists. peak. In this way, even if the template is slightly deviated from the real position of the image object during template matching, it still shows a very high degree of similarity.
    • Steps: First perform a rough search. It does not move the template one pixel at a time, but overlaps the template and the image every few pixels, and calculates the matching scale. Thus, the approximate range in which the object exists can be found. Then, within this rough range, the template is moved every other pixel, and the location of the object is determined based on the obtained matching scale. In this way, the overall number of template matching calculations is reduced, the calculation time is short, and the matching speed is improved.
    • Disadvantages: There is a danger of missing the most appropriate locations in the image using this method.

• High-precision positioning template matching. In order to obtain the precise position of the object in the image, it is always hoped that the similarity distribution will be as sharp as possible. To achieve this goal, feature matching methods based on pattern contours have been proposed. Compared with the general matching method, the pattern contour matching shows a sharper distribution of similarity, which is conducive to accurate positioning.

Generally speaking, when performing template matching when the size and direction of the detection object are unknown, it is necessary to have templates of various sizes and directions, use various templates for matching to find the most consistent object and its location. In addition, when the shape of the object is complex, it is best not to use the entire object as a template, but to divide the object into several patterns and use each sub-pattern as a template for matching , and then study the positional relationship between the sub-patterns to find the position of the object in the image. In this way, even if the shape of the object changes slightly, the position can be determined well.

application

The template matching method is often used inthe field of industrial product appearance inspection; if the shape of the target is fixed, the template matching method can be used as a simple and effective method. Object detection methods are used, such as detecting rigid objects of fixed size photographed at fixed positions.

Template matching algorithm implementation

principle

If the input image (original image) size is W * H and the template size is w * h, the size of the return value is (W-w+1)*(H-h+1). As shown below.

Function cv2.matchTemplate():

Opencv provides matchTemplate to support template matching methods. Methods to measure whether the target image matches the sliding window during template matching include squared difference, standard squared difference, correlation, standard correlation, correlation coefficient, standard correlation system number, etc.

The syntax format is:

result=cv2.matchTemplate(image,templ,method[,mask])

• image: It is the original image, which must be an 8-bit or 32-bit floating point image (grayscale image);
• templ: is the template image. It must be smaller than or equal to the size of the original image and of the same type as the original image;
• method: is the matching method. This parameter is implemented through TemplateMatchModes and has 6 possible values, as shown in the table:
  
  Its specific corresponding calculation formula:
  
• mask: Mask for the template image. It must be of the same type and size as the template image templ. Normally, the default value can be used for this value. Currently, this parameter only supports two values: TM_SQDIFF and TM_CCORR_NORMED.

The function cv2.matchTemplate()'sreturn valueresult is a result set. Type is single-channel 32-bit floating point. It is composed of the comparison results of each position. (position of the upper left corner of the template)

Find the best value

It should be noted that the function cv2.matchTemplate() determines to use different search methods through the parameter method. For different search methods, the return value result has different meanings.

• When the value of method is cv2.TM_SQDIFF and cv2.TM_SQDIFF_NORMED, the result value of 0 indicates the best match, and the larger the value, the worse the match.
• When the value of method is cv2.TM_CCORR, cv2.TM_CCORR_NORMED, cv2.TM_CCOEFF and cv2.TM_CCOEFF_NORMED, the smaller the value of result, the worse the matching, and the larger the value, the better the matching.

To find the maximum value (extreme value) and the location of the maximum value, you can use the cv2.minMaxLoc() function to achieve this. The syntax format is as follows:

• minVal,maxVal,minLoc,maxLoc=cv2.minMaxLoc(src[,mask])
  • src: is a single-channel array.
  • minVal: is the minimum value returned. If there is no minimum value, it can be NULL (null value).
  • maxVal: is the maximum value returned. If there is no minimum value, it can be NULL.
  • minLoc: is the position of the maximum value. If there is no maximum value, it can be NULL.
  • maxLoc: is the position of the maximum value. If there is no maximum value, it can be NULL.
  • mask: used to select a subset of masks, optional

You can use the function cv2.rectangle() to mark this position in white.

• Img=cv.rectangle(img,pt1,pt2,color[,thickness])
  • img: Indicates the target image to be marked.
  • pt1: is the vertex of the rectangle.
  • pt2: is the diagonal vertex of pt1.
  • color: is the color or gray level (grayscale image) of the rectangle to be drawn.
  • Thickness: is the width of the rectangular edge.

Multiple target template matching

In some cases, the template image to be searched may appear multiple times in the input image, and in this case multiple matching results need to be found.
The function where() in the numpy module can obtain the set of template matching positions. For different inputs, the returned values ​​are different.

• When the input (parameter) is a one-dimensional array, the return value is a one-dimensional index, with only one set of index arrays.
• When the input is a two-dimensional array, the position index of the matching value is returned, so there will be two sets of index arrays representing the position of the return value.

Adaptive size template matching

The shortcomings of general template matching: it can only perform translational searches and cannot cope with scenes where the target shape, angle, and size change. An adaptive size template matching method is proposed. The idea is to perform template matching by scaling a certain ratio each time after loading the template image to obtain ROIs at different ratios, compare the similarity between all ROI areas and our template images, and select the matching image with the highest similarity. Get the best matching ratio at the same time.