Template matching is a method of locating a target in an image. By sliding the input image pixel by pixel on the actual image, the feature similarity is calculated to determine the probability that the position of the current slider image is the target image.
In Opencv, template matching defines 6 similarity comparison methods:
CV_TM_SQDIFF Squared Difference Matching Method: Calculate the sum of distances between image pixels. The best match is 0. The larger the value, the lower the probability of being the target.
CV_TM_CCORR Correlation matching method: a multiplication operation; from small to large values, the matching probability becomes higher and higher.
CV_TM_CCOEFF correlation coefficient matching method: from -1 to 1, the matching probability is getting higher and higher.
CV_TM_SQDIFF_NORMED Normalized squared difference matching
CV_TM_CCORR_NORMED Normalized correlation matching
CV_TM_CCOEFF_NORMED Normalized correlation coefficient matching
The tracking of moving objects in video files is essentially target tracking on images. The template matching method can be used to achieve a simple matching and tracking effect. However, template matching requires moving pixel by pixel to match the target image, which requires a large amount of calculation and poor real-time performance.
As a benefit for this article, you can receive a Qt development learning package and technical videos for free , including (Qt practical projects, C++ language basics, C++ design patterns, introduction to Qt programming, QT signal and slot mechanism, QT interface development-image drawing, QT network, QT database programming, QT project practice, QSS, OpenCV, Quick module, interview questions, etc.) ↓↓↓↓↓↓See below↓↓Click at the bottom of the article to receive the fee↓↓
The following code implements target tracking based on template matching. Draw a rectangle on the video by clicking the mouse to define the target to be tracked. When the target is matched, the input image is refreshed with the target image:
#include "core/core.hpp"
#include "highgui/highgui.hpp"
#include "imgproc/imgproc.hpp"
#include<iostream>
using namespace cv;
using namespace std;
Mat image; //视频流
Mat imageCopy; //绘制矩形框时用来拷贝原图的图像
Mat rectImage; //子图像
bool leftButtonDownFlag=false; //左键单击后视频暂停播放的标志位
Point originalPoint; //矩形框起点
Point processPoint; //矩形框终点
int resultRows; //模板匹配result的行
int resultcols; //模板匹配result的列
Mat ImageResult; //模板匹配result
double minValue; //模板匹配result最小值
double maxValude; //模板匹配result最大值
Point minPoint; //模板匹配result最小值位置
Point maxPoint; //模板匹配result最大值位置
int frameCount=0; //帧数统计
void onMouse(int event,int x,int y,int flags ,void* ustc); //鼠标回调函数
int main(int argc,char*argv[])
{
VideoCapture video(argv[1]);
double fps=video.get(CV_CAP_PROP_FPS); //获取视频帧率
double pauseTime=1000/fps; //两幅画面中间间隔
namedWindow("Man",0);
setMouseCallback("Man",onMouse);
while(true)
{
if(!leftButtonDownFlag) //鼠标左键按下绘制矩形时，视频暂停播放
{
video>>image;
frameCount++; //帧数
}
if(!image.data||waitKey(pauseTime+30)==27) //图像为空或Esc键按下退出播放
{
break;
}
if(rectImage.data)
{
ImageResult=Mat::zeros(resultRows,resultcols,CV_32FC1);
matchTemplate(image,rectImage,ImageResult,CV_TM_SQDIFF); //模板匹配
minMaxLoc(ImageResult,&minValue,&maxValude,&minPoint,&maxPoint,Mat()); //最小值最大值获取
rectangle(image,minPoint,Point(minPoint.x+rectImage.cols,minPoint.y+rectImage.rows),Scalar(0,0,255),2);
//更新当前模板匹配的模板
Mat resultImage=image(Rect(minPoint,Point(minPoint.x+rectImage.cols,minPoint.y+rectImage.rows)));
rectImage=resultImage.clone();
//当前帧数输出到视频流
stringstream ss;
ss<<frameCount;
string h="Current frame is: ";
string fff=h+ss.str();
putText(image,fff,Point(50,60),CV_FONT_HERSHEY_COMPLEX_SMALL,2,Scalar(0,0,255),2);
}
imshow("Man",image);
}
return 0;
}
//*******************************************************************//
//鼠标回调函数
void onMouse(int event,int x,int y,int flags,void *ustc)
{
if(event==CV_EVENT_LBUTTONDOWN)
{
leftButtonDownFlag=true; //标志位
originalPoint=Point(x,y); //设置左键按下点的矩形起点
processPoint=originalPoint;
}
if(event==CV_EVENT_MOUSEMOVE&&leftButtonDownFlag)
{
imageCopy=image.clone();
processPoint=Point(x,y);
if(originalPoint!=processPoint)
{
//在复制的图像上绘制矩形
rectangle(imageCopy,originalPoint,processPoint,Scalar(0,0,255),2);
}
imshow("Man",imageCopy);
}
if(event==CV_EVENT_LBUTTONUP)
{
leftButtonDownFlag=false;
Mat subImage=image(Rect(originalPoint,processPoint)); //子图像
rectImage=subImage.clone();
resultRows=image.rows-rectImage.rows+1;
resultcols=image.cols-rectImage.rows+1;
imshow("Sub Image",rectImage);
}
} Tracking targets selected by the box:
Tracking effect:
When the target characteristics do not change very quickly, the tracking effect is okay, but there are also two problems:
1. The speed of template matching is very slow: the original video image size is a 1920*1080 color RGB image, and it is directly used for target matching. It takes about 1 second on my machine, which is not real-time at all. The optimization direction can consider pyramid scaling of both the original image and the input image, and the speed improvement should be obvious.
2. Tracking drift exists: This online tracking method, which updates the tracking target at any time, can easily lead to tracking drift problems, especially when the characteristics of the target itself change greatly. In severe cases, the target may be completely lost and permanently lost. Tracking of targets. The optimization direction can consider using the cumulative weight method to generate the next input image for the target images detected in history. For another situation where the target is completely lost, a threshold can be set for the probability of target matching. If it is less than the threshold, an object that is very different from the target may be detected, and the input image will not be updated.
As a benefit for this article, you can receive a Qt development learning package and technical videos for free , including (Qt practical projects, C++ language basics, C++ design patterns, introduction to Qt programming, QT signal and slot mechanism, QT interface development-image drawing, QT network, QT database programming, QT project practice, QSS, OpenCV, Quick module, interview questions, etc.) ↓↓↓↓↓↓See below↓↓Click at the bottom of the article to receive the fee↓↓