Section 2 Object Tracking
OpenCV's video module provides several object tracking methods based on optical flow.
1、cv::buildOpticalFlowPyramid、cv::calcOpticalFlowPyrLK
1) cv::buildOpticalFlowPyramid : Construct an image pyramid that can be passed to calcOpticalFlowPyrLK.
int cv::buildOpticalFlowPyramid (InputArray img,OutputArrayOfArrays pyramid,Size winSize,int maxLevel,bool withDerivatives = true,int pyrBorder = BORDER_REFLECT_101,int derivBorder = BORDER_CONSTANT,bool tryReuseInputImage = true)
The function returns the number of layers in the construction pyramid. Can be less than maxLevel. The parameters are as follows:
| parameter name | Parameter Description |
|---|---|
| img | 8-bit input image |
| pyramid | Output pyramid |
| winSize | The window size of the optical flow algorithm. Must be no less than the winSize parameter of calcOpticalFlowPyrLK. The padding required for the pyramid level needs to be calculated. |
| maxLevel | The maximum pyramid level number starting from 0. |
| withDerivatives | Set to precompute the gradient for each pyramid level. If the pyramid is built without gradients, then calcOpticalFlowPyrLK will calculate it internally. |
| pyrBorder | The border mode of the pyramid layer. |
| derivBorder | Gradient border pattern. |
| tryReuseInputImage | If possible, put the ROI of the input image into a pyramid. You can pass false to force the data to be copied. |
2) cv::calcOpticalFlowPyrLK : Use the iterative Lucas-Kanade method with pyramids to calculate the optical flow of the sparse feature set.
void cv::calcOpticalFlowPyrLK(InputArray prevImg,InputArray nextImg,InputArray prevPts,InputOutputArray nextPts,OutputArray status,OutputArray err,Size winSize = Size(21, 21),int maxLevel = 3,TermCriteria criteria = TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 30, 0.01),int flags = 0,double minEigThreshold = 1e-4)
This function implements a sparse iterative version of Lucas-Kanade optical flow in the pyramid. See [Jean-Yves Bouguet. Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm . Intel Corporation , 5, 2001.]. This function is parallelized with the TBB library. The parameters are as follows:
| parameter name | Parameter Description |
|---|---|
| prevImg | The first 8-bit input image or pyramid constructed by buildOpticalFlowPyramid. |
| nextImg | A second input image or pyramid of the same size and type as the previous one. |
| prevPts | Need to find the 2D point vector of the flow; the point coordinates must be single-precision floating point numbers. |
| nextPts | The output vector of a two-dimensional point (with single-precision floating-point coordinates), which contains the calculated new position of the input feature in the second image; when the OPTFLOW_USE_INITIAL_FLOW flag is passed, the size of the vector must be the same as the size in the input. |
| status | Output state vector (unsigned character); if the process corresponding to the feature is found, each element of the vector will be set to 1, otherwise it will be set to 0. |
| err | Error output vector; each element of the vector is set to error for the corresponding feature, and the type of error metric can be set in the flags parameter; if the stream is not found, the error is undefined (use the status parameter to find such cases). |
| winSize | The size of the search window for each pyramid level. |
| maxLevel | The maximum number of pyramid levels starting from 0; if set to 0, no pyramid (single level) is used; if set to 1, two levels are used, and so on; if the pyramid is passed to the input, the algorithm will use and There are as many levels as the pyramid, but no more than maxLevel. |
| criteria | Parameter, specify the termination condition of the iterative search algorithm (after the specified maximum number of iterations criteria.maxCount, or when the search window moves less than criteria.epsilon. |
| flags | Operation flag: OPTFLOW_USE_INITIAL_FLOW uses the initial estimate stored in nextPts; if this flag is not set, copy prevPts to nextPts and treat it as the initial estimate. OPTFLOW_LK_GET_MIN_EIGENVALS uses the smallest eigenvalue as the error metric (see minEigThreshold description); if this flag is not set, the distance between the color patch between the original point and the moving point of L1 divided by the number of pixels in the pixel window is used as the error metric . |
| minEigThreshold | The algorithm calculates the minimum eigenvalue of the 2x2 normal matrix of the optical flow equation (this matrix is divided by [[Jean-Yves Bouguet. Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm . Intel Corporation , 5, 2001.] The number of pixels in the window; if this value is less than minEigThreshold, the corresponding function will be filtered out and the process will not be processed, so it can eliminate dead pixels and improve performance. |
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
int main()
{
cv::VideoCapture cap("videos/vtest.avi");
if(!cap.isOpened()){
cerr << "cannot open camera\n";
return EXIT_FAILURE;
}
cv::Mat frame,gray,grayPre,framePre,status,err;
const int maxLevel = 3;
vector<cv::Point2f> prevPts, nextPts;
vector<cv::Mat> pyramid1,pyramid2;
cap >> frame;
if(frame.empty()){
cerr << "grab first frame error.\n";
return EXIT_FAILURE;
}
cv::cvtColor(frame,grayPre,cv::COLOR_BGR2GRAY);
cv::Size subPixWinSize(10,10);
cv::TermCriteria termcrit(cv::TermCriteria::COUNT|cv::TermCriteria::EPS,20,0.03);
while(cap.isOpened()){
cap >> frame;
if(frame.empty()){
cerr << "cannot grab frame from camera.\n";
break;
}
cv::cvtColor(frame,gray,cv::COLOR_BGR2GRAY);
// 检测触点
goodFeaturesToTrack(gray, nextPts, 100, 0.01, 2.0);
cornerSubPix(gray, nextPts, subPixWinSize, cv::Size(-1,-1), termcrit);
goodFeaturesToTrack(grayPre, prevPts, 100, 0.01, 2.0);
cornerSubPix(gray, prevPts, subPixWinSize, cv::Size(-1,-1), termcrit);
// 构造流光金字塔
cv::buildOpticalFlowPyramid(gray,pyramid1,cv::Size(21,21),maxLevel);
cv::buildOpticalFlowPyramid(grayPre,pyramid2,cv::Size(21,21),maxLevel);
// 使用LK流光算法检测
cv::calcOpticalFlowPyrLK(pyramid1,pyramid2,prevPts,nextPts,status,err);
gray.copyTo(grayPre);
size_t i, k;
for( i = k = 0; i < nextPts.size(); i++ ){
cv::circle( frame, nextPts[i], 3, cv::Scalar(0,0,255), -1, 8);
}
// 显示图像
cv::imshow("frame",frame);
if(cv::waitKey(10) == 27){
break;
}
}
return 0;
}
2、cv::calcOpticalFlowFarneback
Use Gunnar Farneback's algorithm to calculate dense optical flow.
void cv::calcOpticalFlowFarneback(InputArray prev,InputArray next,InputOutputArray flow,double pyr_scale,int levels,int winsize,int iterations,int poly_n,double poly_sigma,int flags)
This function uses [Gunnar Farnebäck. Two-frame motion estimation based on polynomial expansion . In Image Analysis , pages 363–370. Springer, 2003.] algorithm to find the optical flow for each previous pixel, so:
prev ( y , x ) ∼ next ( y + flow ( y , x ) [ 1 ] , x + flow ( y , x ) [ 0 ] ) \texttt{prev} (y,x) \sim \texttt{next} ( y + \texttt{flow} (y,x)[1], x + \texttt{flow} (y,x)[0]) prev ( and ,x)∼next(y+flow(y,x)[1],x+flow(y,x)[0])
The parameters are as follows:
| parameter name | Parameter Description |
|---|---|
| prev | The first 8-bit single-channel input image. |
| next | The second input image with the same size and type as the previous one. |
| flow | The calculated stream image has the same size as prev and the type is CV_32FC2. |
| pyr_scale | Parameter, specify the image scale (<1) to build a pyramid for each image; pyr_scale = 0.5 means the classic pyramid, where the next layer is twice smaller than the previous one. |
| levels | Including the number of pyramid layers of the initial image; level = 1 means no additional layers are created, only the original image is used. |
| winsize | Average window size; a larger value can improve the algorithm's robustness to image noise and provide more opportunities for fast motion detection, but will produce more motion field blur. |
| iterations | The number of iterations of the algorithm performed at each pyramid level. |
| poly_n | 用于查找每个像素中的多项式展开的像素邻域的大小； 较大的值表示将使用更平滑的表面近似图像，从而产生更鲁棒的算法和更模糊的运动场，通常poly_n = 5或7。 |
| poly_sigma | 高斯标准偏差，用于平滑用作多项式展开基础的导数； 对于poly_n = 5，可以设置poly_sigma = 1.1，对于poly_n = 7，好的值应该是poly_sigma = 1.5。 |
| flags | 操作标志可以是以下各项的组合：OPTFLOW_USE_INITIAL_FLOW使用输入流作为初始流近似值。OPTFLOW_FARNEBACK_GAUSSIAN使用高斯winsize×winsize过滤器代替相同大小的盒式过滤器进行光流 估计 通常，此选项使z流量比使用箱式过滤器更精确，但速度较低； 通常，应将高斯窗口的winsize设置为较大的值，以实现相同级别的鲁棒性。 |
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main(int argc, char* argv[])
{
cv::Mat res, framePre, frameNext, frameNextOriginal, frameNextOriginalC;
// 读取视频文件
cv::VideoCapture cap;
cap.open("videos/vtest.avi");
if(!cap.isOpened()){
cerr << "cannot open video.\n";
return EXIT_FAILURE;
}
cv::namedWindow("video", cv::WINDOW_AUTOSIZE);
// 读取第一帧图像
cap >> framePre;
// 转换成灰度图像
cvtColor(framePre, framePre, COLOR_BGR2GRAY);
cv::Size tamano((int)cap.get(cv::CAP_PROP_FRAME_WIDTH), (int)cap.get(cv::CAP_PROP_FRAME_HEIGHT));
while(cap.isOpened()) {
// 读取下一帧图像
cap >> frameNext;
if (frameNext.empty()) {
cerr << "cannot grab frame from video.\n";
break;
}
frameNext.copyTo(frameNextOriginalC);
// 转换成灰度图像
cvtColor(frameNext, frameNext, COLOR_BGR2GRAY);
frameNext.copyTo(frameNextOriginal);
// 计算流光
cv::calcOpticalFlowFarneback(framePre, frameNext, res, .4, 1, 12, 2, 8, 1.2, 0);
// 绘制物体运动方向
for (int y = 0; y < frameNext.rows; y += 5) {
for (int x = 0; x < frameNext.cols; x += 5)
{
// get the flow from y, x position * 3 for better visibility
const Point2f flowatxy = res.at<Point2f>(y, x) * 1;
// 绘制流光方向
line(frameNextOriginalC, Point(x, y), Point(cvRound(x + flowatxy.x), cvRound(y + flowatxy.y)), Scalar(255, 0, 0));
// 绘制原始点
circle(frameNextOriginalC, Point(x, y), 1, Scalar(0, 0, 0), -1);
}
}
frameNextOriginal.copyTo(framePre);
imshow("video", frameNextOriginalC);
if (cv::waitKey(1) == 27) {
break;
}
}
cap.release();
return 0;
}
3、cv::CamShift
查找对象的中心，大小和方向。
RotatedRect cv::CamShift(InputArray probImage,Rect& window,TermCriteria criteria)
参数如下：
| 参数名称 | 参数描述 |
|---|---|
| probImage | 对象直方图的反向投影。 请参阅calcBackProject。 |
| window | 初始搜索窗口。 |
| criteria | 底层meanShift的停止条件。返回值（在旧的接口中）CAMSHIFT收敛所需的迭代次数该函数实现CAMSHIFT对象跟踪算法 。首先，它使用meanShift查找对象中心，然后调整窗口大小并找到最佳旋转角度。 该函数返回旋转后的矩形结构，其中包括对象的位置，大小和方向。 可以使用RotatedRect :: boundingRect()获得搜索窗口的下一个位置。 |
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
int main()
{
cv::VideoCapture cap("videos/football-in-motion.mp4");
// cv::VideoCapture cap(0);
if(!cap.isOpened()){
cerr << "cannot open video\n";
return EXIT_FAILURE;
}
cv::namedWindow("video");
cv::Mat objectFrame,frame,imageHSV,calcBackImage,dstHist,objectHSV;
bool inited = false;
//保存目标轨迹
std::vector<cv::Point> pt;
//直方图
int histSize = 200;
float histR[] = { 0,255 };
const float *histRange = histR;
int channels[] = { 0,1,2 };
// 终止条件
cv::TermCriteria criteria(cv::TermCriteria::MAX_ITER +
cv::TermCriteria::EPS, 10, 1);
cv::Rect rect;
while(true){
cap >> frame;
if(frame.empty()){
break;
}
cv::imshow("video",frame);
// 选择所要跟踪的对象
if(!inited && cv::waitKey(10) == 's'){
rect = cv::selectROI("video",frame);
objectFrame = frame(rect).clone();
cv::cvtColor(objectFrame,objectHSV,cv::COLOR_BGR2HSV);
//直方图计算
cv::calcHist(&objectHSV, 3, channels, cv::Mat(), dstHist, 1, &histSize, &histRange, true, false);
cv::normalize(dstHist, dstHist, 0, 255, cv::NORM_MINMAX);
inited = true;
}
if(!inited){
continue;
}
// 转换HSV颜色空间
cvtColor(frame, imageHSV, cv::COLOR_BGR2HSV);
//反向投影
cv::calcBackProject(&imageHSV, 3, channels,
dstHist, calcBackImage, &histRange);
// 跟踪
cv::CamShift(calcBackImage, rect, criteria);
objectHSV = imageHSV(rect);
// 更新模板
cv::Mat imageROI = imageHSV(rect);
// 计算直方图
cv::calcHist(&imageROI, 2, channels, cv::Mat(),
dstHist, 1, &histSize, &histRange);
//归一化
cv::normalize(dstHist, dstHist, 0.0, 1.0, cv::NORM_MINMAX);
//目标绘制
cv::rectangle(frame, rect, cv::Scalar(255, 0, 0), 3);
// 绘制目标运动轨迹
pt.push_back(cv::Point(rect.x + rect.width / 2,
rect.y + rect.height / 2));
for (size_t i = 0; i < pt.size() - 1; i++)
{
cv::line(frame, pt[i], pt[i + 1], cv::Scalar(0, 0, 255), 2.5);
}
cv::imshow("video",frame);
if(cv::waitKey(10) == 27){
break;
}
}
return 0;
}
4、cv::computeECC、cv::findTransformECC
1)cv::computeECC：计算两个图像之间的增强的相关系数值。
double cv::computeECC(InputArray templateImage,InputArray inputImage,InputArray inputMask = noArray())
| 参数名称 | 参数描述 |
|---|---|
| templateImage | 单通道模板图像； CV_8U或CV_32F阵列。 |
| inputImage | 扭曲单通道输入图像以提供类似于templateImage的图像，其类型与templateImage相同。 |
| inputMask | 一个可选的掩码，用于指示inputImage的有效值。 |
**2）cv::findTransformECC：**根据ECC准则[Georgios D Evangelidis and Emmanouil Z Psarakis. Parametric image alignment using enhanced correlation coefficient maximization. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 30(10):1858–1865, 2008.]查找两个图像之间的几何变换（扭曲）。
double cv::findTransformECC(InputArray templateImage,InputArray inputImage,InputOutputArray warpMatrix,int motionType,TermCriteria criteria,InputArray inputMask,int gaussFiltSize)
函数根据ECC准则估算最佳变换（warpMatrix），即：
warpMatrix = arg max W ECC ( templateImage ( x , y ) , inputImage ( x ′ , y ′ ) ) \texttt{warpMatrix} = \arg\max_{W} \texttt{ECC}(\texttt{templateImage}(x,y),\texttt{inputImage}(x',y')) warpMatrix=argmaxWECC(templateImage(x,y),inputImage(x′,y′))
其中， [ x ′ y ′ ] = W ⋅ [ x y 1 ] \begin{bmatrix} x' \\ y' \end{bmatrix} = W \cdot \begin{bmatrix} x \\ y \\ 1 \end{bmatrix} [x′y′]=W⋅⎣⎡xy1⎦⎤
该方程具有单应性的齐次坐标。 它返回最终的增强相关系数，即模板图像和最终变形的输入图像之间的相关系数。 当给定的3×3矩阵的motionType = 0、1或2时，将忽略第三行。
与findHomography和estimateRigidTransform不同，函数findTransformECC实现了基于强度相似性的基于区域的对齐方式。 从本质上讲，该函数将更新初始转换，以使图像大致对齐。 如果缺少此信息，则将身份扭曲（统一矩阵）用作初始化。 注意，如果图像经历强的位移/旋转，则需要大致对准图像的初始变换（例如，简单的欧几里德/相似性变换，其允许图像近似地示出相同的图像内容）。 在第二张图像中使用反扭曲来拍摄接近第一张图像的图像，即，将标志WARP_INVERSE_MAP与warpAffine或warpPerspective一起使用。
| 参数名称 | 参数描述 |
|---|---|
| templateImage | 单通道模板图像； CV_8U或CV_32F阵列。 |
| inputImage | 单通道输入图像，应使用最终的warpMatrix进行变形，以提供类似于templateImage的图像，其类型与templateImage相同。 |
| warpMatrix | 浮点2×3或3×3映射矩阵（扭曲）。 |
| motionType | 参数，指定运动类型：MOTION_TRANSLATION设置平移运动模型； warpMatrix为2×3，其中前2×2部分为单位矩阵，其余两个参数为估算值。MOTION_EUCLIDEAN将欧几里德（刚性）变换设置为运动模型； 估计三个参数； warpMatrix为2×3。** MOTION_AFFINE 设置仿射运动模型（DEFAULT）； 估计了六个参数； warpMatrix为2×3。 MOTION_HOMOGRAPHY **将单应性设置为运动模型； 估计八个参数；“ warpMatrix”为3×3。 |
| criteria | 参数，指定ECC算法的终止标准； criteria.epsilon定义了两次迭代之间相关系数增量的阈值（负准则。epsilon使criteria.maxcount成为唯一终止准则）。 默认值显示在上面的声明中。 |
| inputMask | 一个可选的掩码，用于指示inputImage的有效值。 |
| gaussFiltSize | 一个可选值，指示高斯模糊滤波器的大小； （默认值：5） |
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
//读取图像
Mat img_01 = imread("images/tsucuba_right.png");
Mat img_02 = imread("images/tsucuba_left.png");
//定义运动模型
const int warp_mode = MOTION_EUCLIDEAN;
//建立变化矩阵
Mat warp_matrix;
if (warp_mode == MOTION_HOMOGRAPHY)
warp_matrix = Mat::eye(3, 3, CV_32F);
else
warp_matrix = Mat::eye(2, 3, CV_32F);
//最大迭代数
int number_of_iterations = 5000;
//迭代精度
double termination_eps = 1e-10;
//迭代标准
TermCriteria criteria(TermCriteria::COUNT + TermCriteria::EPS,
number_of_iterations, termination_eps);
// 分离通道
vector<cv::Mat> img_01_channels,img_02_channels,results;
split(img_01,img_01_channels);
split(img_02,img_02_channels);
//计算对齐后的图像
Mat result;
for(size_t i = 0;i < 3;i++){
//计算变换矩阵
findTransformECC(img_01_channels[i],img_02_channels[i],warp_matrix,warp_mode,criteria);
Mat temp;
if (warp_mode != MOTION_HOMOGRAPHY){
warpAffine(img_02_channels[i], temp, warp_matrix, img_01.size(),
INTER_LINEAR + WARP_INVERSE_MAP);
}
else{
warpPerspective(img_02_channels[i], temp, warp_matrix, img_01.size(),
INTER_LINEAR + WARP_INVERSE_MAP);
}
results.push_back(temp);
}
merge(results,result);
imshow("image01",img_01);
imshow("image02",img_02);
imshow("result",result);
waitKey();
return 0;
}
5、cv::estimateRigidTransform
计算两个2D点集之间的最佳仿射变换。
Mat cv::estimateRigidTransform(InputArray src,InputArray dst,bool fullAffine)
| 参数名称 | 参数描述 |
|---|---|
| src | 存储在std :: vector或Mat或存储在Mat。 |
| dst | 与A大小或类型相同的第二个输入2D点集，或另一个图像。 |
| fullAffine | 如果为true，则该函数将找到最佳仿射变换，而没有其他限制（6个自由度）。 否则，要选择的转换类别仅限于平移，旋转和均匀缩放（4个自由度）的组合。 |
该函数找到最佳仿射变换[A | b]（一个2 x 3浮点矩阵），该仿射变换在以下情况之间最佳地近似仿射变换：
-
两点套
-
两个光栅图像。 在这种情况下，该函数首先在src映像中找到一些功能，然后 在dst图片中找到相应的特征。 在那之后，问题减少到第一个 案件。
对于点集，问题的表述如下：您需要找到2x2矩阵A和2x1向量b：
[ A ∗ ∣ b ∗ ] = a r g min [ A ∣ b ] ∑ i ∥ dst [ i ] − A src [ i ] T − b ∥ 2 [A^*|b^*] = arg \min _{[A|b]} \sum _i \| \texttt{dst}[i] - A { \texttt{src}[i]}^T - b \| ^2 [A∗∣b∗]=argmin[A∣b]∑i∥dst[i]−Asrc[i]T−b∥2
其中src [i]和dst [i]分别是src和dst中的第i个点，[A | b]可以是任意的（当fullAffine = true时）或具有以下形式
[ a 11 a 12 b 1 − a 12 a 11 b 2 ] \begin{bmatrix} a_{11} & a_{12} & b_1 \\ -a_{12} & a_{11} & b_2 \end{bmatrix} [a11−a12a12a11b1b2]
当fullAffine = false时。
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
//读取图像
Mat img_01 = imread("images/tsucuba_right.png");
Mat img_02 = imread("images/tsucuba_left.png");
vector<Mat> img1_channels,img2_channels,results;
split(img_01,img1_channels);
split(img_02,img2_channels);
Mat dst;
for(size_t i = 0;i < 3;i++){
Mat temp1;
Mat m1 = estimateRigidTransform(img1_channels[i],img2_channels[i],false);
warpAffine(img1_channels[i],temp1,m1,img_01.size());
results.push_back(temp1);
}
merge(results,dst);
imshow("src1",img_01);
imshow("src2",img_02);
imshow("dst",dst);
waitKey();
return 0;
}
当fullAffine=true时，结果如下：
6、cv::meanShift
在反投影图像上找到对象。
int cv::meanShift(InputArray probImage,Rect & window,TermCriteria criteria)
| 参数名称 | 参数描述 |
|---|---|
| probImage | Back projection of the object histogram. For details, see calcBackProject. |
| window | Initial search window. |
| criteria | The stopping condition of the iterative search algorithm. Returns: the number of iterations of CAMSHIFT convergence. This function implements an iterative object search algorithm. It accepts the input back projection and initial position of the object. The center of mass in the window of the rear projection image is calculated, and the center of the search window is moved to the center of mass. Repeat the process until the specified number of iterations criteria.maxCount is completed or until the center offset of the window is less than criteria.epsilon. This algorithm is used inside CamShift. Unlike CamShift, the size or direction of the search window will not change during the search. You can simply pass the output of calcBackProject to this function. However, if you pre-filter the back projection and remove the noise, you can get better results. For example, you can do this in the following way: use findContours to retrieve connected components, discard the contour with a smaller area (ContourArea), and use drawContours to render the remaining contours. |
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
int main()
{
cv::VideoCapture cap("videos/football-in-motion.mp4");
// cv::VideoCapture cap(0);
if(!cap.isOpened()){
cerr << "cannot open video\n";
return EXIT_FAILURE;
}
cv::namedWindow("video");
cv::Mat objectFrame,frame,imageHSV,calcBackImage,dstHist,objectHSV;
bool inited = false;
//保存目标轨迹
std::vector<cv::Point> pt;
//直方图
int histSize = 200;
float histR[] = { 0,255 };
const float *histRange = histR;
int channels[] = { 0,1,2 };
// 终止条件
cv::TermCriteria criteria(cv::TermCriteria::MAX_ITER +
cv::TermCriteria::EPS, 10, 1);
cv::Rect rect;
while(true){
cap >> frame;
if(frame.empty()){
break;
}
cv::imshow("video",frame);
if(!inited && cv::waitKey(10) == 's'){
rect = cv::selectROI("video",frame);
objectFrame = frame(rect).clone();
cv::cvtColor(objectFrame,objectHSV,cv::COLOR_BGR2HSV);
//直方图计算
cv::calcHist(&objectHSV, 3, channels, cv::Mat(), dstHist, 1, &histSize, &histRange, true, false);
cv::normalize(dstHist, dstHist, 0, 255, cv::NORM_MINMAX);
inited = true;
}
if(!inited){
continue;
}
// 转换HSV颜色空间
cvtColor(frame, imageHSV, cv::COLOR_BGR2HSV);
//反向投影
cv::calcBackProject(&imageHSV, 3, channels,
dstHist, calcBackImage, &histRange);
// 跟踪
cv::meanShift(calcBackImage, rect, criteria);
objectHSV = imageHSV(rect);
// 更新模板
cv::Mat imageROI = imageHSV(rect);
// 计算直方图
cv::calcHist(&imageROI, 2, channels, cv::Mat(),
dstHist, 1, &histSize, &histRange);
//归一化
cv::normalize(dstHist, dstHist, 0.0, 1.0, cv::NORM_MINMAX);
//目标绘制
cv::rectangle(frame, rect, cv::Scalar(255, 0, 0), 3);
// 绘制目标运动轨迹
pt.push_back(cv::Point(rect.x + rect.width / 2,
rect.y + rect.height / 2));
for (size_t i = 0; i < pt.size() - 1; i++)
{
cv::line(frame, pt[i], pt[i + 1], cv::Scalar(0, 0, 255), 2.5);
}
cv::imshow("video",frame);
if(cv::waitKey(10) == 27){
break;
}
}
return 0;
}
7、cv::readOpticalFlow、cv::writeOpticalFlow
**1) cv::readOpticalFlow: **The function readOpticalFlow loads the flow field from a file and returns it as a single matrix. As a result, the type of Mat is CV_32FC2- floating point 2 channels. The first channel corresponds to the flow in the horizontal direction (u), and the second channel corresponds to the flow in the vertical direction (v).
Mat cv::readOpticalFlow (const String & path)
2) cv::writeOpticalFlow : This function stores the flow field in a file and returns true if it succeeds, otherwise it returns false. The flow field must be a 2-channel floating point matrix (CV_32FC2). The first channel corresponds to the flow in the horizontal direction (u), and the second channel corresponds to the flow in the vertical direction (v).
bool cv::writeOpticalFlow(const String & path,InputArray flow)