This article briefly describes the typical application scenarios of histograms in image processing. It is a relatively old application technology, but you might as well take it out and learn again, review the past and learn the new, and pave the way for new knowledge.
Histogram Back Projection
Do you still remember the knowledge of image histogram calculation/equalization that you learned before? Back projection is based on this. Students who need to review can read the article here .
First, let’s talk about the concept of histogram backprojection:
- Back projection is to use the distribution/probability value in the histogram model of the source image to perform a projection at the source target image position. The histogram model of the H channel of the HSV color space is usually used for projection.
Note: Note that the HSV color space is used here, which is slightly different from the HSI color model introduced before. Here is a brief summary: In fact, the two are essentially the same theory. Why is one called a color space and one color model? The most essential difference is that V and I are different. V refers to value, V=max(R,G,B). HSV only has the lower half of the cone, and the central axis is grayscale. The position of the pure white point in HSV corresponds to the position of the central axis median gray level in HSI. If the saturation of HSV is for pure white, the saturation of HSI is for pure median gray. The difference between I and V in the two models leads to the difference between H and S. In the HSV color space, the value range of H is [0~180], S and V are [0~255] [PS: The palette in the drawing software that comes with Windows computers is based on the HSI model]
In order to deepen your understanding, let me give you an application example to explain what histogram backprojection is:
(1) For example, the grayscale of a 4x4 image is as follows:
Image= 0 1 2 3
4 5 6
7 8 9 10 11
8 9 14 15(2) Define the grayscale interval of the histogram as [0,3), [4,7), [8,11), [12,16),
then its histogram: Histogram = 4 4 6 2
After normalization is: Histogram = 0.25 0.25 0.375 0.125(3) After the histogram is back-projected, the pixel value at position (0,0) is 0, and the corresponding bin is [0,3), so the value of the reverse histogram at this position is 4.
Image_BackProjection=
4 4 4 4
4 4 4 4
6 6 6 6
6 6 2 2
We can see that the 256 grayscale values of the original image are actually set to a few values. How many values are there depends on how many intervals the grayscale levels from 0 to 255 are divided into (HSV H of space, that is, dividing the interval from 0 to 180) In the feature matrix of back projection, the value of the specific position is the gray histogram value of the interval where the point in the original image corresponds to it. So we can see that the more points an interval has, the brighter it will be in the back-projection matrix.
It can also be seen from this process that we first find the histogram of the original image, and then obtain the feature matrix by back-projecting the histogram. From the histogram to the projection matrix is actually a reverse process, so it is called inversion. projection.
Histogram backprojection-opencv-4.5.5 C project implementation
#include <opencv2/opencv.hpp>
#include <iostream>
using namespace cv;
using namespace std;
void Hist_And_BackProjection(int, void*);
Mat src, hsv, hue;
string titleStr = "test backProjection ";
int hist_bins = 12;
int main()
{
//读取测试图片
src = imread("F:\\other\\learncv\\ic_launcher.png");
namedWindow(titleStr + "src", WINDOW_KEEPRATIO);
imshow(titleStr + "src", src);
//rgb转hsv
cvtColor(src, hsv, COLOR_BGR2HSV);
hue.create(hsv.size(), hsv.depth());
//从灰度图图像中拷贝0通道到 直方图输入图像的0通道
int nchannels[] = { 0, 0 };
mixChannels(&hsv, 1, &hue, 1, nchannels, 1);
//动态调整直方图的 bins ，并做反向投影
createTrackbar("Histogram Bins", titleStr + "src", &hist_bins, 180, Hist_And_BackProjection);
Hist_And_BackProjection(0, 0);
waitKey(0);
return 0;
}
void Hist_And_BackProjection(int, void*)
{
float range[] = { 0,180 };
const float* histRanges = { range };
Mat hist;
// 直方图计算及归一化处理
calcHist(&hue, 1, 0, Mat(), hist, 1, &hist_bins, &histRanges, true, false);
normalize(hist, hist, 0, 255, NORM_MINMAX, -1, Mat());
//画直方图的分布图
int hist_h = 400;
int hist_w = 400;
int bin_w = hist_w / hist_bins;
Mat histImage(hist_w, hist_h, CV_8UC3, Scalar(255, 255, 255));
for (int i = 1; i <= hist_bins; i++)
{
rectangle(histImage,
Point((i - 1) * bin_w, (hist_h - cvRound(hist.at<float>(i - 1) * (400 / 255)))),
Point(i * bin_w, hist_h),
Scalar(85, 85, 85),
-1);
}
imshow(titleStr + "Histogram", histImage);
//直方图反向投影
Mat backPrjImage;
calcBackProject(&hue, 1, 0, hist, backPrjImage, &histRanges, 1, true);
namedWindow(titleStr, WINDOW_FREERATIO);
imshow(titleStr, backPrjImage);
}
As shown in the example, when the number of histogram intervals is controlled to 5, back-projection can basically extract the features of the input image. The input image tested here is relatively simple. The feature map produced by back-projection has no noise. If there are some additional The noise can be masked by using the previously learned expansion/erosion operations.
Template Match
Next, we briefly introduce template matching in opencv. Let’s give a text introduction first:
- Template matching is to find small areas that match a given sub-image in the entire area of the target image.
- Template matching first requires a template image T (the focus is on this template T, it is very important!!!)
- On the target image to be detected, the similarity between the template image and the sub-image is calculated from left to top and from top to bottom. The greater the similarity, the greater the possibility that the contents of the two are equivalent.
First introduce the opencv related API——matchTemplate
cv::matchTemplate(
InputArray image,// 源图像，必须是8-bit或者32-bit浮点数图像
InputArray templ,// 模板图像，类型与输入图像一致
OutputArray result,// 输出结果，必须是单通道32位浮点数,假设源图像WxH,模板图像wxh,
// 则结果必须为W-w+1, H-h+1的大小。
int method, //匹配相似性的方法标识
InputArray mask=noArray()
)
The third output parameter, the size of OutputMat is (W-w+1, H-h+1) because template matching is on the target image to be detected, from left to top, and from top to bottom, the template image and sub-image are calculated. , and then output the similarity degree to the corresponding position.
Then let’s focus on learning the similarity methods of template matching:
The core idea of template matching is to find a suitable index from the template image to measure the similarity between template T and search area I. For example, whether the two triangles learned in primary school are similar, so the similarity has evolved into the above sets of mathematical relationships.
Among them (x', y') represents the position of template T, (x, y) represents the starting position of the area of target image I, then I(x+x',y+y') actually means It is based on the position of the target image with (x, y) as the starting point and offset (x', y'). R(x,y) is the similarity between the region image with (x,y) as the starting point and the size of (x',y') and the template.
- TM_SQDIFF/TM_SQDIFF_NORMED: sum of squared differences/normalized sum of squared differences
The similarity between the two is measured by calculating the sum of the squared pixel differences of all corresponding pixels in the template and the search area. The smaller the calculation result, the more similar the two are. When the calculation result is 0, it means that the two are exactly the same.
- TM_CCORR / TM_CCORR_NORMED: Correlation coefficient / normalized correlation coefficient
The correlation coefficient between T and I is used as a measurement index. The larger the correlation coefficient, the more similar the two are.
- TM_CCOEFF / TM_CCOEFF_NORMED: Mean-removed correlation coefficient/Mean-removed normalized correlation coefficient
The difference between the mean-removing correlation coefficient and the correlation coefficient is that when calculating the mean-removing correlation coefficient, both T and I subtract their respective means, while the correlation coefficient does not subtract the mean. This is done to eliminate the impact of different brightnesses of T and I on the calculation results. For example, if T and I have very similar structures but very different brightnesses, then they subtract their respective averages and use the brightness offset of each pixel to calculate the result. Calculated rather than using the brightness itself, thus eliminating the effects of different brightness.
Careful students may find that there are obvious shortcomings in starting template matching, or that there are many prerequisite restrictions when using template matching:
1. The size of the template is fixed. The area searched in the target image is the same size as the template. It cannot intelligently identify similar feature areas of different sizes. In target image areas of different sizes, even the same similarity algorithm will Lead to completely different results.
2. If the target image has rotation, scaling, and local non-rigid deformation, the effect of template matching is not very good. In other words, it is necessary to perform operations such as rotation, scaling, and local non-rigid deformation on the template image before template matching can be used.