I. Introduction
I wrote an article in the public account before - affine transformation and perspective transformation of image processing . This article is a further in-depth study of perspective transformation.
Two perspective transformation
Perspective transformations are more general than affine transformations. They do not necessarily maintain "parallelism" between lines, but because they are more general and practical, almost all transformations encountered in everyday images are perspective transformations. Ever wondered why two orbits seem to meet at a distance?
This is because the image in your eye is like a perspective transformation, which does not necessarily keep parallel lines parallel. If you stand on it and look at the rails in Figure 1.1, they don't seem to meet at all.
Given a 3x3 perspective transformation matrix, warpPerspective() applies the following transformation:
Note that +1 is not required for the determinant of the upper left 2x2 part of the perspective transformation matrix. Also, because of the division in the transformation shown earlier, multiplying all elements of the perspective ratio transformation matrix by a constant does not make any difference in the transformation represented. Therefore, it is common to make M33 = 1 when computing the perspective transformation matrix. This gives us eight free numbers in M, and thus four pairs of corresponding points are sufficient to recover the perspective transformation between the two images. The OpenCV function findHomography() does this for you. Interestingly, if you specify the flag CV_RANSAC when calling this function, it can even take more than four points and use the RANSAC algorithm to robustly estimate the transformation of all those points. RANSAC insulates the transform estimation process from noisy "wrong" correspondences. The code provided below reads two images (related by perspective transform), asks the user to click on eight pairs of points, robustly estimates the perspective transform using RANSAC, and displays the difference between the original and new perspective transform images to verify the estimate transformation. The link of the entire engineering test code has been uploaded to GitHub, you can follow the WeChat public account: Visual IMAX acquisition.
#include <opencv2/opencv.hpp>
using namespace std; using namespace cv;
void on_mouse(int event, int x, int y, int, void* _p)
{
Point2f* p = (Point2f *)_p;
if (event == CV_EVENT_LBUTTONUP)
{
p->x = x;
p->y = y;
}
}
class perspective_transformer {
private:
Mat im, im_transformed, im_perspective_transformed, im_show, im_transformed_show;
vector<Point2f> points, points_transformed;
Mat M;
Point2f get_click(string, Mat);
public:
perspective_transformer();
void estimate_perspective();
void show_diff();
};
perspective_transformer::perspective_transformer()
{
im = imread("./DataFiles/image.bmp");
im_transformed = imread("./DataFiles/transformed.bmp");
}
Point2f perspective_transformer::get_click(string window_name, Mat im)
{
Point2f p(-1, -1);
setMouseCallback(window_name, on_mouse, (void *)&p);
while (p.x == -1 && p.y == -1)
{
imshow(window_name, im);
waitKey(20);
}
return p;
}
void perspective_transformer::estimate_perspective()
{
namedWindow("Original", 1);
namedWindow("Transformed", 1);
imshow("Original", im);
imshow("Transformed", im_transformed);
cout << "To estimate the Perspective transform between the original and transformed images you will have to click on 8 matching pairs of points" << endl;
im_show = im.clone();
im_transformed_show = im_transformed.clone();
Point2f p;
for (int i = 0; i < 8; i++)
{
cout << "POINT " << i << endl;
cout << "Click on a distinguished point in the ORIGINAL image" << endl;
p = get_click("Original", im_show);
cout << p << endl;
points.push_back(p);
circle(im_show, p, 2, Scalar(0, 0, 255), -1);
imshow("Original", im_show);
cout << "Click on a distinguished point in the TRANSFORMED image" << endl;
p = get_click("Transformed", im_transformed_show);
cout << p << endl;
points_transformed.push_back(p);
circle(im_transformed_show, p, 2, Scalar(0, 0, 255), -1);
imshow("Transformed", im_transformed_show);
}
//Estimate perspective transform
M = findHomography(points, points_transformed, CV_RANSAC, 2);
cout << "Estimated Perspective transform = " << M << endl;
// Apply estimated perspecive trasnform
warpPerspective(im, im_perspective_transformed, M, im.size());
namedWindow("Estimated Perspective transform", 1);
imshow("Estimated Perspective transform", im_perspective_transformed);
imwrite("./DataFiles/im_perspective_transformed.bmp", im_perspective_transformed);
}
void perspective_transformer::show_diff()
{
imshow("Difference", im_transformed - im_perspective_transformed);
}
int main()
{
perspective_transformer a;
a.estimate_perspective();
cout << "Press 'd' to show difference, 'q' to end" << endl;
if (char(waitKey(-1)) == 'd') {
a.show_diff();
cout << "Press 'q' to end" << endl;
if (char(waitKey(-1)) == 'q') return 0;
}
else
return 0;#include <opencv2/opencv.hpp>
using namespace std; using namespace cv;
void on_mouse(int event, int x, int y, int, void* _p)
{
Point2f* p = (Point2f *)_p;
if (event == CV_EVENT_LBUTTONUP)
{
p->x = x;
p->y = y;
}
}
class perspective_transformer {
private:
Mat im, im_transformed, im_perspective_transformed, im_show, im_transformed_show;
vector<Point2f> points, points_transformed;
Mat M;
Point2f get_click(string, Mat);
public:
perspective_transformer();
void estimate_perspective();
void show_diff();
};
perspective_transformer::perspective_transformer()
{
im = imread("./DataFiles/image.bmp");
im_transformed = imread("./DataFiles/transformed.bmp");
}
Point2f perspective_transformer::get_click(string window_name, Mat im)
{
Point2f p(-1, -1);
setMouseCallback(window_name, on_mouse, (void *)&p);
while (p.x == -1 && p.y == -1)
{
imshow(window_name, im);
waitKey(20);
}
return p;
}
void perspective_transformer::estimate_perspective()
{
namedWindow("Original", 1);
namedWindow("Transformed", 1);
imshow("Original", im);
imshow("Transformed", im_transformed);
cout << "To estimate the Perspective transform between the original and transformed images you will have to click on 8 matching pairs of points" << endl;
im_show = im.clone();
im_transformed_show = im_transformed.clone();
Point2f p;
for (int i = 0; i < 8; i++)
{
cout << "POINT " << i << endl;
cout << "Click on a distinguished point in the ORIGINAL image" << endl;
p = get_click("Original", im_show);
cout << p << endl;
points.push_back(p);
circle(im_show, p, 2, Scalar(0, 0, 255), -1);
imshow("Original", im_show);
cout << "Click on a distinguished point in the TRANSFORMED image" << endl;
p = get_click("Transformed", im_transformed_show);
cout << p << endl;
points_transformed.push_back(p);
circle(im_transformed_show, p, 2, Scalar(0, 0, 255), -1);
imshow("Transformed", im_transformed_show);
}
//Estimate perspective transform
M = findHomography(points, points_transformed, CV_RANSAC, 2);
cout << "Estimated Perspective transform = " << M << endl;
// Apply estimated perspecive trasnform
warpPerspective(im, im_perspective_transformed, M, im.size());
namedWindow("Estimated Perspective transform", 1);
imshow("Estimated Perspective transform", im_perspective_transformed);
imwrite("./DataFiles/im_perspective_transformed.bmp", im_perspective_transformed);
}
void perspective_transformer::show_diff()
{
imshow("Difference", im_transformed - im_perspective_transformed);
}
int main()
{
perspective_transformer a;
a.estimate_perspective();
cout << "Press 'd' to show difference, 'q' to end" << endl;
if (char(waitKey(-1)) == 'd') {
a.show_diff();
cout << "Press 'q' to end" << endl;
if (char(waitKey(-1)) == 'q') return 0;
}
else
return 0;
}
}
From the above analysis, for perspective transformation, when multiple points are selected (here needs ≥ 8), use findHomography() to obtain the correspondence matrix M than use getPerspectiveTransform() (this function selects the first four points by default for multiple points) The obtained corresponding effect is better.
Three Practices
For us, we need to perform perspective transformation on the encoded marker points in the following picture, so that it can transform the ellipse perspective into a circle, and the surrounding annulus is also corrected. We need to find at least eight sets of corresponding points in the image below and in the rectified image.
After the perspective transformation of the ellipse into a circle, the relationship between the coordinates of each position on the ellipse and the coordinates of the corresponding points of the circle is shown in Figure 3.2:
According to the above relationship, nine sets of corresponding points can be easily found: the coordinates of the four intersections of the long and short axes and the coordinate axes, the four coordinates of the diagonal, and the coordinates of the center point.
The renderings after perspective transformation of nine groups of corresponding points are shown in Figure 3.3:
Four Summary
This article is a complement to the previous article, but it is also a different approach, which can be used according to needs.
Finally, I would like to thank Zhu Yuke for his help on the corresponding point of the ellipse, and thank Teacher Zhang for his great care and guidance for me to explore new methods and ideas.
