The stereoscopic binocular vision correction

In this article, the stereoscopic image is corrected based on the relative positions of the two cameras on the article obtained

1. Introduction
   When the two image lines aligned completely planar, stereo disparity calculation is simple. However, there are two cameras is almost impossible to accurately aligned coplanar line and the imaging plane, a perfect alignment structure is hardly present in the real three-dimensional system. Accordingly, the corrected three-dimensional object image plane is re-projected two cameras, so that they fall on exactly the same plane, and the image line structure completely aligned parallel to the forward.

   To ensure that images of the two cameras are aligned after the correction, so that the subsequent stereo matching more reliable, more computationally feasible. Because the other image searching for matching points in a row above the image only can the reliability and efficiency of the algorithm. So that each image plane falls on a common plane and the imaging results are horizontally aligned poles are at infinity. I.e. the projection center of an image with another image on the image plane is parallel. However, since the front plane parallel to a selectable number is limited, thus requiring more constraints, including the view minimizes distortion and maximize overlap.

   Results of the two images are aligned with a plane after eight, four each of the left and right cameras (camera matrix after distortion vector rotation matrix correction, the uncorrected camera matrix)

2. Both algorithms calculate the correction term

   1. Hartley algorithm (used to generate a substantially non-calibration matrix stereoscopic) non-calibrated perspective correction
      1. Hartley algorithm is to find the object is mapped to the poles at infinity homography matrix, while the error calculated between the two stereoscopic images is minimized, to achieve this algorithm to be achieved by matching corresponding points between two images, by which method, can bypass the calculated camera parameters of the two cameras, such as the parameter information implicit in the matching point
      2. Hartley disadvantages Algorithm
         1) advantages: through the observation point in the scene easily mitochondrial calibration
         2) disadvantages: unknown scene image ratio
         
         stereoscopic reconstruction having non-uniqueness, if the size of the object is unknown, then the object may appear to be the same of different sizes , depending on the distance between them and the camera, if the camera parameters are unknown, it may appear different projection same - e.g., at the main focal point and different circumstances
      3. Algorithmic descriptions (assuming known fundamental matrix)
         1. Using the fundamental matrix, the relationship
            
            is calculated around two poles
         2. The first single should first seek matrix Hr, he will be mapped to the right pole at infinity (1,0,0) two-dimensional homogeneous at the point T. Since a single matrix should have 7 constraint, three of which are infinity maps, and the remaining four degrees of freedom to choose matrix Hr, most of which four degrees of freedom would lead to confusion, since most of the selected Hr will result in a highly distorted image. To find a good Hr of, the possibility that we choose twisted in the image minimum, allowing only the rigid rotation and translation. For this a reasonable choice of the origin image is further assumed that the pole er = (k, 0,1) T falls on the x-axis, the following matrix and will achieve such a mapping point to infinity.
            
         3. Selecting a point of interest on an image (here to choose the origin), the calculation point to the origin of the image translation matrix T and the pole point (er) T = (k, 0,1) of the rotation matrix R, the desired single should matrix is ​​Hr = GRT.
         4. Then there is the single search should match the matrix Hl, he left pole will be sent to infinity, and to ensure that the two images are aligned rows. Landline can easily take you to infinity in step 2 of the three constraints. In order to align the line, relying on the alignment of the line is based on the distance and the minimum, according to all the two images match point. That is, the search on Hl, such that the total disparity minimized around the matching point, i.e.
            
            the minimum, then the two homography is defined a perspective correction.
   2. Bouguet algorithm (using two camera rotation and translation parameters) Calibration perspective correction
      1. Algorithm Overview
         a rotation matrix and translation matrix (R, T) between the stereoscopic image set, the purpose of the algorithm is to re-Bouguet a projected two times each image is minimized, while maximizing the area of observation. In order to minimize distortion of the projected image reconstruction, the image plane is rotated to the right camera image plane of the left camera rotation matrix R is separated into two parts, said two synthetic r1 about a rotation matrix of the camera and r2. Each camera is rotated half, so the camera can be rotated so that the coplanar line but not aligned. To calculate the left camera transformation pole and electrode lines to infinity horizontally aligned Rrect. Create a rotation matrix starting from pole el direction.
      2. Algorithm Description
         get the main point (cx, cy) as the origin of the left image, the pole direction is the direction of the translation vector between the center of projection of two cameras:
         
         a next orthogonal vectors e1 and e2 must, therefore is to choose the best option a direction perpendicular to the principal ray (typically along the image plane). This product can be obtained by calculating the principal ray directions e1 and fork, he then normalized to the unit vector:
         
         This vector can be obtained a third e3, It can e2 e1 and cross product obtained:
         
         In this case, converting the poles of the left camera to infinity following matrix:
         
         this matrix is left image rotates about the projection, so that the source line becomes horizontal, and the pole at infinity. Aligning two cameras line by the following equation implemented:
         
         calculation about the camera and the corrected matrix Mrect_l Mrect_r, but returned with the projection matrix Pl and Pr:
         
         and
         
         wherein α_l and α_r pixel distortion ratio, almost equal in modern computers 0. The projection matrix transforms 3D point homogeneous coordinates into two-dimensional point in homogeneous coordinates:
         
         the screen coordinates (x / w, y / w ). If a given parameter matrix camera and screen coordinates, two-dimensional point can also be re-projected onto the three-dimensional reprojection matrix as follows:
         
         the above formula, except cx 'All parameters are outside the image from the left, cx' main point x coordinate on the right image. If the principal rays intersect at infinity, then cx = cx ', and the entry for Ranger 0. Given a two-dimensional point and secondly its associated disparity d, this point can be projected into three dimensions:
         
         so that three-dimensional coordinates (X / W, Y / W , Z / W), where d is unknown, the following section will be introduced

3. From the above three articles you can obtain the following effects:
   

References: Adrian Kaehler, Gary Bradski, Learning Opencv3 [M], Tsinghua University Press, 2018.7

---

The following is the author of the micro-channel public number, welcome attention, will continue to update c ++, python, tensorflow, machine learning, deep learning, computer vision and other articles, public No. contains 300+ this pdf e-books, certainly you need one, you can receive public attention No. Oh.

If you are interested in JAVA aspects, can focus on the following JAVAERS public number, accompany you learn together, grow together, sharing poetry and distant JAVA road together. In which public numbers are JAVA friend of this world, the public will have number-technical articles every day, by dry surface, but also the advanced architecture of e-books, such as Spring combat, SpringBoot practical, high-performance MySQL, in-depth understanding of JVM, RabbitMQ combat , Redis design and implementation of a number of high-quality books and so on, you can receive public attention No. Oh.