Stereo matching algorithm is
a classification algorithm stereo matching
stereo matching, matching problem is to find two sets of data can be seen as related to the degree of process. Classified by a variety of stereo matching algorithm.
① The constraint algorithm runtime Range: divided into local (local) and global matching algorithm (Free Join) matching algorithm.
② disparity map generated based on: Dense can be divided (the Dense) matching and sparse (Sparse) match. Dense matching: disparity map is generated based on the disparity value is determined for all pixels are generated, called dense matching. Sparse Match: Select only the key pixel [typically corner or edge point] A method for calculating a disparity value is referred to as sparse matching, the algorithm faster, but subsequent need to calculate the missing disparity values of pixels by interpolation algorithm there are very limited on, so the application scenario.
Because of his recent research has focused on the local and global matching algorithm matching algorithm, the following will also be described here for the next.
1, the global matching algorithm
stereo matching algorithm is global (semi-global) is mainly used a method of global optimization theory of estimating a disparity, the establishment of a global energy function, which includes a data item and smoothness term, the optimal obtained by minimizing the global energy function disparity values. Wherein the graph cut (Graph cuts, GC), a belief propagation (Belief Propagation, BP), dynamic programming (Dynamic Programming, DP), particle swarm optimization (Particle Swarm Optimization, PSO), genetic algorithm (Genetic Algorithm, GA) and the like optimized energy minimization algorithms are commonly used methods for solving systems.
Matching algorithms generally defined as global energy function:
Wherein items described matching degree, smoothing term reflects the constraints defined scenario, C is a matching cost, P is the pixel p and the two different functions q parallax, generally called penalty term (Penalty), when p and q Point point parallax not equal, P> 0, and, the greater the greater the difference between the two P-values. When the values of p and q parallax, P = 0. Because the global matching algorithm optimization problems in mathematics is a function of energy, so you can find the optimal solution. This problem is demonstrated in two-dimensional space is difficult NP. Thus, even if the global algorithm has the advantage of high accuracy, the calculation speed is very slow indeed, in the case of high real-time requirement is not suitable for use global stereo matching algorithm.
Taking into account the complexity of the energy optimization problem in one-dimensional space is a polynomial, so some research trying to do something to reduce the complexity approximation algorithm. For example, semi-global algorithm (SGM) on the use of this feature will simplify the problem of the two-dimensional 8 to 16 one-dimensional problems, to achieve a better approximation. After calculating the cost accumulated in various directions, each direction is obtained by adding the total cost of the price, thus simulated two-dimensional optimization problem. SGM is gradually replacing stereo matching key technology LIDAR generated disparity map, but also the most widely used commercial software stereo matching algorithm.
2, the local matching algorithm
partial perspective window-based matching algorithm or method known as a method based on the area. By weight of a suitable size, shape and weight of window algorithm for each pixel in the reference image, the disparity value then the weighted average of the window. Ideal support should completely cover the window weak texture area, and a continuous depth within the window. Similar global stereo matching algorithm, the best parallax is calculated by the method of optimizing a cost function. However, in local energy function stereo matching algorithm, the only constraint data items based on a local area, without smoothing term. Local matching algorithm using only one bit of information grayscale, color, gradient neighborhood is calculated matching cost, low computational complexity, most real-time stereo matching algorithms are classified as a local stereo matching, but a low partial perspective matching algorithm a texture area, the texture area is repeated, and the disparity discontinuity occlusion region matching is not ideal.
Based on the local region stereo algorithms it is the first study, mature algorithm, simple, fast calculation can be performed in real-time image processing, high matching accuracy. Rationale: selecting a reference image a point, select the point neighborhood of a support window, and then based on some similarity criterion, to find and support the window most similar to the child window to be matched image, the sub-window corresponding to It is the corresponding pixel matching point.
Fixed window polymerization consideration window of fixed size and shape as the cost of the polymerization cell, typically a rectangle, and assuming the other pixels within the window and the support to be matched points having the same disparity. Fixed window method accuracy is not high, but easy to implement, short time-consuming, in some real-time demanding situations has been applied.
Algorithm is still fixed window size and shape based on the cost of the polymerization bilateral filtering, but the elements in the right window of different weights, calculated from the weight difference between the target image and the gray pixel within the window from the center of the window. Based on the cost of the polymerization bilateral filtering algorithms high accuracy, but the computational complexity, poor real-time, the increase of the window size algorithm performance index.
The main idea of the polymerization segmentation algorithm is based on the cost of: dividing in advance the reference image as the left, with respect to the window center pixel is within the support window of the same segment, the corresponding weight values becomes 1, with a much less than an otherwise positive number. However, image segmentation is a very time-consuming operation, the same can not be used in high real-time requirements of the occasion.
Support aggregation algorithm window shape (Cross-based Cost Aggregation, CBCA) based on the cost of the cross is not determined, will vary depending on the gray value of the neighborhood matching point, specific implementation will be introduced later in the update. The method may use the GPU parallel computing has better real-time, the cost of the various algorithms are widely used in the polymerization step.
Note Well: global local matching algorithm is computationally intensive matching algorithm is smaller, at best algorithm support enables real-time processing.
Second, the evaluation platform stereo matching algorithm
①Middlebury test platform: providing a dedicated stereo matching algorithm for evaluating test images (Stereo Pair), comprising a pair of test images Tsukuba, Venus test images, Teddy and Cones of the test image on the test image their resolutions are 384 288,434 383 * 375 and 450, and also gave a true disparity map images of these tests.
②KITTI algorithm evaluation platform: the object aimed at evaluating the performance (motor vehicles, non-motor vehicles, pedestrians, etc.) detection, object tracking computer vision technology in the car environment for automotive driver assistance applications do technical evaluation and technical reserves. KITTI real image data contained urban, rural and highway scene acquisition.
Third, the stereo matching step
1) matching cost calculation (Cost Computation):
Matching cost calculation, i.e. the calculation of each pixel on the reference image IR§, in all likelihood to match the disparity cost value corresponding to the point on the target image IT (pd), so that the calculated cost values may be stored in a H W D ( MAX) three-dimensional array, commonly referred to as the three-dimensional array disparity map (disparity space Image, DSI). When the stereo matching based matching cost, design anti-noise, insensitive to changes in illumination matching cost, can improve the accuracy of stereo matching. Therefore, the cost of matching algorithm design in the global and local algorithms are the focus of research.
2) the cost of the polymerization (Cost Aggregation)
Typically the cost of the overall algorithm does not require a polymerization, and the need for a local matching cost algorithm in the window obtained by polymerizing support point on the reference image accumulated in the p d at the expense of the CA disparity (p, d by summing, averaging or other means ), the cost of a process called polymerization. By matching the cost of the polymerization, it can reduce the influence of outliers, improved signal to noise ratio (SNR, Signal Noise Ratio) and to improve matching accuracy. Consideration aggregation strategy often the core of local matching algorithm quality policies directly related to the quality of the final disparity map (Disparity maps) a.
3) parallax calculation (Disparity Computation):
Thought partial perspective matching algorithm, matching cost After the polymerization in the support window, the parallax acquisition process is relatively simple. Usually "winner takes all" strategy (WTA, Winner Take All), i.e., selects an optimum point corresponding to the accumulated cost of a matching point in the search range disparity, the disparity corresponding to a disparity is also desired. I.e., point P is the parallax.
4) treatment (Post Process)
After general, left and right picture shows respectively a reference image, the above three steps is about two parallax images can be obtained. The resulting disparity map but there are some problems, such as parallax Occlusion inaccurate, the presence of noise points, mismatching points like, it is also necessary to optimize the disparity map used for the processing step further corrected disparity map executed. Commonly used methods are interpolated (Interpolation), enhanced subpixel (Subpixel Enhancement), fine (Refinement), the filtered image (Image Filtering) and other operations.
IV Study similarity measure (matching cost) stereo matching
similarity measure function is used to measure the similarity matching primitives matches the reference image and the target image primitives, i.e., the reference image and the determination target image to the corresponding points the possibility of matching points, also called matching cost, denoted as C (p, d).
① The three most common price for the match and the absolute difference (Sum of Absolute Differences, SAD), and cut absolute difference (Sum of Truncated Absolute Differences, STAD), sum of squared differences (Sum of squared Differences, SSD).
Note: wherein p represents N§ support window, when N§ reduced to contain only the point p, i.e., pixel by pixel matching cost calculation. These three matching cost is very sensitive to changes in the intensity of exposure, but the calculation is relatively simple, very fast, is still relatively widespread use in the industry. Matching cost calculating the time complexity is O (W H N§), integral image may be used (Integral Image) or filter block (Box Filtering) process down to make the time complexity O (w * h).
②Z.Gu first proposed to introduce into the stereo matching Rank transformation function, compared to other similar measure, Rank transformation brightness difference is less sensitive to image noise and a stereoscopic image, and calculates the fast, real-time. Rank transformation function formula is as follows:
③census full account of the cost of local image characteristics, rather than directly using grayscale values make the difference, with anti distortion of light and shadow effects, high efficiency, high stability, ideological core idea is similar to the LBP algorithm is a very effective cost calculation method. Census matching cost is calculated as follows:
Wherein HAMMING (a, b) for calculating a sequence of binary Hamming distances a and b (Hamming Distance), XOR operation can be calculated. seq [I§] The function generates a sequence of binary pixel values of pixel value p and p support window.
④ there is additional conventional matching cost normalized cross correlation NCC (Normalized Cross Correlation), BT cost function (S.Birchfield & C.Tomasi) and the like.
Normalized cross-correlation NCC:
Note: Composite costs proved to have good results, is being increasingly used by the algorithm. Census and the following equation X.Mei SAD combined.
Different different matching cost processing capability, and the cost of the composite can be such that they complement each other, to improve the stability of the algorithm. However, how to determine the cost of each of the weights is still a question to be considered. Some prior algorithm parameters, some algorithms are determined during operation, both advantages and disadvantages.
