Implementation of SGBM stereo matching algorithm in opencv based on python

Article directory

Preface
1. What is the difference between SGBM and SGM?
2. Implementation of SGBM’s python-opencv
SGBM parameter selection
Reference article

Preface

The core of SGBM is the SGM algorithm, which has been open source since OpenCV2.4.6. It is very convenient and widely used.

1. What is the difference between SGBM and SGM?

Refer to the article by the boss: Stereo Matching Algorithm Reasoning Notes - SGBM Algorithm (1)
[Algorithm] A brief analysis of the OpenCV-SGBM algorithm and source code. The
original SGM algorithm process is as follows:
Insert image description here

The algorithm flow of SGBM is as follows:
Insert image description here
After comparison, it can be found that the difference between SGBM and SGM lies in the calculation of matching cost: SGBM uses SAD-BT, while SGM uses MI.

1. Preprocessing

SGBM uses the horizontal Sobel operator for image preprocessing. The formula is to
Sobel(x,y)=2[P(x+1,y)-P(x-1,y)]+ P(x+1,y-1)-P(x-1,y-1)+ P(x+1,y+1)-P(x-1,y+1)
map the result of the x-sobel operator [0, preFilterCap*2], and preFilterCap is a constant parameter.
Insert image description here

2. Cost calculation

The cost consists of two parts:
1. The gradient information of the image obtained through preprocessing
2. The gradient cost obtained through the sampling-based method.
3. The SAD cost of the original image is obtained by the sampling-based method. Because the BT cost is a one-dimensional matching, it is usually necessary to combine the SAD idea and use the neighborhood summation method to calculate the SAD-BT. The calculated cost is the local Block cost, the matching cost of each pixel will include information about the surrounding local area.
Insert image description here

3. Dynamic programming

The dynamic programming algorithm itself has a tailing effect, and erroneous matching is easy to occur at disparity mutations. Using dynamic programming to accumulate one-dimensional energy will propagate erroneous disparity information to subsequent paths. The semi-global algorithm uses information in multiple directions to try to eliminate the interference of wrong information, which can significantly reduce the tailing effect produced by the dynamic programming algorithm.
The semi-global algorithm attempts to establish a global Markov energy equation through the constraints of one-dimensional paths in multiple directions on the image. The final matching cost of each pixel is the superposition of all path information, and the disparity selection of each pixel is just Simply decided by WTA (Winner Takes All).
Insert image description here
Among them, dynamic programming is very important. Two parameters P1 and P2 are set like this:
P1 =8 cn sgbm.SADWindowSize sgbm.SADWindowSize;
P2 = 32 cn sgbm.SADWindowSize sgbm.SADWindowSize;
cn is the number of channels of the image, SADWindowSize is the SAD window Size, the value is an odd number.

4. Post-processing

The post-processing of opencvSGBM includes the following steps:
uniqueness detection,
sub-pixel interpolation,
left and right consistency detection,
and detection of connected regions.

2. Implementation of SGBM’s python-opencv

The results of using python-opencv are as follows. It has been written and you can download it directly.
Implementing SGBM based on python-opencv

Insert image description here
In addition, WLS filtering has been added to increase the graphics connection effect.

Directly modify the image position to run, as follows:

Use wls filtering: set use_wls = True.
For ease of use, SGBM_slider.py is added. Just replace the image file directly, so that everyone can observe the parameter effect. Just download and run it. The running results are as follows:
Implementing SGBM based on python-opencv, with sliding window
Insert image description here