Guide image filter is a kind of adaptive weight filter, which can maintain the boundary while smoothing the image. For the specific formula derivation, please refer to the original document "Guided Image Filtering" and the matlab source code: http://kaiminghe.com/ eccv10/index.html . Here only talk about the principle of adaptive weighting, C++ implementation of gray-scale image and color image guide map filtering, and verification results.
As a linear filter, the adaptive weight principle guide graph filter can be simply defined as the following form:
Where I is a guided image, P is the input image to be filtered, Q is the filtered output image, and W is the weight value determined according to the guided image I. The weight value W can be expressed by the following formula (the original document has a detailed derivation):
μk is the mean value of the pixels in the window, Ii and Ij refer to the values of two adjacent pixels, σk represents the variance of the pixels in the window, and ε is a Penalty value. The adaptive weight can be obtained according to the above analysis: when Ii and Ij are on both sides of the boundary, (Ii-μk) and (Ij-μk) have different signs, otherwise, they have the same sign. The weight value of the opposite sign will be much smaller than the weight value of the same sign, so the pixels in the flat area will be weighted more, and the smoothing effect will be more obvious, while the pixels on both sides of the boundary will be added. The smaller the weight, the weaker the smoothing effect, which can maintain the boundary effect.
The penalty value ε also has a great influence on the filtering effect. When the ε value is very small, the filtering is as described above; when the ε value is large, the weight calculation formula will approximate an average filter, and the smoothing effect will be more obvious.
Similarly, we can also look at the adaptive weight principle of the guide graph filter according to the linear filter formula. The local linear filter model formula is as follows:
I refers to the guiding image, Q is the output image, and the two coefficients of ak and bk are jointly determined based on the guiding image I and the input image P. Calculating the gradient on both sides of the above equation, we can get ▽q=a*▽I, that is, the gradient information of the output image is completely determined by the gradient information of the guiding image. When there is a boundary in the guiding image, the corresponding position in the output image will also have a boundary. The values of a and b will determine the weight of the gradient information and smoothing information.
By observing the formulas of a and b, the numerator of a is the covariance of I and P, and the denominator is the variance of I plus the cutoff value ε; the value of b is the mean of P minus the mean of a times I. It can be seen that when the value of a is very small, b is approximately equal to the mean value pk of the pixels in the window, which is similar to mean filtering; when the value of a is large, the output mainly depends on the size of a*▽I, and the gradient information can be retained .
- C++ Realizes the Guide Image Filtering of Gray Image and Color Image
According to the pseudo-code provided in the original document, it is not difficult to use C++ to implement the guide image filtering algorithm. The pseudo code is as follows:
There are two cases here, 1. The guide picture is a single-channel grayscale picture; 2. The guide picture is a three-channel color picture.
- The guide image is a single channel
to calculate the average image of I and P, and the average image of I² and I*P; then the variance image of I and the covariance image of I*P are calculated; the values of a and b are calculated using formulas ; Calculate the average value of a and b in the window; then calculate the output image according to the formula.
There are two situations here:
①The input picture is a single channel: just follow the above steps to calculate.
②The input picture is three channels: first separate the three channels, perform the above filtering operation on each channel, and then merge the channels. - The guide picture is three-channel.
At this point, the above model is slightly modified as follows: The
difference is that when calculating a, the original variance σ is replaced by the covariance 3x3 matrix ∑k, which is expressed as follows:
U is a 3x3 identity matrix, find it out The a will no longer be a value, but a 1*3 vector, and then find b. a is a 1*3 vector, μk is a 3*1 vector, b is a constant after multiplication, and the value of the constant b can be obtained from this.There are two situations here:
①The input picture is a single channel: just follow the above steps to calculate.
②The input picture is three channels: first separate the three channels, perform the above filtering operation on each channel, and then merge the channels.
Note: The guide map is a color map than the guide map is a grayscale map, and the boundary protection is more obvious. See the original text.
-
Effectiveness verification
The averaging part of the code can be implemented by the boxFilter() function in OpenCV, or the blur() function. In short, it is an average filter. The reason why it has nothing to do with the size of the window is that the average filter realized by using histogram can greatly reduce the calculation time. The code implemented by VS2015+OpenCV3.4.0 is placed on my code cloud code: https://gitee.com/rxdj/guidedFilter.git .
The main input parameters are the guide image I, the input image P, the window radius r, the cutoff value ε, and the output parameter is the filtered image Q. The guide image I and the input image P can be the same or different. For example, in stereo matching, the original reference image is often used as the guide image, and the cost space image is filtered by the guide image to realize cost aggregation. In this way, the matching cost of the boundary area of the original image can be preserved as much as possible, and the matching cost of the smooth flat area can be smoothed.
(Note: The code in the literature is written by matlab. When reading the image in matlab, the image will be automatically normalized to 0-1, so the cutoff value ε is also much smaller, such as 0.1, 0.01, etc. And this article After reading the image, the normalization operation is not performed, so the setting of the cutoff value ε will be different. If you need to normalize, you can read the image by yourself and divide by 255).
- Single-channel grayscale original image
Change cutoff value ε
r=10, ε=0 r=10, ε=100
r=10, ε=2000 r=10, ε=8000 Change window radius r r=0, ε=500 r=5, ε=500
r = 10, ε = 500 r = 20, ε = 500
- Three-channel color image
Original image
Change cutoff value ε
r=10, ε=0 r=10, ε=100
r=10, ε=2000 r=10, ε=8000
Change window radius r
r=0, ε=500 r =5, ε=500
r=10, ε=500 r=20, ε=500
-
Fast guided filter algorithm The
fast guided filter algorithm can be found in the document "fast guided filter", which is also available on the above homepage. The fast place is mainly to use the image pyramid idea. Proceed as follows:
- Perform 1/s down-sampling on the guide image I and the input image P to obtain I', P';
- Use I'and P'to calculate the coefficients a and b, and calculate the output image Q';
- Up-sampling Q'by s times to obtain the final output image Q.
Since the calculation part is a down-sampled image, the amount of calculation will be greatly reduced without introducing obvious distortion, so it becomes a fast guide image filter. For the code, see the fastGuidedFilter branch of the code cloud code link above.
Undertake programming in Matlab, Python and C++, machine learning, computer vision theory implementation and guidance, both undergraduate and master's degree, salted fish trading, professional answers please go to know, please contact QQ number 757160542 for details, if you are the one.