Digital Image Processing (9) Bilateral Filtering

---

1. What is bilateral filtering?

  Bilateral filter (Bilateral filter) is a nonlinear filtering method, which is a compromise filtering method that combines the spatial proximity of images and the similarity of pixel values.
  Meaning of linear filtering: $g(i,j)=T[f(i,j)]$ , where$T$ is a linear operator. That is, the pixel values ​​of the input image are linearly transformed to obtain the pixel values ​​of the output image, such as mean filtering, Gaussian filtering, etc. The original data and filtering results of the linear filter are arithmetic operations, which are realized by operations such as addition, subtraction, multiplication, and division. Since the linear filter is an arithmetic operation, it has a fixed template.
  The meaning of nonlinear filtering: the operator of nonlinear filtering includes nonlinear operations such as taking absolute value and zeroing. For example: maximum filter, median filter, etc. The original data of the nonlinear filter and the filtering result are a logical relationship, which is realized by comparing the gray value in a certain neighborhood, and there is no fixed template.

2. Why use bilateral filtering?

  Filtering methods such as median filter, Gaussian filter, and Wiener filter are easy to blur the edge details of the picture, and the protection effect on high-frequency details is not obvious. In comparison, the bilateral filter can well protect the edges while reducing noise.
  The convolution kernel of bilateral filtering is nonlinear, so the computational complexity is high. The convolution kernels in different positions are different, so the FFT cannot be pre-calculated or executed, and the calculation is time-consuming.

3. The principle of bilateral filtering

1. Spatial Domain Kernel

  Spatial domain kernel template weights $w_d$如下。
$$w_d(i,j,k,l)=exp(-\frac{(i-k{)}^2+(j-l{)}^2}{2p_d^2})$$
$w_d$Represents a point $q(k,l)$ and the center point$p(i,j)$ Euclidean distance. $p_d$is the standard deviation of the Gaussian function. There is no difference between the filter template generated using this formula and the template used by the Gaussian filter.

2. Range kernel

  Template weight $w_r$如下。
$$w_r(i,j,k,l)=exp(-\frac{\Vert f(i,j)-f(k,l){\Vert }^2}{2p_r^2})$$
, where$f(i,j)$ means that the image is at point$(i,j)$ ; the pixel value at$f(k,l)$ is the pixel value of the central coordinate point of the template window. $p_r$is the Gaussian function and the standard deviation.

3. Template multiplication

  The template weight of the bilateral filter is obtained by multiplying the above two templates.
$$w(i,j,k,l)=w_d(i,j,k,l)\times w_r(i,j,k,l)=exp(-\frac{(i-k{)}^2+(j-l{)}^2}{2p_d^2}-\frac{\Vert f(i,j)-f(k,l){\Vert }^2}{2p_r^2})$$

The pixel values ​​of the filtered image are: $$g(i,j)=\frac{{\sum }_{kl}f(k,l)w(i,j,k,l)}{{\sum }_{kl}w(i,j,k,l)}$$

四、$w_d$Sum $w_r$and σ \sigmaUnderstanding of$\sigma$

  Spatial domain weight $w_d$Measures $p,q$ The Euclidean distance between two points, the farther the distance$w_d$smaller. This is the same as the Gaussian filter. The closer the pixel position of the Gaussian filter is to the center point, the greater the weight.
  Range weight $w_r$Measures $p,q$ The similarity of pixel values ​​between two points, the more similar$w_r$bigger.

Visually analyze the effect of bilateral filtering from the flat area and edge area of ​​the image.

• In a flat area, the difference between adjacent pixels and the central pixel is small, corresponding to $w_r$Close to 1, at this time the airspace weight $w_d$It plays the main role, which is equivalent to directly using the Gaussian filter to filter the image.
• There will be a large change in the gray level of the edge area, and the wr w_r of the area similar to the gray value of the convolved pixel$w_r$The larger is close to 1, the wr w_r of the area with a larger difference from the gray value of the convolved pixel$w_r$smaller tends to 0. In this way, the convolved pixels are less affected, thus maintaining the detail information.

  $p_d$Selection: Same as Gaussian filtering, $p_d$The larger the image is, the smoother it is; $p_d$The smaller the value, the greater the weight of the central point, the smaller the weight of the surrounding points, and the smaller the filtering effect on the image. When it tends to 0, the output is equivalent to the original image.
  $p_r$Choose: $p_r$The larger the value, the blurrier the edges. When $p_r$At infinity, the range coefficients are approximately equal ( $exp(0)=1$ ), multiplied by the spatial domain template can be considered equivalent to Gaussian filtering. $p_r$The smaller it is, the sharper the edges. When $p_r$When infinitely close to 0, the range coefficient is approximately 0 except for the center position of 1 ( $exp(-\infty )=0$ ), and the result of filtering by multiplying with the spatial domain template is equivalent to the original image.

Five, C++ code implementation

1. A little knowledge of Mat in opencv

  In ordinary C++ coding, if we want to change the value of a variable like int or float through a function, we need to pass the address of the variable into the function. In opencv, if we want to change the value of a Mat, we only need to pass Mat into the function. For details, please refer to here .

2. About the processing of the boundary

  When the filter is used to process the border pixels, the filter will appear out of bounds. At this time, we replace it with the pixel corresponding to the out of bounds point in the original image.

As shown in the figure above, the pink position is the pixel to be filtered, and the filter size is $3\times 3.$ The position numbered 1 on the filter is obviously out of bounds (no pixel value), at this time we replace the pixel value at position 1 with the pixel value at position numbered 2.

3. Bilateral filtering code

double Wd(const int &i, const int &j, const float &sigma_d)
{
    
    
    return double(exp(-(pow(i, 2) + pow(j, 2)) / (2 * pow(sigma_d, 2))));
}
double Wr(const float &x, const float &sigma_r)
{
    
    
    return double(exp(-(pow(x, 2) / (2 * pow(sigma_r, 2)))));
}
// 边界处理函数
int reflect(int M, int x)
{
    
    
    if (x < 0)
        return -x;
    if (x >= M)
        return 2 * M - x-1;
    return x;

}

void ValueProcess(const cv::Mat &image, cv::Mat bFilterImage, const int &x, const int &y, const int &d, const float &sigma_d, const float &sigma_r)
{
    
    
    int k = d / 2;
    double iFiltered = 0;
    double wP = 0;

    int height = image.rows;
    int width = image.cols;

    int neighbor_x = 0;
    int neighbor_y = 0;

    for (int i = -k; i <= k; i++)
    {
    
    
        for (int j = -k; j <=k; j++)
        {
    
    
            //构造w_d权重
            double wd = Wd(i, j, sigma_d);
            //越界处理
            neighbor_x = reflect(height, x + i);
            neighbor_y = reflect(width, y + j);
            //构造w_r权重
            double wr = Wr(image.at<uchar>(neighbor_x, neighbor_y) - image.at<uchar>(x, y), sigma_r);
            double w = wd * wr;

            iFiltered = iFiltered + image.at<uchar>(neighbor_x, neighbor_y)*w;
            wP = wP + w;
        }
    }
    iFiltered = iFiltered / wP;
    bFilterImage.at<double>(x, y) = iFiltered;
}


cv::Mat BFilter(const cv::Mat &image, const int &d, const float &sigma_d, const float &sigma_r)
{
    
    
    int height = image.rows;
    int width = image.cols;

    cv::Mat bFilterImage = cv::Mat::zeros(height, width, CV_64FC1);

    for (int i = 0; i < height; i++)
    {
    
    
        for (int j = 0; j < width; j++)
        {
    
    
            //对每个像素点进行处理
            ValueProcess(image, bFilterImage, i, j, d, sigma_d, sigma_r);
        }
    } 
    return bFilterImage;

}

int main()
{
    
    
    cv::Mat image = cv::imread("LenaRGB.bmp");
    cv::Mat grayImage;
    cv::cvtColor(image, grayImage, CV_BGR2GRAY);
    
    // call b filter
    int d = 7;
    float sigma_d = 12.0;
    float sigma_r = 16.0;
    cv::Mat bFilterImage = BFilter(grayImage, d, sigma_d, sigma_r);
    cv::convertScaleAbs(bFilterImage, bFilterImage);  // 转换到8位uchar类型
    return 0;
}

Reference link:

https://www.cnblogs.com/snowxshy/p/3855011.html
https://zhuanlan.zhihu.com/p/127023952
https://blog.csdn.net/leonardohaig/article/details/118058527