Gaussian blur and image processing (Gaussian Blur)

Gaussian blur is widely used in image processing. In addition to conventional blur effects, it can also be used for image pyramid decomposition, anti-aliasing, high and low frequency decomposition, noise suppression, lighting effects, etc. Just because Gaussian blur is too basic and widely used, it is necessary to understand this ability as deeply as possible to avoid unintentional digging in practical applications.

One-dimensional Gaussian kernel function

official

$$G(x)=\frac{1}{\sqrt{2p.m}p}{e}^{-\frac{(x-\mu {)}^2}{2p^2}}$$

where $\mu$ is the axis of symmetry, and μ ≠ 0is usually not considered in image processing$m\ne 0$ , so this parameter can be ignored (this article ignores$m$ ).

$\sigma$ determines the height of the function,$The\; smaller\; the\; \sigma$ , the thinner it is, and the larger it is, the chunkier it is. If it is used for blurring, tall and thin means that the central pixel has a large contribution to the calculation result, while the contribution of distant pixels is relatively small, that is to say, the blurred result is more dominated by the central pixel, so the degree of blurring is low; otherwise, short and fat Means high degree of blur. So when other conditions remain unchanged, increase$\sigma$ will strengthen the blur, and vice versa will weaken the blur.

The coefficient before the e index ensures that the kernel function is at $(-\infty ,+\infty )$ is 1, but in image processing, it is obviously impossible to use an infinite radius, so the coefficients of the Gaussian kernel will be normalized, then the coefficients before the e index will be reduced in the normalization , so there is no need to calculate it at all.

Curve shape

The figure below is the curve shape of the one-dimensional Gaussian kernel function.

Note that the figure below still calculates the coefficient before the e index, and verifies the integral of the kernel function, indicating that no matter $How\; \sigma$ changes, its integral is always 1.

# -*- coding: utf-8 -*-
import numpy as np
import matplotlib.pyplot as plt


def gaussian_1d(x, sigma):
    coef = 1. / (np.sqrt(2 * np.pi) * sigma)
    return coef * np.exp(-x * x / (2 * sigma * sigma))


def plot_gaussian_1d():
    dx = 0.01
    xx = np.arange(-10, 10, dx)
    plt.figure(0, figsize=(8, 6))
    for sigma in [0.5, 1, 2]:
        yy = gaussian_1d(xx, sigma)
        ss = np.sum(yy * dx)
        plt.plot(xx, yy, label="sigma = %0.1f, integral = %0.2f" % (sigma, ss))
    plt.legend(loc='best')
    plt.savefig('gaussian_1d.png', dpi=300)
    plt.close(0)


if __name__ == '__main__':
    plot_gaussian_1d()

Effective Blur Radius

In general, increasing the blur radius increases the blur effect. However, it can be seen from the shape of the curve that the coefficient of the Gaussian kernel will become very small when it is far away from the center of symmetry. When doing image blurring, it means that the contribution of these distant pixels to the blurring will become very small. When increasing the radius, the degree of blur hardly increases, but the amount of calculation does not decrease at all.

So here should pay attention to the concept of an effective blur radius. Because $\sigma$ will affect the height of the curve, so it is easy to know that the effective blur radius is affected by$\sigma$ influence. In order to define the effective blur radius, we must define a standard, which should have a relative concept. For example, simply speaking, we define the ratio of the value of a point on the curve to the maximum value:
$$\underset{x}{\mathrm{argmax}}(\frac{G(x,p}{G(0,s)})>limitIf$$
we call the maximum x that satisfies the above formula the effective effective blur radius, then we can look for its relationship with$\sigma$ relationship.

Finding out the effective blur radius is very important for the practice of image processing, so as to avoid wasting computing power but not getting the corresponding blur effect. However, this concept is rarely discussed.

The following is the relationship between sigma and effective blur radius under different limit conditions. It is easy to see that this is a linear relationship.
Therefore, if we want to develop a "fuzzy" function and only open a fuzzy degree parameter, then generally speaking, this parameter needs to be linked to the radius, but in order for the radius to really work, we need an adapted sigma, this sigma You can use $sigma=radius/a$ to calculate. For image blurring, according to experience,[2, 2.5]any number between a can be used.

def effective_radius():
    dx = 0.01
    xx = np.arange(0, 100, dx)
    limits = [0.1, 0.01, 0.001]
    colors = ['red', 'green', 'blue']

    sigmas = np.arange(0.1, 10, 0.1)

    plt.figure(0, figsize=(8, 6))
    for i, limit in enumerate(limits):
        effective_r = []
        for sigma in sigmas:
            yy = gaussian_1d(xx, sigma)
            yy = yy / np.max(yy)
            radius = np.max(np.argwhere(yy >= limit))
            effective_r.append(radius * dx)
        plt.plot(sigmas, effective_r, color=colors[i],
                 label='limit=%0.3f' % limit)

    plt.legend(loc='best')
    plt.grid()
    plt.xlabel('sigma')
    plt.ylabel('effective radius')
    plt.savefig('effective radius', dpi=300)
    plt.close(0)

Two-dimensional Gaussian kernel function

official

$$G(x,y)=\frac{1}{2p.s{\_}_x{p}_y}{e}^{-(\frac{(x-m_x{)}^2}{2p_x^2}+\frac{(y-m_y{)}^2}{2p_y^2})}$$

For image blurring, we don't care about the mean value $\mu$ and the previous coefficients, so the formula can be simplified as:
$$G(x,y)=e^{-(\frac{x^2}{2p_x^2}+\frac{y^2}{2p_y^2})}$$

The following is the code and result of generating the kernel

# -*- coding: utf-8 -*-
import cv2
import numpy as np


def gaussian_2d(radius, sigma_x, sigma_y):
    coord = np.arange(-radius, radius, 0.01)
    xx, yy = np.meshgrid(coord, coord)
    res = np.exp(
        -(xx * xx / (sigma_x * sigma_x) + yy * yy / (sigma_y * sigma_y)) / 2)
    res = res / np.sum(res)
    return res


def plot_gaussian_2d():
    radius = 10
    sigma_x = 1.0
    sigma_y = 1.0

    kernel = gaussian_2d(radius, sigma_x, sigma_y)
    kernel = kernel / np.max(kernel)
    kernel = np.clip(np.round(kernel * 255), 0, 255)
    kernel = np.uint8(kernel)

    cv2.imwrite('radius=%d, sigma=(%d, %d).png' % (radius, sigma_x, sigma_y),
                kernel)

Gaussian Blur with Nested Loops of Rows and Columns

# -*- coding: utf-8 -*-
import cv2
import numpy as np


def gaussian_2d(radius, sigma_x, sigma_y, d_radius=1):
    coord = np.arange(-radius, radius + d_radius, d_radius)
    xx, yy = np.meshgrid(coord, coord)
    res = np.exp(-(xx * xx / (sigma_x * sigma_x) +
                   yy * yy / (sigma_y * sigma_y)) / 2)
    res = res / np.sum(res)
    return res


def convolve_2d(image, kernel, border='transparent'):
    image_height, image_width = image.shape[:2]
    kernel_height, kernel_width = kernel.shape[:2]
    radius_h = kernel_height // 2
    radius_w = kernel_width // 2

    # check: kernel_height and kernel_width must be odd
    if radius_h * 2 + 1 != kernel_height or radius_w * 2 + 1 != kernel_width:
        raise ValueError('kernel_height and kernel_width must be odd')

    res = np.zeros_like(image, dtype=np.float32)
    for row in range(image_height):
        # print(row)
        for col in range(image_width):
            pix = 0.0
            weight = 0
            for i in range(kernel_height):
                for j in range(kernel_width):
                    row_k = row + i - radius_h
                    col_k = col + j - radius_w
                    if border == 'transparent':
                        if 0 <= row_k < image_height and \
                                0 <= col_k < image_width:
                            pix += image[row_k, col_k] * kernel[i, j]
                            weight += kernel[i, j]
                    elif border == 'zero':
                        if 0 <= row_k < image_height and \
                                0 <= col_k < image_width:
                            pix += image[row_k, col_k] * kernel[i, j]
                        weight += kernel[i, j]
                    elif border == 'copy':
                        if row_k < 0:
                            row_k = 0
                        elif row_k >= image_height:
                            row_k = image_height - 1
                        if col_k < 0:
                            col_k = 0
                        elif col_k >= image_width:
                            col_k = image_width - 1
                        pix += image[row_k, col_k] * kernel[i, j]
                        weight += kernel[i, j]
                    elif border == 'reflect':
                        if row_k < 0:
                            row_k = np.abs(row_k)
                        elif row_k >= image_height:
                            row_k = 2 * (image_height - 1) - row_k
                        if col_k < 0:
                            col_k = np.abs(col_k)
                        elif col_k >= image_height:
                            col_k = 2 * (image_width - 1) - col_k
                        pix += image[row_k, col_k] * kernel[i, j]
                        weight += kernel[i, j]
                    else:
                        raise ValueError('border must be one of '
                                         '[transparent, zero, copy, reflect]')
            res[row, col] = np.float32(pix / weight)
    res = np.uint8(np.clip(np.round(res), 0, 255))
    return res


def main_gaussian_blur():
    radius = 20
    sigma_x = 10.0
    sigma_y = 10.0
    borders = ['transparent', 'zero', 'copy', 'reflect']

    image = cv2.imread('lena_std.bmp')
    kernel = gaussian_2d(radius, sigma_x, sigma_y, d_radius=1)

    for border in borders:
        blur_image = convolve_2d(image, kernel, border)
        cv2.imwrite('blur_radius=%d,sigma=(%0.1f,%0.1f),border=%s.png' % (
            radius, sigma_x, sigma_y, border), blur_image)

The above code is only used as an example calculation process, because python's for loop is very slow, so as long as the kernel is slightly larger, the above code will run for a very, very long time. Since the image needs to be cycled in two directions, the kernel also needs to be cycled in two directions, so the overall calculation amount is o(n^4), which is very slow.

In addition to the regular image Gaussian blur, the above code also illustrates several boundary conditions:

• Transparent: Indexes beyond the boundaries do not participate in the calculation. Such boundary conditions must accumulate weights, and finally use the accumulated weights for normalization. Because this boundary condition must accumulate the weight, the amount of calculation is slightly larger, but the effect is the best and most stable among several boundary conditions.
• zero: Out-of-bounds indices are replaced with 0. Because 0 represents black, as the kernel increases, more and more 0s participate in the blurring of the boundary, so this boundary condition will cause black edges, and the effect is the worst. But for parallel acceleration, its implementation is relatively easy. It is used a lot in the convolution of neural networks, but try to avoid using such boundary conditions in traditional image processing.
• copy: The index beyond the border uses the nearest edge pixel value, the effect is quite satisfactory, and the implementation is also simple.
• reflect: The index beyond the border, taking the pixel value of the corresponding position inside the image with the border as the symmetric center, the calculation of the index will be a bit troublesome, especially when the kernel size is very large, so that the mirror index exceeds the opposite border. But when the kernel size is not so extreme, the effect of this boundary condition is generally not bad.

The figure below is the effect of the boundary conditions in 4, radius=20, sigma=10.
Upper left: transparent, upper right: zero,
lower left: copy, lower right: reflect

Gaussian blur with row and column separation

Gaussian blur using nested loops is obviously poorly written. In practice, we usually write it in the form of row-column separation, and the calculation amount is reduced from o(n^4) to o(n^3).
Above we wrote the two-dimensional Gaussian kernel function as:
$$G(x,y)=e^{-(\frac{x^2}{2p_x^2}+\frac{y^2}{2p_y^2})}$$
Note that this function can be disassembled and written in the form of x and y separation.
Remember
$$\begin{gathered} G(x)=e^{-\frac{x^2}{2p_x^2}} \\ G(y)=e^{-\frac{y^2}{2p_y^2}} \end{gathered}$$
那么有：
$$G(x,y)=e^{-\frac{x^2}{2p_x^2}}\cdot e^{-\frac{y^2}{2p_y^2}}=G\left(x\right)G\left(y\right)$$
and the calculation process of a Gaussian blur is:
$$\begin{array}{rl}blur(x,y)& =\int \int I(x,y)G(x,y)dxdy\\ & =\int \int I(x,y)G(x)G(y)dxdy\\ & =\int [\int I(x,y)G(x)dx]\cdot G\left(y\right)dy\\ & (written\; in\; discrete\; form)\\ & =\sum _{y_i}[\sum _{x_i}I(x_i,y_i)G(x_i)]G(y_i)\end{array}$$

That is to say, blur in the x direction, and then blur the intermediate result in the y direction. Only loop in one direction each time, and the two loops are not nested, but separated. The following code does not implement multiple boundary conditions, but only writes the transparent type.

# -*- coding: utf-8 -*-
import cv2
import numpy as np


def gaussian_1d(radius, sigma, d_radius=1):
    xx = np.arange(-radius, radius + d_radius, d_radius)
    res = np.exp(-xx * xx / (sigma * sigma) / 2)
    res = res / np.sum(res)
    return res


def convolve_2d_xy(image, kernel_x, kernel_y):
    image_height, image_width = image.shape[:2]
    kernel_x_len = len(kernel_x)
    kernel_y_len = len(kernel_y)
    radius_x = kernel_x_len // 2
    radius_y = kernel_y_len // 2

    # check: kernel_height and kernel_width must be odd
    if radius_x * 2 + 1 != kernel_x_len or radius_y * 2 + 1 != kernel_y_len:
        raise ValueError('kernel size must be odd')

    res = np.zeros_like(image, dtype=np.float32)
    # convolve in x direction
    for row in range(image_height):
        for col in range(image_width):
            pix = 0.0
            weight = 0
            for j in range(kernel_x_len):
                col_k = col + j - radius_x
                if 0 <= col_k < image_width:
                    pix += image[row, col_k] * kernel_x[j]
                    weight += kernel_x[j]
            res[row, col] = pix / weight

    # convolve in y direction
    image = res.copy()
    for col in range(image_width):
        for row in range(image_height):
            pix = 0.0
            weight = 0
            for i in range(kernel_y_len):
                row_k = row + i - radius_y
                if 0 <= row_k < image_height:
                    pix += image[row_k, col] * kernel_y[i]
                    weight += kernel_y[i]
            res[row, col] = pix / weight
    res = np.uint8(np.clip(np.round(res), 0, 255))
    return res


def main_gaussian_blur_xy():
    radius = 20
    sigma_x = 10.0
    sigma_y = 10.0

    image = cv2.imread('lena_std.bmp')
    kernel_x = gaussian_1d(radius, sigma_x, d_radius=1)
    kernel_y = gaussian_1d(radius, sigma_y, d_radius=1)

    blur_image = convolve_2d_xy(image, kernel_x, kernel_y)
    cv2.imwrite('xy_blur_radius=%d,sigma=(%0.1f,%0.1f).png' % (
        radius, sigma_x, sigma_y), blur_image)

The time-consuming of row and column separation is about 45s on my computer, and the nested loop takes about 1020 seconds. Although python's loop is extremely slow, this time comparison may not be counted, but the difference in magnitude is more telling.

The left picture below is the result of the previous nested loop, the middle is the result of the separation of rows and columns, and the right picture is the absolute value of the difference between the two and then multiplied by 50. As a result, the two fuzzy results are exactly the same.

Gaussian blur with step

When the fuzzy radius is very large, some steps can be added appropriately to save calculations.
At this time, it should be noted that in order to achieve the expected blur effect, not only the index of image patch and kernel multiplication and addition needs to add step, but also the calculation of kernel coefficient needs to add step , otherwise the coefficient will be very large compared to the time without step difference.

# -*- coding: utf-8 -*-
import cv2
import numpy as np
import time


def gaussian_1d(radius, sigma, d_radius=1.0):
    xx = np.arange(-radius, radius + d_radius, d_radius)
    res = np.exp(-xx * xx / (sigma * sigma) / 2)![请添加图片描述](https://img-blog.csdnimg.cn/0d509cd3e63a4f4680df49de7b807309.png)

    res = res / np.sum(res)
    return res


def convolve_2d_xy_step(image, kernel_x, kernel_y, step):
    image_height, image_width = image.shape[:2]
    kernel_x_len = len(kernel_x)
    kernel_y_len = len(kernel_y)
    radius_x = kernel_x_len // 2
    radius_y = kernel_y_len // 2

    # check: kernel_height and kernel_width must be odd
    if radius_x * 2 + 1 != kernel_x_len or radius_y * 2 + 1 != kernel_y_len:
        raise ValueError('kernel size must be odd')

    res = np.zeros_like(image, dtype=np.float32)
    # convolve in x direction
    for row in range(image_height):
        for col in range(image_width):
            pix = 0.0
            weight = 0
            for j in range(kernel_x_len):
                col_k = int(round(col + (j - radius_x) * step))
                if 0 <= col_k < image_width:
                    pix += image[row, col_k] * kernel_x[j]
                    weight += kernel_x[j]
            res[row, col] = pix / weight

    # convolve in y direction
    image = res.copy()
    for col in range(image_width):
        for row in range(image_height):
            pix = 0.0
            weight = 0
            for i in range(kernel_y_len):
                row_k = int(round(row + (i - radius_y) * step))
                if 0 <= row_k < image_height:
                    pix += image[row_k, col] * kernel_y[i]
                    weight += kernel_y[i]
            res[row, col] = pix / weight
    res = np.uint8(np.clip(np.round(res), 0, 255))
    return res


def main_gaussian_blur_xy():
    radius = 20
    sigma_x = 10.0
    sigma_y = 10.0
    step = 4.0

    image = cv2.imread('lena_std.bmp')
    kernel_x = gaussian_1d(radius, sigma_x, d_radius=step)
    kernel_y = gaussian_1d(radius, sigma_y, d_radius=step)

    blur_image = convolve_2d_xy_step(image, kernel_x, kernel_y, step)
    cv2.imwrite('xy_blur_radius=%d,sigma=(%0.1f,%0.1f)_step.png' % (
        radius, sigma_x, sigma_y), blur_image)

The left picture below is the result of the previous nested loop, the middle is the result of row and column separation + step=4, and the right picture is the absolute value of the difference between the two and then multiplied by 50.
At this time, the calculated diff is no longer all zeros. If you look closely at the intermediate results, it shows some blocky textures, indicating that the quality of the blur has begun to be affected.

Generation of Arbitrary 2D Gaussian Kernels

Here any Gaussian kernel refers to a Gaussian kernel with an arbitrary aspect ratio and a rotation angle.
The idea of ​​formula derivation is to regard the Gaussian kernel G(x,y) as an image, and then use the idea of ​​image rotation to transform the formula of the two-dimensional Gaussian kernel.

Below we put the Gaussian kernel formula here again.
$$G(x,y)=e^{-(\frac{x^2}{2p_x^2}+\frac{y^2}{2p_y^2})}$$
According to the principle of image rotation, when a point (x, y) in the image takes the center of the circle as the center of rotation and rotates counterclockwise, the coordinate formula of the new point (x', y') is: { x ′ =
$$\{\begin{array}{l}{x}^{\prime }=x\ast cos\theta -y\ast sin\theta \\ y^{\prime }=x\ast sin\theta +y\ast cos\theta \end{array}$$

Substituting it into the Gaussian kernel formula, and expressing it as (x, y) for convenience, then there are:
$$\begin{array}{rl}G(x,y)& =e^{-(\frac{(x\ast cos\theta -y\ast sin\theta {)}^2}{2p_x^2}+\frac{(x\ast sin\theta +y\ast cos\theta {)}^2}{2p_y^2})}\\ & =e^{-(\frac{x^2\ast co{s}^2\theta +y^2\ast si{n}^2\theta -2xy\ast sin\theta cos}{2p_x^2}+\frac{x^{2}si{n}^2\theta +y^2\ast co{s}^2\theta +2xy\ast sin\theta cos}{2p_y^2})}\\ & =e^{-(x^2(\frac{co{s}^2}{2p_x^2}+\frac{si{n}^2}{2p_y^2})+xy(-\frac{sin2\theta }{2p_x^2}+\frac{sin2\theta }{2p_y^2})+y^2(\frac{si{n}^2}{2p_x^2}+\frac{co{s}^2}{2p_y^2}))}\end{array}$$

The above formula is what an arbitrary 2D Gaussian kernel looks like.
The following is the implementation code and diagram of python, which rotates a kernel that is relatively long in the horizontal direction by 20 degrees counterclockwise.

# -*- coding: utf-8 -*-
import cv2
import numpy as np


def generalized_gaussian_kernel(ksize,
                                sigma_x,
                                sigma_y=None,
                                angle=0.0):
    """
    Generate generalized gaussian kernel.

    Parameters
    ----------
    ksize: kernel size, one integer or a list/tuple of two integers, must be
        odd
    sigma_x: standard deviation in x direction
    sigma_y: standard deviation in y direction
    angle: rotate angle, anti-clockwise

    Returns
    -------
    kernel: generalized gaussian blur kernel
    """
    # check parameters
    if not isinstance(ksize, (tuple, list)):
        ksize = [ksize, ksize]
    else:
        ksize = list(ksize)
    ksize[0] = ksize[0] // 2 * 2 + 1
    ksize[1] = ksize[1] // 2 * 2 + 1

    if sigma_y is None:
        sigma_y = sigma_x

    # meshgrid coordinates
    radius_x = ksize[0] // 2
    radius_y = ksize[1] // 2
    x = np.arange(-radius_x, radius_x + 1, 1)
    y = np.arange(-radius_y, radius_y + 1, 1)
    xx, yy = np.meshgrid(x, y)

    # coefficients of coordinates
    angle = angle / 180 * np.pi
    cos_square = np.cos(angle) ** 2
    sin_square = np.sin(angle) ** 2
    sin_2 = np.sin(2 * angle)

    alpha = 0.5 / (sigma_x * sigma_x)
    beta = 0.5 / (sigma_y * sigma_y)
    a = cos_square * alpha + sin_square * beta
    b = -sin_2 * alpha + sin_2 * beta
    c = sin_square * alpha + cos_square * beta

    # generate and normalize kernel
    kernel = np.exp(-(a * xx * xx + b * xx * yy + c * yy * yy))
    kernel = kernel / np.sum(kernel)
    return kernel


if __name__ == '__main__':
    ksize = (511, 511)
    sigma_x = 100.0
    sigma_y = 20.0
    angle = 20.0

    gaussian_kernel = generalized_gaussian_kernel(ksize=ksize,
                                                  sigma_x=sigma_x,
                                                  sigma_y=sigma_y,
                                                  angle=angle)
    gaussian_kernel = gaussian_kernel / np.max(gaussian_kernel)
    gaussian_kernel = np.uint8(np.round(gaussian_kernel * 255))
    cv2.imwrite('generalized_gaussian_kernel_1.png', gaussian_kernel)