OpenCV image processing (medium) image smoothing + histogram

1. Image smoothing

1. Image noise

Because image acquisition, processing, transmission and other processes will inevitably be polluted by noise, which hinders people's understanding and analysis of images. Common image noises include Gaussian noise, salt and pepper noise, etc.

1.1 Salt and Pepper Noise

Salt and pepper noise, also known as impulse noise, is a kind of noise that is often seen in images. It is a random white point or black point, which may be black pixels in bright areas or white pixels in dark areas ( or both). The salt and pepper noise may be caused by sudden and strong interference of the video signal, analog-to-digital converter or bit transmission error, etc. For example, a failed sensor results in a minimum pixel value, and a saturated sensor results in a maximum pixel value.

1.2 Gaussian noise

Gaussian noise refers to a type of noise whose noise density function obeys a Gaussian distribution. This noise (also known as normal noise) model is often used in practice due to the mathematical tractability of Gaussian noise in the spatial and frequency domains. The probability density function of a Gaussian random variable z is given by:

Where z represents the gray value, μ represents the mean or expected value of z, and σ represents the standard deviation of z. The square of the standard deviation $p^2$ is called the variance of z. The curve of the Gaussian function is shown in the figure.

2. Introduction to image smoothing

From the perspective of signal processing, image smoothing is to remove high-frequency information and retain low-frequency information. So we can perform low-pass filtering on the image. Low-pass filtering can remove the noise in the image and smooth the image.

According to different filters, it can be divided into mean filter, Gaussian filter, median filter and bilateral filter.

2.1 Mean filtering

Image noise is filtered out by means of a mean filter template. Let $S_{xy}$Indicates the coordinate group of a rectangular sub-image window whose center is at point (x, y) and whose size is m×n. The mean filter can be expressed as:

completed by a normalized convolution box. It just replaces the center element with the average of all pixels in the area covered by the convolution box.

For example, the 3x3 normalized average filter is as follows:

the advantage of the average filter is that the algorithm is simple and the calculation speed is fast, but the disadvantage is that it removes a lot of details while denoising and blurs the image.

API:

cv.blur(src, ksize, anchor, borderType)

parameters :

• src: input image
• ksize: the size of the convolution kernel
• anchor: the default value (-1,-1), indicating the core center
• borderType: border type

Example :

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
# 1 图像读取
img = cv.imread('./image/dogsp.jpeg')
# 2 均值滤波
blur = cv.blur(img,(5,5))
# 3 图像显示
plt.figure(figsize=(10,8),dpi=100)
plt.subplot(121),plt.imshow(img[:,:,::-1]),plt.title('原图')
plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(blur[:,:,::-1]),plt.title('均值滤波后结果')
plt.xticks([]), plt.yticks([])
plt.show()

2.2 Gaussian filtering

The two-dimensional Gaussian is the basis for constructing a Gaussian filter, and its probability distribution function is as follows:

The distribution of G(x,y) is in the shape of a raised hat. Here σ can be regarded as two values, one is the standard deviation $p_x$​​​​ , the other is the standard deviation σ y \sigma_y in the y direction$p_y$。

当$p_x$和$p_y$The larger the value, the whole shape tends to be flat; when $p_x$和$p_y$​​ , the whole shape is more prominent.

The normal distribution is a bell-shaped curve, the closer to the center, the larger the value, and the farther away from the center, the smaller the value. When calculating the smoothing result, you only need to use the "central point" as the origin, and assign weights to other points according to their positions on the normal curve to obtain a weighted average.

Gaussian smoothing is very effective at removing Gaussian noise from images.

Gaussian smoothing process :

• First determine the weight matrix

Assuming that the coordinates of the center point are (0,0), then the coordinates of the 8 points closest to it are as follows:

Further points and so on.

In order to calculate the weight matrix, the value of σ needs to be set. Assuming σ=1.5, the weight matrix with a blur radius of 1 is as follows:

The sum of the weights of these 9 points is equal to 0.4787147. If only the weighted average of these 9 points is calculated, the sum of their weights must be equal to 1, so the above 9 values ​​must be divided by 0.4787147 to obtain the final weight matrix.

• Calculate Gaussian Blur

With the weight matrix, the value of Gaussian blur can be calculated.

Assuming that there are 9 pixels, the gray value (0-255) is as follows:

Each point is multiplied by the corresponding weight value:

get

Adding up these 9 values ​​is the value of the Gaussian blur at the center point.

Repeating this process for all points yields a Gaussian blurred image. If the original image is a color image, Gaussian smoothing can be performed on the three channels of RGB respectively.

API：

cv2.GaussianBlur(src,ksize,sigmaX,sigmay,borderType)

Parameters :

• src: input image
• ksize: the size of the Gaussian convolution kernel, note: the width and height of the convolution kernel should be odd and can be different
• sigmaX: standard deviation in the horizontal direction
• sigmaY: the standard deviation in the vertical direction, the default value is 0, which means the same as sigmaX
• borderType: padding border type

Example :

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
# 1 图像读取
img = cv.imread('./image/dogGasuss.jpeg')
# 2 高斯滤波
blur = cv.GaussianBlur(img,(3,3),1)
# 3 图像显示
plt.figure(figsize=(10,8),dpi=100)
plt.subplot(121),plt.imshow(img[:,:,::-1]),plt.title('原图')
plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(blur[:,:,::-1]),plt.title('高斯滤波后结果')
plt.xticks([]), plt.yticks([])
plt.show()

2.3 Median filtering

Median filtering is a typical nonlinear filtering technique. The basic idea is to replace the gray value of the pixel with the median value of the neighborhood gray value of the pixel.

Median filtering is especially useful for salt-and-pepper noise because it does not rely on values ​​in the neighborhood that differ significantly from typical values.

API：

cv.medianBlur(src, ksize )

Parameters :

• src: input image
• ksize: the size of the convolution kernel

Example:

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
# 1 图像读取
img = cv.imread('./image/dogsp.jpeg')
# 2 中值滤波
blur = cv.medianBlur(img,5)
# 3 图像展示
plt.figure(figsize=(10,8),dpi=100)
plt.subplot(121),plt.imshow(img[:,:,::-1]),plt.title('原图')
plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(blur[:,:,::-1]),plt.title('中值滤波后结果')
plt.xticks([]), plt.yticks([])
plt.show()

2. Histogram

1 Gray Histogram

1.1 Principle

A histogram is a way of performing statistics on data and organizing the statistical values ​​into a series of implementation-defined bins. Among them, bin is a concept often used in the histogram, which can be translated as "straight bar" or "group interval", and its value is the characteristic statistic calculated from the data, which can be such as gradient, direction, color or any other feature.

An image histogram is a histogram used to represent the brightness distribution in a digital image, plotting the number of pixels for each brightness value in the image. In this histogram, the left side of the abscissa is the darker area, while the right side is the brighter area. Therefore, the data in the histogram of a darker image is more concentrated on the left and middle parts, while the overall bright image with only a few shadows is the opposite.

Note: Histograms are drawn from grayscale images, not color images. Suppose there is an image information (gray value 0 - 255, the range of known numbers contains 256 values, so this range can be divided into sub-regions (that is, bins) according to certain rules. For example:

Then count the number of pixels in each bin(i). The following figure can be obtained (where the x-axis represents the bin, and the y-axis represents the number of pixels in each bin):

Some terms and details for histograms :

• dims: The number of features that need to be counted. In the above example, dims = 1 because only grayscale values ​​are counted.
• bins: The number of subsections of each feature space, which can be translated as "straight bar" or "group distance". In the above example, bins = 16.
• range: The value range of the feature to be counted. In the above example, range = [0, 255].

The meaning of the histogram :

• A histogram is a graphical representation of the distribution of pixel intensities in an image.
• It counts the number of pixels each intensity value has.
• Histograms of different images may be the same

1.2 Calculation and drawing of histogram

We use the method in OpenCV to count the histogram and use matplotlib to draw it.

API：

cv2.calcHist(images,channels,mask,histSize,ranges[,hist[,accumulate]])

Parameters :

• images: Original images. When passing in a function, it should be enclosed in square brackets [], for example: [img].
• channels: If the input image is a grayscale image, its value is [0]; if it is a color image, the incoming parameters can be [0], [1], [2] which correspond to channels B, G, R.
• mask: mask image. To count the histogram of the entire image set it to None. But if you want to count the histogram of a certain part of the image, you need to make a mask image and use it. (examples follow)
• histSize: The number of BINs. It should also be enclosed in square brackets, eg: [256].
• ranges: range of pixel values, usually [0, 256]

Example :

As shown in the figure below, draw the corresponding histogram

import numpy as np
import cv2 as cv
from matplotlib import pyplot as plt
# 1 直接以灰度图的方式读入
img = cv.imread('./image/cat.jpeg',0)
# 2 统计灰度图
histr = cv.calcHist([img],[0],None,[256],[0,256])
# 3 绘制灰度图
plt.figure(figsize=(10,6),dpi=100)
plt.plot(histr)
plt.grid()
plt.show()

1.3 Application of mask

A mask is to use a selected image, figure or object to block the image to be processed to control the area of ​​image processing.

In digital image processing, we usually use a two-dimensional array of matrices for masking. The mask is a binary image composed of 0 and 1, and the image to be processed is masked by using the mask image, in which the area with a value of 1 is processed, and the area with a value of 0 is masked and will not be processed.

The main uses of masks are:

• Extract the region of interest: use the pre-made mask of the region of interest and the image to be processed to perform the "AND" operation to obtain the image of the region of interest. The image values ​​​​in the region of interest remain unchanged, while the values ​​​​of images outside the region are all 0.
• Shielding function: Use a mask to shield certain areas on the image so that they do not participate in processing or calculation of processing parameters, or only process or count the shielded areas.
• Structural feature extraction: Use similarity variables or image matching methods to detect and extract structural features similar to masks in images.
• special shape image production

Masks are widely used in remote sensing image processing. When extracting roads, rivers, or houses, a mask matrix is ​​used to filter the pixels of the image, and then the features or signs we need are highlighted.

We use cv.calcHist()to find the histogram of the full image. What if you want to find the histogram of some region of the image? Just create a mask image of white on the areas where you want to find the histogram, black otherwise, and pass it as a mask.

Example:

import numpy as np
import cv2 as cv
from matplotlib import pyplot as plt
# 1. 直接以灰度图的方式读入
img = cv.imread('./image/cat.jpeg',0)
# 2. 创建蒙版
mask = np.zeros(img.shape[:2], np.uint8)
mask[400:650, 200:500] = 255
# 3.掩模
masked_img = cv.bitwise_and(img,img,mask = mask)
# 4. 统计掩膜后图像的灰度图
mask_histr = cv.calcHist([img],[0],mask,[256],[1,256])
# 5. 图像展示
fig,axes=plt.subplots(nrows=2,ncols=2,figsize=(10,8))
axes[0,0].imshow(img,cmap=plt.cm.gray)
axes[0,0].set_title("原图")
axes[0,1].imshow(mask,cmap=plt.cm.gray)
axes[0,1].set_title("蒙版数据")
axes[1,0].imshow(masked_img,cmap=plt.cm.gray)
axes[1,0].set_title("掩膜后数据")
axes[1,1].plot(mask_histr)
axes[1,1].grid()
axes[1,1].set_title("灰度直方图")
plt.show()

2. Histogram equalization

2.1 Principle and application

Imagine what would happen if the pixel values ​​of most of the pixels in an image were concentrated in a certain small gray value range? If an image is bright overall, the number of values ​​for all pixel values ​​should be high. Therefore, its histogram should be stretched horizontally (as shown in the figure below), which can expand the distribution range of image pixel values ​​and improve the contrast of the image. This is what histogram equalization should do.

"Histogram equalization" is to change the gray histogram of the original image from a relatively concentrated gray-scale interval to a distribution in a wider gray-scale range. Histogram equalization is to stretch the image nonlinearly and redistribute the pixel values ​​of the image so that the number of pixels in a certain gray scale range is roughly the same.
This method improves the overall contrast of the image, especially when the pixel value distribution of the useful data is relatively close. It is widely used in X-ray images, which can improve the display of the skeleton structure. In addition, it can be better highlighted in over- or under-exposed images. detail.
When using opencv for histogram statistics, use:

API：

dst = cv.equalizeHist(img)

Parameters :

• img: grayscale image

returns :

• dst : the result after equalization

Example:

import numpy as np
import cv2 as cv
from matplotlib import pyplot as plt
# 1. 直接以灰度图的方式读入
img = cv.imread('./image/cat.jpeg',0)
# 2. 均衡化处理
dst = cv.equalizeHist(img)
# 3. 结果展示
fig,axes=plt.subplots(nrows=2,ncols=2,figsize=(10,8),dpi=100)
axes[0].imshow(img,cmap=plt.cm.gray)
axes[0].set_title("原图")
axes[1].imshow(dst,cmap=plt.cm.gray)
axes[1].set_title("均衡化后结果")
plt.show()

2.2 Adaptive histogram equalization

In the above histogram equalization, we consider the global contrast of the image. It is true that after the histogram equalization, the contrast of the background of the picture is changed, the cat legs are too dark, we lose a lot of information, so in many cases, the effect of this is not good. As shown in the image below, comparing the images of the statue in the next two images, we lost a lot of information due to being too bright.

To solve this problem, adaptive histogram equalization is used. At this point, the entire image will be divided into many small blocks, these small blocks are called "tiles" (the default size of tiles in OpenCV is 8x8), and then histogram equalization is performed on each small block. So in each area, the histogram will be concentrated in a small area). Noise is amplified, if it is present. To avoid this situation use contrast limiting. For each small block, if the bin in the histogram exceeds the upper limit of the contrast, the pixels in it are evenly distributed to other bins, and then the histogram is equalized.

Finally, in order to remove the boundary between each small block, the bilinear difference is used to stitch each small block.

API：

cv.createCLAHE(clipLimit, tileGridSize)

Parameters :

• clipLimit: contrast limit, default is 40
• tileGridSize: The size of the tile, the default is 8*88∗8

Example :

import numpy as np
import cv2 as cv
# 1. 以灰度图形式读取图像
img = cv.imread('./image/cat.jpeg',0)
# 2. 创建一个自适应均衡化的对象，并应用于图像
clahe = cv.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
cl1 = clahe.apply(img)
# 3. 图像展示
fig,axes=plt.subplots(nrows=1,ncols=2,figsize=(10,8),dpi=100)
axes[0].imshow(img,cmap=plt.cm.gray)
axes[0].set_title("原图")
axes[1].imshow(cl1,cmap=plt.cm.gray)
axes[1].set_title("自适应均衡化后的结果")
plt.show()