Table of contents
1. Experimental content and methods
Five, grayscale image equalization
1. Non-scaling library equalization
1. Split and merge equalization
2. Library function equalization
3. HSI model<->RGB model exchange and equalization
Series Article Directory
Experiment 1: Image denoising and histogram equalization complete code: Li Yiru - Gitee.com
This series of blogs focuses on the concept, principle and code practice of computer vision, and does not include tedious mathematical derivation (if you have any questions, please discuss and point them out in the comment area, or contact me directly by private message).
Chapter 1 Computer Vision - Image Denoising and Histogram Equalization (Image Enhancement)
synopsis
This blog mainly introduces the image representation of grayscale images and color images, the increase of various image noises, the method of denoising/reduction, the custom realization of the histogram construction and equalization of different images, and compares the effect with the opencv library Analysis (python code attached)
1. Experimental content and methods
Experiment content:Select one or two images, do not call library functions, program by yourself to achieve image denoising (add Gaussian noise and salt and pepper noise) and histogram equalization, and compare and analyze the effect with OpenCV library functions;
Experimental pictures : The pictures selected for the experiment are shown in Figure 1 and Figure 2 :
2. Image noise addition
Image noise refers to unnecessary or redundant interference information existing in image data . According to its probability distribution and generation method, it can be divided into different types. In this experiment, Gaussian noise and salt and pepper noise are taken as examples.
1. Gaussian noise
Gaussian noise : refers to a type of noise that obeys a Gaussian distribution (normal distribution), usually sensor noise caused by poor lighting and high temperature.
The code implementation of adding Gaussian noise (take the noise addition of grayscale image as an example) is as follows:
def gaussian_noise(img, mean, sigma):
"""
此函数用将产生的高斯噪声加到图片上
传入: img : 原图 mean : 均值 sigma : 标准差
返回: gaussian_out : 噪声处理后的图片
"""
# 将图片灰度标准化
img = img / 255
# 产生高斯 noise
noise = np.random.normal(mean, sigma, img.shape)
# 将噪声和图片叠加
gaussian_out = img + noise
# 将超过 1 的置 1，低于 0 的置 0
gaussian_out = np.clip(gaussian_out, 0, 1)
# 将图片灰度范围的恢复为 0-255
gaussian_out = np.uint8(gaussian_out * 255)
# 将噪声范围搞为 0-255
# noise = np.uint8(noise*255)
return gaussian_out # 这里也会返回噪声，注意返回值The picture to be tested in Figure 1 is shown before and after adding Gaussian noise (take mean = 0 , sigma = 0.1 as an example)
2. Salt and pepper noise
Salt and Pepper Noise: Salt and Pepper Noise is added to the picture 黑白噪点. Pepper refers to 黑色的噪点(0,0,0)salt 白色的噪点(255,255,255). The ratio of added noise is controlled by setting. amountThe larger the value, the more noise is added, and the image damage is more serious.
The code implementation of adding salt and pepper noise (take the noise addition of grayscale image as an example) is as follows:
def sp_noise(noise_img, proportion):
"""
添加椒盐噪声
proportion的值表示加入噪声的量，可根据需要自行调整
return: img_noise
"""
height, width = noise_img.shape[0], \
noise_img.shape[1] # 获取高度宽度像素值
num = int(height * width * proportion) # 一个准备加入多少噪声小点
for i in range(num):
w = random.randint(0, width - 1)
h = random.randint(0, height - 1)
if random.randint(0, 1) == 0:
noise_img[h, w] = 0
else:
noise_img[h, w] = 255
return noise_imgThe pictures to be tested in Figure 1 are shown before and after adding Gaussian noise (take mean = 0 , sigma = 0.1 as an example) and salt and pepper noise ( proportion = 0.1 as an example) .
3. Image denoising/reduction
The existence of noise seriously affects the quality of remote sensing images, so before image enhancement processing and classification processing, image denoising/reduction methods must be used to correct it. This experiment takes the median filter and mean filter as examples.
1. Mean filtering
The average value of pixels included in the filter field is essentially a method of taking the average value of the target pixel and surrounding pixels and then filling in the target pixel to achieve the purpose of filtering. The weighted average formula is shown in the figure.
The code implementation using the mean filter (taking the linear non-weighted noise reduction of the grayscale image as an example) is as follows:
# 均值降噪转换器
def convert_2d(r):
n = 3
# 3*3 滤波器, 每个系数都是 1/9
window = np.ones((n, n)) / n ** 2
# 使用滤波器卷积图像
# mode = same 表示输出尺寸等于输入尺寸
# boundary 表示采用对称边界条件处理图像边缘
s = scipy.signal.convolve2d(r,
window, mode='same', boundary='symm')
return s.astype(np.uint8)Add Gaussian noise (take mean = 0 , sigma = 0.1 as an example) and salt-and-pepper noise ( proportion = 0.1 as an example) before and after denoising the picture to be tested in Figure 1 , as shown in the figure.
Analysis : It can be seen from the figure that the linear non-weighted mean filter has poor noise reduction effect, and the filtered image is blurred and loses more information. Therefore, a weighted mean filter or other filters are generally used in actual use.
2. Median filtering
Similar to the mean filtering method, the pixel is replaced by the intermediate value in the pixel field , and the formula and principle are shown in 0.
The code implementation using the median filter (taking the noise reduction of the grayscale image as an example) is as follows:
# 自定义中值降噪滤波器
def medium_filter(im, x, y, step):
sum_s = []
for k in range(-int(step / 2), int(step / 2) + 1):
for m in range(-int(step / 2), int(step / 2) + 1):
sum_s.append(im[x + k][y + m])
sum_s.sort()
return sum_s[(int(step * step / 2) + 1)]Add Gaussian noise (take mean = 0 , sigma = 0.1 as an example) and salt-and-pepper noise ( proportion = 0.1 as an example) before and after denoising the picture to be tested in Figure 1 , as shown in the figure.
Analysis : It can be seen from the figure that the noise in the picture is significantly reduced, which proves that the median filter has a better noise reduction effect (compared to the mean filter), and the filter can better preserve the sharpness of the edge and the details of the image while removing the noise , so this experimental denoising method finally uses the median filter.
4. Histogram and equalization
1. Histogram
Histogram : In image processing, it is often necessary to analyze the brightness of the image (that is, the distribution of the pixel level), which requires the use of histograms, such as color histograms, grayscale histograms, etc.
The histogram is constructed by using the discrete function + normalization shown in the figure for the digital image (color map decomposition RGB) with gray level in the range [0, L-1].
The core code of the histogram construction (taking the grayscale image as an example) is as follows:
def histogram(Image): # 灰度图像直方图统计
im = Image
a = [0] * 256 # 构造储存像素数的一维数组
w = im.shape[0] # 得到图像的宽高
h = im.shape[1]
for i in range(w): # 遍历所有像素点统计
for j in range(h):
gray = im[i, j]
a[gray] += 1
y = a # y表示灰度值的像素个数
x = [i for i in range(256)] # x表示0-256的灰度The histograms of Figure 1 (original image) and Figure 2 (original image) are shown in the figure.
2. Histogram equalization
Histogram equalization : An image with a known gray-scale probability density distribution undergoes a gray-scale transformation (the discrete form of the transformation function is shown in the figure ), so that it evolves into an image with a uniform gray-scale probability density distribution (and as far as possible A new image that occupies all the grayscales), thereby increasing the dynamic range of pixel grayscale values to achieve the effect of enhancing the overall contrast of the image.
The implementation steps of histogram equalization are shown in the figure
Five, grayscale image equalization
1. Non-scaling library equalization
According to the definition of histogram and equalization, the histogram construction and equalization of the noise-reduced Figure 1 are carried out. The equalization code of the non-adjusted grayscale image is implemented as follows:
def homogenisation(a, Image): # 均衡化函数：a--灰度像素数，Image--图片
im = array(Image)
b = [0] * 256 # 储存灰度像素所占的比例
c = [0] * 256 # 储存累计分布概率
w = im.shape[0] # 图像的宽和高
h = im.shape[1]
mn = w * h # mn表示像素点总数
for i in range(len(a)): # 计算灰度分布密度
b[i] = a[i] / mn
for i in range(len(c)): # 计算累计分布密度c和累计直方图数据a
if i == 1:
c[i] = b[i]
else:
c[i] = c[i - 1] + b[i]
a[i] = int(255 * c[i])
for i in range(w): # 更新每一个像素点的灰度值
for j in range(h):
im[i, j] = a[Image[i, j]]
return imThe histogram before and after equalization is shown in the figure:
The picture before and after equalization is shown in the figure:
Analysis : According to the figure, after equalization, the histogram grayscale distribution is uniform and basically covers all possible grayscales, which meets the relevant definitions. Observe the picture, the contrast of the picture after equalization is enhanced, and the visible details increase, which meets expectations, and the equalization is successful.
2. Adjustment and balance of storage
The grayscale equalization function in the Opencv (cv2) library in Python is cv2.equalizeHist(img) , which returns the equalized grayscale image .
6. Color image equalization
The essence of the color map is an RGB three-channel map. Different methods can be used for equalization according to different considerations. This experiment uses splitting and merging (reproducing library functions), HSI model<->RGB model interchange, and library function three This method is used as an example to achieve color image equalization.
1. Split and merge equalization
The core principle (similar to the opencv library function): first split the RGB three channels of the color picture, and then equalize each channel by adopting a method similar to gray level equalization, and finally merge the channels to obtain the equalized color picture, the core code as follows:
def my_equalizehist(single):
hist = cv.calcHist([single], [0], None, [256], [0, 256])
num_of_pixels = single.size
ratio = np.zeros(256)
transf_map = np.zeros(256)
result = single.copy()
j = 0
for i in hist:
if j > 0:
ratio[j] = i / num_of_pixels + ratio[j - 1]
else:
ratio[j] = i / num_of_pixels
transf_map[j] = round(ratio[j] * 255)
j = j + 1
for i in range(single.shape[0]):
for j in range(single.shape[1]):
result[i][j] = transf_map[single[i][j]]
# result[result==j]=k[j]
return resultThe pictures before and after equalization are as follows:
2. Library function equalization
The color image equalization function in the Opencv (cv2) library in Python is cv.equalizeHist(img) , which can be used to equalize each channel and use cv.merge to merge. The specific usage is as follows:
hist_res=cv.calcHist([res],[0],None,[256],[0,256])
plt.plot(hist_res);
plt.show()
EB = cv.equalizeHist(B)
EG = cv.equalizeHist(G)
ER = cv.equalizeHist(R)
equal_test = cv.merge((EB, EG, ER)) # merge it back3. HSI model<->RGB model exchange and equalization
The HSI color model is a highly abstract mathematical model that meets the needs of computer digital color management. The HSI model starts from the human visual system and directly uses the three elements of color - hue (Hue ), saturation (Saturation ) and brightness (Intensity , sometimes translated as density or grayscale) to describe the color, RBG is converted to the HIS space formula As shown in the figure, the formula for converting HSI to RGB is shown in the figure.
code show as below:
import cv2
import numpy as np
import matplotlib.pyplot as plt
import collections
def rgb_hsi(rgb_image):
# 保存原始图像的行列数
rows = int(rgb_image.shape[0])
cols = int(rgb_image.shape[1])
# 图像复制
hsi_image = rgb_image.copy()
# 通道拆分
b = rgb_image[:, :, 0]
g = rgb_image[:, :, 1]
r = rgb_image[:, :, 2]
# 归一化到[0,1]
b = b / 255.0
g = g / 255.0
r = r / 255.0
for i in range(rows):
for j in range(cols):
num = 0.5 * ((r[i, j]-g[i, j])+(r[i, j]-b[i, j]))
den = np.sqrt((r[i, j]-g[i, j])**2+(r[i, j]-b[i, j])*(g[i, j]-b[i, j]))
if den == 0:
hsi_h = 0
else:
theta = float(np.arccos(num / den))
if b[i, j] <= g[i, j]:
hsi_h = theta
else:
hsi_h = 2*np.pi - theta
min_RGB = min(min(b[i, j], g[i, j]), r[i, j])
sum = b[i, j]+g[i, j]+r[i, j]
if sum == 0:
hsi_s = 0
else:
hsi_s = 1 - 3*min_RGB/sum
hsi_h = hsi_h/(2*np.pi)
hsi_i = sum/3.0
# 输出HSI图像，扩充到255以方便显示，一般H分量在[0,2pi]之间，S和I在[0,1]之间
hsi_image[i, j, 0] = hsi_h*255
hsi_image[i, j, 1] = hsi_s*255
hsi_image[i, j, 2] = hsi_i*255
return hsi_image
def hsi_rgb(hsi_image):
# 保存原始图像的行列数
rows = np.shape(hsi_image)[0]
cols = np.shape(hsi_image)[1]
# 对原始图像进行复制
rgb_image = hsi_image.copy()
# 对图像进行通道拆分
hsi_h = hsi_image[:, :, 0]
hsi_s = hsi_image[:, :, 1]
hsi_i = hsi_image[:, :, 2]
# 把通道归一化到[0,1]
hsi_h = hsi_h / 255.0
hsi_s = hsi_s / 255.0
hsi_i = hsi_i / 255.0
B, G, R = hsi_h, hsi_s, hsi_i
for i in range(rows):
for j in range(cols):
hsi_h[i, j] *= 360
if 0 <= hsi_h[i, j] < 120:
B = hsi_i[i, j] * (1 - hsi_s[i, j])
R = hsi_i[i, j] * (1 + (hsi_s[i, j] * np.cos(hsi_h[i, j] * np.pi / 180)) / np.cos(
(60 - hsi_h[i, j]) * np.pi / 180))
G = 3 * hsi_i[i, j] - (R + B)
elif 120 <= hsi_h[i, j] < 240:
hsi_h[i, j] = hsi_h[i, j] - 120
R = hsi_i[i, j] * (1 - hsi_s[i, j])
G = hsi_i[i, j] * (1 + (hsi_s[i, j] * np.cos(hsi_h[i, j] * np.pi / 180)) / np.cos(
(60 - hsi_h[i, j]) * np.pi / 180))
B = 3 * hsi_i[i, j] - (R + G)
elif 240 <= hsi_h[i, j] <= 300:
hsi_h[i, j] = hsi_h[i, j] - 240
G = hsi_i[i, j] * (1 - hsi_s[i, j])
B = hsi_i[i, j] * (1 + (hsi_s[i, j] * np.cos(hsi_h[i, j] * np.pi / 180)) / np.cos(
(60 - hsi_h[i, j]) * np.pi / 180))
R = 3 * hsi_i[i, j] - (G + B)
rgb_image[i, j, 0] = B * 255
rgb_image[i, j, 1] = G * 255
rgb_image[i, j, 2] = R * 255
return rgb_image
# 计算灰度图的直方图
def draw_histogram(grayscale):
# 对图像进行通道拆分
hsi_i = grayscale[:, :, 2]
color_key = []
color_count = []
color_result = []
histogram_color = list(hsi_i.ravel()) # 将多维数组转换成一维数组
color = dict(collections.Counter(histogram_color)) # 统计图像中每个亮度级出现的次数
color = sorted(color.items(), key=lambda item: item[0]) # 根据亮度级大小排序
for element in color:
key = list(element)[0]
count = list(element)[1]
color_key.append(key)
color_count.append(count)
for i in range(0, 256):
if i in color_key:
num = color_key.index(i)
color_result.append(color_count[num])
else:
color_result.append(0)
color_result = np.array(color_result)
return color_result
def histogram_equalization(histogram_e, lut_e, image_e):
sum_temp = 0
cf = []
for i in histogram_e:
sum_temp += i
cf.append(sum_temp)
for i, v in enumerate(lut_e):
lut_e[i] = int(255.0 * (cf[i] / sum_temp) + 0.5)
equalization_result = lut_e[image_e]
return equalization_result
x = []
for i in range(0, 256): # 横坐标
x.append(i)
# 原图及其直方图
rgb_image = cv2.imread("lena.jpg")
cv2.imshow('rgb', rgb_image)
histogram = draw_histogram(rgb_image)
plt.bar(x, histogram) # 绘制原图直方图
plt.savefig('before_histogram.png')
plt.show()
# rgb转hsi
hsi_image = rgb_hsi(rgb_image)
cv2.imshow('hsi_image', hsi_image)
cv2.imwrite('hsi_result.png', hsi_image)
# hsi在亮度分量上均衡化
histogram_1 = draw_histogram(hsi_image)
lut = np.zeros(256, dtype=hsi_image.dtype) # 创建空的查找表
result = histogram_equalization(histogram_1, lut, hsi_image) # 均衡化处理
cv2.imshow('his_color_image', result)
cv2.imwrite('his_color.png', result) # 保存均衡化后图片
# hsi转rgb
image_equ = cv2.imread('his_color.png') # 读取图像
rgb_result = hsi_rgb(image_equ)
cv2.imshow('rgb_image', rgb_result)
cv2.imwrite('gbr_result.png', rgb_result)
rgb = cv2.imread("gbr_result.png")
histogram_2 = draw_histogram(rgb)
plt.bar(x, histogram_2)
plt.savefig('after_histogram.png')
plt.show()
plt.plot(x, lut) # 绘制灰度级变换曲线图
plt.savefig('Grayscale_transformation_curve.png')
plt.show()
cv2.waitKey(0)
cv2.destroyAllWindows()
Seven, image enhancement
Image enhancement is an image processing technology that makes images more suitable for specific applications . It is mainly divided into spatial domain image enhancement and frequency domain image enhancement. The framework of common image enhancement methods is shown in the figure. In addition to histogram equalization, the introduction and principle of several common image enhancement algorithms are as follows:
( 1 ) Logarithmic transformation (non-linear transformation) : use the formula s= c log (1+r) to transform the grayscale, so that the narrow-band low-grayscale input image value is mapped to a wideband output value, as shown in Figure 35 before and after the grayscale change , an example of image enhancement (Fourier spectrum) is shown in Figure 36.
Figure 35 before and after logarithmic transformation grayscale change
Figure 36 Logarithmic transformation image enhancement example (picture source ppt)
(2 ) Subtraction operation (algebraic operation) : The image subtraction operation belongs to the algebraic operation, that is, the operation of point-to-point addition, subtraction, multiplication, and division of two input images to obtain the output image. The subtraction operation is to obtain the image through the formula C(x,y) = f(x,y) - h(x,y) , which is often used to remove background noise (slowly changing additive non-random noise), image enhancement samples are shown in the figure 37.
Figure 37 Subtractive image enhancement example (picture source ppt)
(3 ) Smoothing filter : The mean and median filters used in this experiment are all smoothing filters. The principle can be seen in the experimental steps and process. The core is to replace the pixel with some value in the pixel field .
(4 ) Sharpening filter : Contains Laplace operator ( differential operator, application emphasizes the sudden change of gray level in the image, and does not emphasize the area of slowly changing gray level ) and Sobel operator ( mainly used in industrial detection images edge detection ), the operator adopts a method similar to convolution when running, moves the template on the image and calculates the gradient value corresponding to the central pixel at each position, that is, the result obtained by calculating the gradient for a grayscale image is a A gradient map, the definition of the Sobel operator is shown in Figure 38 , and Figure 39 before and after sharpening .
Figure 38 Sobel operator definition
Figure 39 Sample images before and after sharpening (image source network)
(5 ) Frequency domain filtering : Frequency domain filtering actually performs Fourier transform on the image, then processes it in the transform domain, and then performs inverse Fourier transform to convert back to the space domain . The principle is that the function characteristics expressed by Fourier transform can be completely reconstructed by inverse Fourier transform without losing any information (because any periodic or non-periodic function can be expressed as the sum of sine and cosine functions of different frequencies form).
High-pass filtering method: the definition of the ideal high-pass filter ( IHPF ) is shown in Figure 40 , and the samples before and after filtering are shown in Figure 41 .
Figure 40 Ideal High Pass Filter ( IHPF ) Definition
Figure 41 Before and after image processing by high-pass filtering
Tips: In addition to the framework of Figure 34 and the methods mentioned above, there are some classic algorithms for image enhancement, such as grayscale world algorithm, automatic white balance, automatic color balance, etc., and a large number of deep learning algorithms have been used for image enhancement in recent years and image processing.
8. References
5. Image Processing (1): Image Enhancement_QtHalcon's Blog-CSDN Blog_Image Enhancement
6. What are the main methods of image enhancement?
7. [Summary] Summary of common image enhancement methods (attached with implementation code)
9. python+OpenCV color histogram equalization - short book (jianshu.com)
Summarize
This experiment is the first experiment of computer vision. Through this experiment, I am familiar with the image representation and basic elements and channel operations; master the basic image addition/denoising method and practice it manually; understand and apply grayscale The image transformation method realizes the histogram construction and equalization of the grayscale/color image, and compares and analyzes the effect of the custom method and the function method of the OpenCV computer vision library