Image processing --- inverse filtering and Wiener filtering

---

foreword

This paper mainly introduces two methods of degraded image restoration: inverse filtering and Wiener filtering.

---

1. Inverse filtering

The expression of image degradation:
$$\begin{array}{r}g(x,y)=h(x,y)\odot f(x,y)+n(x,y)\end{array}$$
$f(x,y):$ input image
$h(x,y):$ Degenerate function
$n(x,y):$ noise term
$g:$ Degraded image

Converted to the frequency domain by the convolution theorem:
$$\begin{array}{r}G(u,v)=H(u,v)\ast F(u,v)+N(u,v)\end{array}$$

The inverse filtering operation is to remove the degenerate function:
$$\begin{array}{r}\hat{F}=\frac{G(u,v)}{H(u,v)}=F(u,v)+\frac{N(u,v)}{H(u,v)}\end{array}$$

According to the above formula, there are two key problems in inverse filtering:

1. Degenerate function $H(u,The\; estimate\; of\; v)$ is indeed accurate, the clearer the recovery.
2. When the degradation function $H(u,When\; v)$ tends to 0, the value of the latter term becomes too large, and the final formula is affected by the noise term$N(u,v)$ the influence will increase.

1.1 Estimated degradation function $H(u,v)$

There are mainly 3 methods:
$\left(1\right)$ : observation method;$\left(2\right)$ : test method;$\left(3\right)$ : Mathematical modeling methods
These methods are all processed in the frequency domain

1.1.1 Observation method

Find a region with strong signal content ( high-contrast region ), thereby ignoring the influence of the noise term, i.e.:
$$\begin{array}{r}{H}_s(u,v)=\frac{G_s(u,v)}{\widehat{F_s}(u,v)}\end{array}$$
$G_s(u,v):$ The Fourier transform of the image currently to be restored.
$\widehat{F_s}(u,v):$ The restored image after processing (sharpening, mean filtering, etc.).

1.1.2 Test method

In short, it is to do experiments to simulate the real degradation process, and then obtain the degradation function according to the following formula.
$$\begin{array}{r}H(u,v)=\frac{G(u,v)}{A}\end{array}$$
Assume that there is a certain device that can obtain an image very similar to the degraded image, and then image an impulse (small light spot, the small light spot should be bright enough to reduce the influence of noise), and obtain the FFT after the impulse degeneration (corresponding to the above G $G(u,v)$ ), while the FFT of an impulse is a constant$A$。

1.1.3 Modeling method $★$

$\left(1\right)$ Atmospheric turbulence model
$$\begin{array}{r}H(u,v)=e^{-k(u^2+v^2{)}^{5/6}}\end{array}$$
k is the turbulence constant, which reflects the intensity of turbulence.
$$k=\{\begin{array}{l}0.00025slight\; turbulence\\ 0.001moderate\; turbulence\\ 0.0025severe\end{array}$$
code:

import cv2
import numpy as np
from matplotlib import pyplot as plt

# 读取图像
img = cv2.imread('A.png',0)

# 进行傅里叶变换
f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)

#-------------大气湍流模型-----------------
k = 0.001  #湍流程度
degeneration_img = np.zeros(fshift.shape,dtype=complex)   #一定要指定类型
for u in range(fshift.shape[0]):
    for v in range(fshift.shape[1]):
        H = np.exp(-k*np.power(np.float64((u-fshift.shape[0]//2)**2+(v-fshift.shape[1]//2)**2),5/6))
        degeneration_img[u][v] = fshift[u][v] * H
#-----------------------------------------

# 将频域图像在空域进行显示做的处理
img1  = np.fft.ifftshift(degeneration_img)
img1  = np.fft.ifft2(img1)
img1 = np.abs(img1)
img1 = cv2.normalize(img1, None, 0, 255, cv2.NORM_MINMAX)


# 显示原始图像和退化的图像
plt.figure(figsize=(20, 20))
plt.subplot(121), plt.imshow(img, cmap='gray')
plt.title('Input Image'), plt.xticks([]), plt.yticks([])
plt.subplot(122), plt.imshow(img1, cmap='gray')
plt.title('degeneration img'), plt.xticks([]), plt.yticks([])
plt.show()

Note: When degeneration_imginitializing the object, be sure to specify the type dtype=complex. In the process of writing, I forgot to specify the type, resulting in a wrong degraded image (you can try it). Generally speaking, it is not a big problem not to specify the type, but since this is a plural type , if the type is not specified, the imaginary part will be automatically discarded during the degeneration_img[u][v] = fshift[u][v] * Hassignment ! Refer to the picture below:

I didn't know the pitfalls of dynamic languages ​​before, but now I have fallen into the trap once, and I must add type annotations when writing code in Python in the future ! !

另外，也可以用矩阵的方法生成：（下面代码参考文章：https://blog.csdn.net/youcans/article/details/123027287）

def turbulenceBlur(img, k=0.001):  # 湍流模糊传递函数
        # H(u,v) = exp(-k(u^2+v^2)^5/6)
        M, N = img.shape[1], img.shape[0]
        u, v = np.meshgrid(np.arange(M), np.arange(N))
        radius = (u - M//2)**2 + (v - N//2)**2
        kernel = np.exp(-k * np.power(radius, 5/6))
        return kernel

$\left(2\right)$ Motion blur model
When the image is atxxThe speed in$the\; x$$x_0(t)=at/T$ uniform linear motion, at the same time inyyThe speed in$the\; y$$y_0\left(t\right)=bt/T$ by free definition
$$\begin{array}{r}H(u,v)=\frac{T}{\pi (ua+vb)}sin\left[\pi \right(etal+vb)]e^{-j\pi (ua+vb}\end{array}$$

code:

# 功能：实现运动模糊的效果
import cv2
import numpy as np
from matplotlib import pyplot as plt

# 读取图像
img = cv2.imread('0526.tif',0)

# 进行傅里叶变换
f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)

#-------------运动模糊模型-----------------
a = b = 0.1
T = 1
degeneration_img = np.zeros(fshift.shape,dtype=np.complex64)   #一定要指定类型
for u in range(fshift.shape[0]):
    for v in range(fshift.shape[1]):
        uv = ((u-fshift.shape[0]//2)*a + (v-fshift.shape[1]//2)*b) * np.pi
        if uv == 0:
            degeneration_img[u][v] = fshift[u][v]
        else:
            H = T * np.sin(uv) * np.exp(np.complex64(-1j)*uv) /uv
            degeneration_img[u][v] = fshift[u][v] * H
#-----------------------------------------

# 将频域图像在空域进行显示做的处理
img1  = np.fft.ifftshift(degeneration_img)
img1  = np.fft.ifft2(img1)
img1 = np.abs(img1)
img1 = cv2.normalize(img1, None, 0, 1, cv2.NORM_MINMAX)


# 显示原始图像和退化的图像
plt.figure(figsize=(20, 20))
plt.subplot(121), plt.imshow(img, cmap='gray')
plt.title('Input Image'), plt.xticks([]), plt.yticks([])
plt.subplot(122), plt.imshow(img1, cmap='gray')
plt.title('degeneration img'), plt.xticks([]), plt.yticks([])
plt.show()

$(3)$高斯模糊模型
$$\begin{array}{r}H(u,v)=e^{-D^2(u,v)/2D_0^2}\\ D(u,v)=[(u-P/2{)}^2+(v-Q/2{)}^2{]}^{1/2}\end{array}$$
code:

import cv2
import numpy as np

# 读取输入图像
img = cv2.imread('A.png', 0)

# 进行傅里叶变换
f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)

# 构造高斯滤波器
rows, cols = img.shape
crow, ccol = rows//2, cols//2
d = 60  # 滤波器大小
gauss = np.zeros((rows, cols))
for i in range(rows):
    for j in range(cols):
        gauss[i, j] = np.exp(-((i-crow)**2+(j-ccol)**2)/(2*d**2))

# 将高斯滤波器应用于频域图像
filtered = np.multiply(fshift, gauss)

# 进行反傅里叶变换
f_ishift = np.fft.ifftshift(filtered)
img_back = np.fft.ifft2(f_ishift)
img_back = np.abs(img_back)
img_back = cv2.normalize(img_back, None, 0, 1, cv2.NORM_MINMAX)

# 显示结果
cv2.imshow('Input', img)
cv2.imshow('Output', img_back)
cv2.waitKey(0)
cv2.destroyAllWindows()

1.2 Direct inverse filtering

Disregarding noise:
$$\begin{array}{r}\hat{F}=\frac{G(u,v)}{H(u,v)}\end{array}$$
【noise1.png】:The image after Gaussian blur and Gaussian noise.
Without any prior knowledge, the degradation function of the image is estimated using the atmospheric turbulence model, and then the inverse filtering process is performed.
code show as below:

import cv2
import numpy as np
from matplotlib import pyplot as plt

# 读取图像
img = cv2.imread('noise1.png',0)

# 进行傅里叶变换
f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)

#-------------利用大气湍流模型估计退化传递函数-----------------
k = 0.001  #湍流程度
degeneration = np.zeros(fshift.shape,dtype=complex)   
for u in range(fshift.shape[0]):
    for v in range(fshift.shape[1]):
        H = np.exp(-k*np.power(np.float64((u+fshift.shape[0]//2)**2+(v-fshift.shape[1]//2)**2),5/6))
        degeneration[u][v] =  H
#-----------------------------------------


# 逆滤波
img1 = np.divide(fshift,degeneration)

# 将频域图像在空域进行显示做的处理
img1  = np.fft.ifftshift(img1)
img1  = np.fft.ifft2(img1)
img1 = np.abs(img1)
img1 = cv2.normalize(img1, None, 0, 255, cv2.NORM_MINMAX)


# 显示原始图像和恢复后的图像
plt.figure(figsize=(20, 20))
plt.subplot(121), plt.imshow(img, cmap='gray')
plt.title('Input Image'), plt.xticks([]), plt.yticks([])
plt.subplot(122), plt.imshow(img1, cmap='gray')
plt.title('rec img'), plt.xticks([]), plt.yticks([])
plt.show()

Filtering effect:

However, you only need to change the above statement degeneration[u][v] = H $\to$degeneration[u][v] = H + 0.001 , using 0.001 as an estimate of the noise, a certain effect is obtained:

1.3 Radius limited inverse filtering

考虑噪声：
$$\begin{array}{r}\hat{F}=F(u,v)+\frac{N(u,v)}{H(u,v)}\end{array}$$

Use a low-pass filter to filter out high-frequency noise and then divide by H(u, v)

code:

import cv2
import numpy as np
from matplotlib import pyplot as plt

# 读取图像
img = cv2.imread('noise3.png',0)

# 进行傅里叶变换
f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)

#-------------大气湍流模型-----------------
k = 0.001  #湍流程度
degeneration = np.zeros(fshift.shape,dtype=complex)   #要指定类型
for u in range(fshift.shape[0]):
    for v in range(fshift.shape[1]):
        H = np.exp(-k*np.power(np.float64((u+fshift.shape[0]//2)**2+(v-fshift.shape[1]//2)**2),5/6))
        degeneration[u][v] =  H
#-----------------------------------------


# 构建低通滤波器---理想低通
# rows,cols = fshift.shape
# radius = 1
# mask = np.zeros((rows, cols),dtype=np.float64)
# for i in range(rows):
#     for j in range(cols):
#         if np.sqrt((i-rows//2)**2 + (j-cols//2)**2) <= radius:
#             mask[i, j] = 0.1
# fshift = np.multiply(fshift, mask)

# # 构建高斯低通滤波器
# rows, cols = img.shape
# crow, ccol = rows//2, cols//2
# d = 50  # 滤波器大小
# gauss = np.zeros((rows, cols))
# for i in range(rows):
#     for j in range(cols):
#         gauss[i, j] = np.exp(-((i-crow)**2+(j-ccol)**2)/(2*d**2))

# 构建巴特沃斯低通滤波器
rows, cols = img.shape
crow, ccol = rows//2, cols//2
d = 70  # 滤波器大小
butter = np.zeros((rows, cols))
for i in range(rows):
    for j in range(cols):
        butter[i, j] = 1/(1 + np.power((np.power((i-crow)**2+(j-ccol)**2,0.5)/d),20))

fshift = np.multiply(fshift, butter)
# 逆滤波
img1 = np.divide(fshift,butter)

# 将频域图像在空域进行显示做的处理
img1  = np.fft.ifftshift(img1)
img1  = np.fft.ifft2(img1)
img1 = np.abs(img1)
img1 = cv2.normalize(img1, None, 0, 1, cv2.NORM_MINMAX)


# 显示原始图像和恢复后的图像
plt.figure(figsize=(20, 20))
plt.subplot(121), plt.imshow(img, cmap='gray')
plt.title('Input Image'), plt.xticks([]), plt.yticks([])
plt.subplot(122), plt.imshow(img1, cmap='gray')
plt.title('rec img'), plt.xticks([]), plt.yticks([])
plt.show()

cv2.imshow('org',img)
cv2.imshow('rev',img1)
cv2.waitKey(0)

Filter effect:

Second, the minimum mean square error (Wiener) filter

The estimated Fourier transform of the degraded image is:
$$\begin{array}{r}\widehat{F(u,v)}=[\frac{1}{H(u,v)}\frac{|H(u,v){|}^2}{|H(u,v){|}^2+K}]G(u,v)\end{array}$$

import cv2
import numpy as np
from matplotlib import pyplot as plt

# 读取图像
img = cv2.imread('noise2.png',0)

# 进行傅里叶变换
f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)

#-------------大气湍流模型-----------------
k = 0.001  #湍流程度
degeneration = np.zeros(fshift.shape,dtype=complex)   #要指定类型
for u in range(fshift.shape[0]):
    for v in range(fshift.shape[1]):
        H = np.exp(-k*np.power(np.float64((u+fshift.shape[0]//2)**2+(v-fshift.shape[1]//2)**2),5/6))
        degeneration[u][v] =  H
#-----------------------------------------


# 维纳滤波
K = 0.001
degeneration += 0.1   #加上估计的噪声
img1 = (np.conj(degeneration)/(np.conj(degeneration)*degeneration + K)) * fshift

# 将频域图像在空域进行显示做的处理
img1  = np.fft.ifftshift(img1)
img1  = np.fft.ifft2(img1)
img1 = np.abs(img1)
img1 = cv2.normalize(img1, None, 0, 1, cv2.NORM_MINMAX)


# 显示原始图像和恢复后的图像
plt.figure(figsize=(20, 20))
plt.subplot(121), plt.imshow(img, cmap='gray')
plt.title('Input Image'), plt.xticks([]), plt.yticks([])
plt.subplot(122), plt.imshow(img1, cmap='gray')
plt.title('rec img'), plt.xticks([]), plt.yticks([])
plt.show()

Filter effect:

Summarize

Two methods of Wiener filtering and inverse filtering are mainly shown, but the filtering effect is not obvious, which may be caused by the inconsistency between the estimated degradation function and the real degradation function.

references

[1] Ruan Qiuqi, translated by Ruan Yuzhi; (US) Raphael C. Gonzalez, Richard E. Woods. Digital Image Processing 4th Edition of Foreign E-books and Communication Textbook Series [M]. Beijing: Electronic Industry Press, 2020