[opencv] Fourier transform (analyze the frequency characteristics of various filters, high-pass filtering is an edge detection operation, delete high-frequency blurred images)

4_11_Fourier transform - OpenCV Chinese official documentation

Finding the Fourier Transform of an Image using OpenCV - Take advantage of the FFT functions available in Numpy - Some applications of Fourier Transform - We will see the following functions: cv.dft (), cv.idft () etc.

Pre-feed

Fourier transform is used to analyze the frequency characteristics of various filters.

For images, use the **2D Discrete Fourier Transform** (DFT) to find the frequency domain .

A fast algorithm called the Fast Fourier Transform (FFT) is used for the calculation of the DFT .

More intuitively, for a sinusoidal signal, if the amplitude changes so fast in a short period of time, it can be said to be a high frequency signal. If the change is slow, it is a low frequency signal.

You can extend the same idea to images. Where does the amplitude in the image change drastically? at edge points or noise. Therefore, it can be said that edges and noise are high frequency content in an image. If the amplitude doesn't change much, it's a low frequency component.

High pass filtering is an edge detection operation

Do the inverse transform of the DFT . In the previous section, we created an HPF, this time we will see how to remove high frequency content from the image, i.e. we apply the LPF to the image. It actually blurs the image . To do this, we first create a high value (1) in the low frequency part, i.e. we filter the low frequency content and 0 in the high frequency region.

Personal understanding: When learning opencv, you can often see that the same function can be implemented by Numpy and OpenCV. Generally, the two methods have performance differences. If you consider the magnitude of the project, you may need to pay attention to this problem.

Laplacian is a high pass filter

content

theory

Fourier Transform in Numpy

Fourier Transform in OpenCV

Performance optimization of DFT

Why is the Laplacian a high pass filter?

theory

Fourier transform is used to analyze the frequency characteristics of various filters.

For images, use the **2D Discrete Fourier Transform** (DFT) to find the frequency domain .

A fast algorithm called the Fast Fourier Transform (FFT) is used for the calculation of the DFT .

For a sinusoidal signal , we can say that f is the frequency of the signal, and if we take its frequency domain, we can see the spikes in f. If we sample the signal to form discrete signals, we get the same frequency domain, but in the range [−π,π] or [0,2π] ([0,N][0,N] for N-point DFT ) is periodic. You can think of an image as a signal sampled in both directions. Therefore, Fourier transform is performed in both the X and Y directions, and the frequency representation of the image can be obtained.

More intuitively, for a sinusoidal signal, if the amplitude changes so fast in a short period of time, it can be said to be a high frequency signal. If the change is slow, it is a low frequency signal.

You can extend the same idea to images. Where does the amplitude in the image change drastically? at edge points or noise. Therefore, it can be said that edges and noise are high frequency content in an image. If the amplitude doesn't change much, it's a low frequency component.

Now, we will see how to find the Fourier transform.

Fourier Transform in Numpy

First, we'll see how to use Numpy to find the Fourier transform. Numpy has the FFT package to do this. np.fft.fft2 () gives us the frequency transform, it will be a complex array.

The first parameter is the input image, which is a grayscale image.

The second parameter is optional and determines the size of the output array. If it is larger than the size of the input image, then pad the input image with zeros before computing the FFT. If smaller than the input image, the input image will be cropped. If no arguments are passed, the output array will be the same size as the input.

Now, once the result is obtained, the zero frequency component (the DC component) will be in the upper left corner. If you want to center it, you need to shift the result N2N2 in both directions. Just pass the function **np.fft.fftshift**() to do it. (it's easier to analyze). Once the frequency transform is found, the magnitude spectrum can be found.
import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
img = cv.imread('messi5.jpg',0)
f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)
magnitude_spectrum = 20*np.log(np.abs(fshift))
plt.subplot(121),plt.imshow(img, cmap = 'gray')
plt.title('Input Image'), plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(magnitude_spectrum, cmap = 'gray')
plt.title('Magnitude Spectrum'), plt.xticks([]), plt.yticks([])
plt.show()
Result look like below:
The result looks like this:

Look, you can see more white areas in the center, which indicates more low frequency content.

So you found the frequency transform Now you can do some operations in the frequency domain like high pass filtering and reconstructing the image i.e. find the inverse DFT . To do this, you simply mask out the low frequencies with a rectangular window of size 60x60 . Then, use **np.fft.ifftshift**() to apply an inverse shift so that the DC component appears in the upper left corner again. Then use the **np.ifft2**() function to find the inverse FFT. Again, the result will be a complex number. You can take its absolute value.
rows, cols = img.shape
crow,ccol = rows//2 , cols//2
fshift[crow-30:crow+31, ccol-30:ccol+31] = 0
f_ishift = np.fft.ifftshift(fshift)
img_back = np.fft.ifft2(f_ishift)
img_back = np.real(img_back)
plt.subplot(131),plt.imshow(img, cmap = 'gray')
plt.title('Input Image'), plt.xticks([]), plt.yticks([])
plt.subplot(132),plt.imshow(img_back, cmap = 'gray')
plt.title('Image after HPF'), plt.xticks([]), plt.yticks([])
plt.subplot(133),plt.imshow(img_back)
plt.title('Result in JET'), plt.xticks([]), plt.yticks([])
plt.show()
The result looks like this:

It turns out that high-pass filtering is an edge detection operation . This is what we saw in the "Image Gradients" chapter. This also shows that most of the image data exists in the low frequency region of the spectrum. Anyway, we've seen how to find DFT, IDFT, etc. in Numpy. Now, let's see how to do it in OpenCV.

Look closely at the results, especially the last JET color image, and you'll see some artifacts (an instance marked with a red arrow). It shows some corrugated structure there, called **ringing** .

This is caused by the rectangular window we use for the mask . This mask is converted to a sine shape, causing this problem.

Therefore, rectangular windows are not used for filtering. A better option is a Gaussian window .

Fourier Transform in OpenCV

OpenCV provides **cv.dft**() and **cv.idft**() functions for this. It returns the same result as the previous one, but with two channels. The first channel is the real part of the result and the second channel is the imaginary part of the result. The input image should first be converted to np.float32. Let's see how.
import numpy as np
import cv2 as cv
from matplotlib import pyplot as plt
img = cv.imread('messi5.jpg',0)
dft = cv.dft(np.float32(img),flags = cv.DFT_COMPLEX_OUTPUT)
dft_shift = np.fft.fftshift(dft)
magnitude_spectrum = 20*np.log(cv.magnitude(dft_shift[:,:,0],dft_shift[:,:,1]))
plt.subplot(121),plt.imshow(img, cmap = 'gray')
plt.title('Input Image'), plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(magnitude_spectrum, cmap = 'gray')
plt.title('Magnitude Spectrum'), plt.xticks([]), plt.yticks([])
plt.show()
Note you can also use **cv.cartToPolar**() which returns both magnitude and phase in a single shot

Now we do the inverse transform of the DFT . In the previous section, we created an HPF, this time we will see how to remove high frequency content from the image, i.e. we apply the LPF to the image.

It actually blurs the image . To do this, we first create a high value (1) in the low frequency part, i.e. we filter the low frequency content and 0 in the high frequency region.
rows, cols = img.shape
crow,ccol = rows/2 , cols/2
# 首先创建一个掩码，中心正方形为1，其余全为零
mask = np.zeros((rows,cols,2),np.uint8)
mask[crow-30:crow+30, ccol-30:ccol+30] = 1
# 应用掩码和逆DFT
fshift = dft_shift*mask
f_ishift = np.fft.ifftshift(fshift)
img_back = cv.idft(f_ishift)
img_back = cv.magnitude(img_back[:,:,0],img_back[:,:,1])
plt.subplot(121),plt.imshow(img, cmap = 'gray')
plt.title('Input Image'), plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(img_back, cmap = 'gray')
plt.title('Magnitude Spectrum'), plt.xticks([]), plt.yticks([])
plt.show()
Check out the results:

Note that in general, OpenCV functions **cv.dft**() and **cv.idft**() are faster than Numpy functions. But Numpy functions are easier to use. See the section below for more details on performance issues.

Performance optimization of DFT

For some array sizes, DFT performs better in computation. Fastest when the array size is a power of 2. Arrays of sizes 2, 3, and 5 products can also be processed very efficiently. So if you are concerned about the performance of your code, you can modify the size of the array to any optimal size (by padding with zeros) before finding the DFT. For OpenCV, you have to pad with zeros manually. But with Numpy, you specify the new size of the FFT calculation and it will automatically pad zeros for you.

So how to find the optimal size? OpenCV provides a function for this, cv.getOptimalDFTSize (). It works for both **cv.dft**() and **np.fft.fft2**(). Let's check their performance using the IPython magic command timeit.
In [16]: img = cv.imread('messi5.jpg',0)
In [17]: rows,cols = img.shape
In [18]: print("{} {}".format(rows,cols))
342 548
In [19]: nrows = cv.getOptimalDFTSize(rows)
In [20]: ncols = cv.getOptimalDFTSize(cols)
In [21]: print("{} {}".format(nrows,ncols))
360 576
See, (342,548)modify size to (360，576). Now let's pad it with zeros (for OpenCV) and find its DFT computation performance. You can do this by creating a new zero array and copying the data into it, or using **cv.copyMakeBorder**().
nimg = np.zeros((nrows,ncols))
nimg[:rows,:cols] = img
or:
right = ncols - cols
bottom = nrows - rows
bordertype = cv.BORDER_CONSTANT ＃只是为了避免PDF文件中的行中断
nimg = cv.copyMakeBorder(img,0,bottom,0,right,bordertype, value = 0)
Now, we compute the DFT performance comparison of the Numpy function:
In [22]: %timeit fft1 = np.fft.fft2(img)
10 loops, best of 3: 40.9 ms per loop
In [23]: %timeit fft2 = np.fft.fft2(img,[nrows,ncols])
100 loops, best of 3: 10.4 ms per loop
It shows a 4x speedup. Now, we will try to use OpenCV functions.
In [24]: %timeit dft1= cv.dft(np.float32(img),flags=cv.DFT_COMPLEX_OUTPUT)
100 loops, best of 3: 13.5 ms per loop
In [27]: %timeit dft2= cv.dft(np.float32(nimg),flags=cv.DFT_COMPLEX_OUTPUT)
100 loops, best of 3: 3.11 ms per loop
It also shows a 4x speedup. You can also see that OpenCV functions are about 3 times faster than Numpy functions. The inverse FFT can also be tested, this is left as an exercise for you.

Why is the Laplacian a high pass filter?

A similar question was asked on a forum. The question is, why is Laplace transform a high pass filter? Why is Sobel an HPF? etc. The first answer is about the Fourier transform. For larger FFTs only the Laplace transform is required. Analyze the following code:
import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
# 没有缩放参数的简单均值滤波器
mean_filter = np.ones((3,3))
# 创建高斯滤波器
x = cv.getGaussianKernel(5,10)
gaussian = x*x.T
# 不同的边缘检测滤波器
# x方向上的scharr
scharr = np.array([[-3, 0, 3],
                   [-10,0,10],
                   [-3, 0, 3]])
# x方向上的sobel
sobel_x= np.array([[-1, 0, 1],
                   [-2, 0, 2],
                   [-1, 0, 1]])
# y方向上的sobel
sobel_y= np.array([[-1,-2,-1],
                   [0, 0, 0],
                   [1, 2, 1]])
# 拉普拉斯变换
laplacian=np.array([[0, 1, 0],
                    [1,-4, 1],
                    [0, 1, 0]])
filters = [mean_filter, gaussian, laplacian, sobel_x, sobel_y, scharr]
filter_name = ['mean_filter', 'gaussian','laplacian', 'sobel_x', \
                'sobel_y', 'scharr_x']
fft_filters = [np.fft.fft2(x) for x in filters]
fft_shift = [np.fft.fftshift(y) for y in fft_filters]
mag_spectrum = [np.log(np.abs(z)+1) for z in fft_shift]
for i in xrange(6):
    plt.subplot(2,3,i+1),plt.imshow(mag_spectrum[i],cmap = 'gray')
    plt.title(filter_name[i]), plt.xticks([]), plt.yticks([])
plt.show()
Check out the results:

From the image, you can see the frequency regions that each core blocks and the regions it is allowed to pass through. From this information, we can tell why each core is HPF or LPF

[opencv] Fourier transform (analyze the frequency characteristics of various filters, high-pass filtering is an edge detection operation, delete high-frequency blurred images)

theory

Fourier Transform in Numpy

Fourier Transform in OpenCV

Performance optimization of DFT

Why is the Laplacian a high pass filter?

Guess you like