Chapter 14: The Fourier Transform

Image processing is generally divided into spatial domain processing and frequency domain processing.

Spatial domain:

Spatial domain processing is to directly process the pixels in the image. Spatial domain processing is mainly divided into two forms: grayscale transformation and spatial filtering. Grayscale transformation is to process a single pixel in an image, such as adjusting contrast and processing domain values. Spatial filtering involves quality modification of an image, such as image smoothing. The calculation of spatial domain processing is simple and convenient, and the calculation speed is faster.

Frequency domain:

Frequency domain processing is to transform the image into the frequency domain first, then process the image in the frequency domain, and finally transform the image from the frequency domain to the spatial domain through inverse transformation. The Fourier transform is the most widely used frequency domain transform, which can transform the image from the spatial domain to the frequency domain, and your Fourier transform can transform the frequency domain information into the spatial domain. Fourier transform plays a very important role in the field of image processing.

The following is a brief introduction to the Fourier transform from the perspectives of theoretical basis, basic implementation, and specific applications.

1. Theoretical basis:

The Fourier transform is very abstract. Many people have used the Fourier transform for many years in engineering and have not fully understood what the Fourier transform is all about. In order to better illustrate the Fourier transform, let's look at an example in life.

The following table shows the formula of a certain beverage. The formula is a table expressed in the form of time. The table is very long, and only a part of it is intercepted here. This table records the operations from the time "00:00" to a specific time "00:11".

Analysis of the tables reveals that the formula:

Put a piece of rock sugar every 1 minute.
Put 3 red beans every 2 minutes.
Put 2 mung beans every 3 minutes.
Place 4 tomatoes every 4 minutes.
Put 1 cup of purified water every 5 minutes.

The above text is to explain the recipe from the perspective of operating frequency.

In the process of data processing, information is often expressed in the form of diagrams. From the perspective of time domain, the formula table can be expressed as the following figure.

If expressed from the perspective of frequency (period), this formula table can be expressed as the following figure, the abscissa in the figure is the period (reciprocal of frequency), and the ordinate is the number of ingredients.

For functions, it is also possible to transform it from the time domain to the frequency domain. The figure below is a sinusoid with a frequency of 5 (5 cycles in 1 second) and an amplitude of 1.

If you consider it from the perspective of frequency, it can be drawn as a frequency domain diagram as shown in the figure below. The abscissa in the figure is the frequency, and the ordinate is the amplitude.

The above two diagrams are equivalent, they are two different representations of a function. The corresponding time domain representation can be obtained through the frequency domain representation, and the corresponding frequency domain representation can also be obtained through the time domain representation.

French mathematician Fourier pointed out that any periodic function can be expressed as the sum of sinusoidal functions of different frequencies. In today's view, this theory is taken for granted, but this theory is difficult to understand, but has been greatly questioned.

Let's look at the specific process of Fourier transform. For example, the curve of a periodic function is shown in the upper left corner of the figure below. This periodic function can be expressed as:

y = 3 * np.sin(0.8 * x) + 7 * np.sin(0.5 * x) +2 * np.sin(0.2 * x)

Therefore, the function can be seen as the sum of the following three functions:

y1 = 3 * np.sin(0.8 * x)
y2 = 7 * np.sin(0.5 * x)
y3 = 2 * np.sin(0.2 * x)

The function curves corresponding to the above three functions are shown in the upper right corner, lower left corner, and lower right corner of the figure below.

If considered from the perspective of the frequency domain, the above three sine functions can be expressed as three columns in the figure below. The abscissa in the figure is the frequency, and the ordinate is the amplitude.

From the above analysis, it can be seen that the curve in the upper left corner of the figure can be expressed as the frequency domain figure shown in the figure above.

The process of constructing the frequency domain image shown in the above figure from the time domain function graph in the upper left corner is the Fourier transform.

The time-domain function graph in the upper left corner shows exactly the same information as the frequency-domain graph in the above figure. Fourier transform is to completely express time domain information from the perspective of frequency domain.

In addition to the frequency and amplitude mentioned above, the issue of time should also be considered. For example, beverage formulations require tight control over when ingredients are added in order to control flavor. The operation at the time of "00:00" in the above table, under finer control, is actually shown in the table below.

If the timing of adding the ingredients is changed, the flavor of the drink will change. Therefore, in the actual processing process, the time difference must also be considered. This time difference is in the Fourier transform 相位. The information related to the time difference in the time domain expressed by the phase.

For example, the function

y = 3 * np.sin(0.8 * x) + 7 * np.sin(0.5 * 2 + 2) + 2 * np.sin(0.2 * x + 2)

It can be seen as the sum of the following three functions:

y1 = 3 * np.sin(0.8 * x)
y2 = 7 * np.sin(0.5 * x + 2)
y3 = 2 * np.sin(0.2 * x + 3)

The function curves corresponding to the above four functions are the upper left corner, upper right corner, lower left corner, and lower right corner in the figure below.

In this example, if the abscissa is regarded as the start time, the three sine functions that make up the function y do not all start from time 0, and there is a time difference between them. If the function without time difference is used directly, the function shown in the upper left corner of the above figure cannot be formed, but the periodic function y = 3 * np.sin(0.8 * x) + 7 * np we mentioned earlier will be formed. sin(0.5 * x) + 2 * np. sin(0.2 * x). Therefore, the phase difference is a very important condition in the Fourier transform.

The above uses examples of beverage recipes and functions to introduce the feasibility of time domain and frequency domain conversion to help us understand Fourier transform.

In the process of graphics processing, the Fourier transform is to decompose the image into two parts, the sine component and the cosine component, that is, to convert the image from the spatial domain to the frequency domain. After the digital image is transformed by Fourier, the obtained frequency domain is a complex number. Therefore, to display the results of Fourier transform, it is necessary to use the form of real image plus imaginary image (complex image), or magnitude image (magnitude image) plus phase image (phase image).

Because the magnitude image contains most of the information we need in the original image, only the magnitude image is usually used during image processing. Of course, if it is desired to process the image in the frequency domain, the modified spatial domain image is obtained by inverse Fourier transform. It is necessary to preserve both the magnitude image and the phase image.

After Fourier transforming the image, we will get the low frequency and high frequency information in the image. Low-frequency information corresponds to slowly changing grayscale components in the image. The high-frequency information corresponds to the gray-scale components that change faster and faster in the image, which is caused by the sharp transition of the gray scale . For example, in an image of a prairie, low-frequency information corresponds to details such as vast grasslands with consistent colors, while high-frequency information corresponds to various edge and noise information such as the silhouette of a lion.

The purpose of Fourier Fourier transform plays a very critical role in the field of image processing, which can realize image enhancement, image denoising, edge detection, feature extraction, image compression and encryption, etc.

Second, Numpy implements Fourier transform:

The Numpy module provides the Fourier transform function, and the fft2() function in the Numpy module can realize the Fourier transform of the image. This section introduces how to use the Numpy module to implement the Fourier transform of the image, and filter the low-frequency information of the image in the frequency domain, retain the high-frequency information, and implement high-pass filtering.

1. Implement the Fourier transform:

The Numpy module provides the function numpy.fft.fft2() that implements the Fourier transform. Its syntax format is:

return value = numpy.fft.fft2(original image)

It should be noted here that the type of the parameter "original image" is a grayscale image, and the return value of the function is a complex ndarray.

After processing this function, the spectral information of the image can be obtained. At this time, the zero frequency component in the image spectrum is located in the upper left corner of the spectrum image (frequency domain image). For ease of observation, the numpy.fft.fftshift() function is usually used to move the zero frequency component to the center of the frequency domain image . as the picture shows.

The syntax format of the function numpy.fft.fftshift() is:

return value = numpy.fft.fftshift(raw spectrum)

After processing with this function, the zero-frequency component in the image spectrum will be moved to the center of the frequency-domain image, which is very effective for observing the zero-frequency part in the spectrum after Fourier transform.

After performing Fourier transform on the image, a complex array is obtained. In order to be displayed as an image, their values need to be adjusted to the gray space of [0,255], and the formula used is:

Pixel new value = 20*np.log(np.abs(spectrum value))

Example 1: Implementing the Fourier Transform

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread('../lena.bmp', 0)

f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)
magnitude_spectrum = 20 * np.log(np.abs(fshift))

plt.subplot(1, 2, 1)
plt.imshow(img, cmap='gray')
plt.title('original')
plt.axis('off')

plt.subplot(1, 2, 2)
plt.imshow(magnitude_spectrum, cmap='gray')
plt.title('result')
plt.axis('off')
plt.show()

2. Inverse Fourier transform:

It should be noted that if the numpy.fft.fftshift() function is used to move the zero-frequency component during the Fourier transform process, then the numpy.fft.ifftshift() function needs to be used first in the inverse Fourier transform process Move the zero-frequency component to its original position, and then perform an inverse Fourier transform.

The function numpy.fft.ifftshift() is the inverse function of numpy.fft.fftshift(), and its syntax format is:

adjusted spectrum = numpy.fft.ifftshift(original spectrum)

The numpy.fft.ifft2() function is the inverse of numpy.fft.fft2(). Used to implement the inverse Fourier transform, returning an array of complex numbers in the space domain. The syntax of this function is:

Return value = numpy.fft.ifft2 (frequency domain data)

The return value of the function numpy.fft.ifft2() is still a complex ndarray.

The spatial domain information obtained by the inverse Fourier transform is a complex array, which needs to be adjusted to the [0,255] grayscale space. The formula used is:

iimg=np.abs (result of inverse Fourier transform)

Example 1: Implementing the Inverse Fourier Transform

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread('../boat.512.tiff', 0)
f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)
magnitude_spectrum = 20 * np.log(np.abs(fshift))

ishift = np.fft.ifftshift(fshift)
iimg = np.fft.ifft2(ishift)
iimg = np.abs(iimg)

plt.subplot(131)
plt.imshow(img, cmap='gray')
plt.title('img')
plt.axis('off')

plt.subplot(132)
plt.imshow(magnitude_spectrum, cmap='gray')
plt.title('fft2')
plt.axis('off')

plt.subplot(133)
plt.imshow(iimg, cmap='gray')
plt.title('iimg')
plt.axis('off')
plt.show()

3. Example of high-pass filtering:

In an image, high-frequency signals and low-frequency signals exist at the same time.

Low-frequency signals correspond to slowly changing grayscale components in the image. For example, in an image of a savannah, low-frequency signals correspond to vast expanses of grassland that tend to be uniform in color.
The high-frequency signal corresponds to the gray-scale component that changes faster and faster in the image, which is caused by the sharp transition of the gray scale. If there is also a lion in the savannah image above, then the high frequency signal corresponds to information such as the edge of the lion.

The filter can allow a certain frequency component to pass or reject it, and can be divided into a low-pass filter and a high-pass filter according to its mode of action.

A filter that allows low frequency signals to pass is called a low pass filter. A low-pass filter attenuates high-frequency signals and passes low-frequency signals, blurring the image.
A filter that allows high-frequency signals to pass is called a high-pass filter. A high-pass filter attenuates low-frequency signals and passes high-frequency signals, which will enhance sharp details in the image, but will cause the image to lose contrast.

Fourier transform can separate the high-frequency signal and low-frequency signal of the image. For example, the Fourier transform can place the low-frequency signal at the center of the Fourier-transformed image, as shown in the previous figure, where the low-frequency signal is at the center of the right figure. The high-frequency signal and low-frequency signal obtained by Fourier transform can be processed separately, such as high-pass filtering or low-pass filtering. After the high-frequency or low-frequency signal of the image is processed, the inverse Fourier transform is performed and returned to the space domain, and the frequency domain processing of the image is completed. By processing the image in the frequency domain, operations such as image enhancement, image denoising, edge detection, feature extraction, compression, and encryption can be realized.

For example, as shown in the figure below, the original image on the left is the original image, the result image in the middle is the result obtained after performing Fourier transformation on the original image on the left, and the image on the right is the result after high-pass filtering the result. Replace the low-frequency component values in the Fourier transform result image result with 0 (processed as black), so that the low-frequency signal is shielded, and only the high-frequency signal is retained to achieve high-pass filtering.

To set the pixel values in the middle of the right image in the above figure to zero, you need to calculate the coordinates of its center position first, then select an area of 30 pixels in size from the top, bottom, left, and right sides centered on this coordinate, and set the pixel values in this area to zero. The implementation of this filter is:

rows,cols=img.shape

crow,ccol=int(rows/2),int(cols/2)

fshift[crow-30:crow+30,ccol-30:ccol+30]=0

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread('../boat.512.tiff', 0)

f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)

rows, cols = img.shape
crow, ccol = int(rows / 2), int(cols / 2)
fshift[crow - 30: crow + 30, ccol - 30: ccol + 30] = 0

ishift = np.fft.ifftshift(fshift)
iimg = np.fft.ifft2(ishift)
iimg = np.abs(iimg)

plt.subplot(121)
plt.imshow(img, cmap='gray')
plt.title('original')
plt.axis('off')

plt.subplot(122)
plt.imshow(iimg, cmap='gray')
plt.title('iimg')
plt.axis('off')

plt.show()

3. OpenCV implements Fourier transform:

OpenCV provides functions cv2.dft() and cv2.idft() to implement Fourier transform and inverse Fourier transform.

1. Implement the Fourier transform:

In OpenCV, the function cv2.dft() is used to perform Fourier transform, and the syntax format is:

Return result = cv2.dft (original image, converted logo)

When using this function, you need to pay attention to the usage specifications of the parameters:

For the parameter "original image", first use the np.float32() function to convert the image into np.float32 format.
The value of "conversion flag" is usually "cv2.DFT_COMPLEX_OUTPUT", which is used to output a complex array.

The result returned by the function

After the transformation of the function cv2.dft(), we get the spectral information of the original image. At this time, the zero-frequency component is not at the center. For the convenience of processing, it needs to be moved to the center, which can be realized by the function numpy.fft.fftshift(). For example, the following statement moves the zero-frequency component in the spectrum image dft to the center of the spectrum, and obtains the spectrum image dftshift with the zero-frequency component at the center.

dftShift=np.fft.fftshift(dft)

After the above processing, the spectral image is just a value composed of real and imaginary parts. To display it, further processing is required.

The function cv2.magnitude() can calculate the magnitude of spectral information. The syntax of this function is:

Return value = cv2.magnitude (parameter 1, parameter 2)
- Parameter 1: Floating-point x-coordinate value, which is the real part.
- Parameter 2: The floating-point y-coordinate value, which is the imaginary part, must have the same size as parameter 1 (the size of the value, not the size of the value).

The return value of the function cv2.magnitude() is the square root of the sum of the squares of parameter 1 and parameter 2, the formula is:

I represents the original image, and dst represents the target image.

After obtaining the magnitude of the spectrum information, it is usually necessary to further convert the magnitude value so as to display the spectrum information in the form of an image. To put it simply, it is necessary to map the amplitude value to the gray-scale space [0,255] of the gray-scale image, so that it can be displayed in the form of a gray-scale image.

The formula used here is:

result=20*np.log(cv2.magnitude(real part, imaginary part))

Example 1:

import cv2
import numpy as np

img = cv2.imread('../lena.bmp', 0)
dft = cv2.dft(np.float32(img), flags=cv2.DFT_COMPLEX_OUTPUT)
dft_shift = np.fft.fftshift(dft)

result = 20 * np.log(cv2.magnitude(dft_shift[:, :, 0], dft_shift[:, :, 1]))

print(dft)
print(dft_shift)
print(result)

Example 2: Use the OpenCV function to perform Fourier transform on the image and display its spectral information.

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread('../lena.bmp', 0)
dft = cv2.dft(np.float32(img), flags=cv2.DFT_COMPLEX_OUTPUT)
dft_shift = np.fft.fftshift(dft)
result = 20 * np.log(cv2.magnitude(dft_shift[:, :, 0], dft_shift[:, :, 1]))
plt.subplot(121)
plt.imshow(img, cmap='gray')
plt.title('original')
plt.axis('off')

plt.subplot(122)
plt.imshow(result, cmap='gray')
plt.title('result')
plt.axis('off')

plt.show()

2. Implement the inverse Fourier transform:

In OpenCV, the inverse Fourier transform is implemented using the function cv2.idft(), which is the inverse of the Fourier transform function cv2.dft(). Its syntax format is:

Return result = cv2.idft (original data)

After Fourier transforming the image, the zero-frequency component is usually moved to the center of the spectral image. If the zero-frequency component is moved using the function numpy.fft.fftshift(), then the function numpy.fft.ifftshift() should be used to restore the zero-frequency component to its original position before performing the inverse Fourier transform.

Note that after the inverse Fourier transform, the obtained value is still a complex number, and the function cv2.magnitude() needs to be used to calculate its magnitude.

Example:

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread('../boat.512.tiff', 0)
dft = cv2.dft(np.float32(img), flags=cv2.DFT_COMPLEX_OUTPUT)
dft_shift = np.fft.fftshift(dft)

rst = 20 * np.log(cv2.magnitude(dft_shift[:, :, 0], dft_shift[:, :, 1]))

ishift = np.fft.ifftshift(dft_shift)
iimg = cv2.idft(ishift)
iimg = cv2.magnitude(iimg[:, :, 0], iimg[:, :, 1])

plt.subplot(131)
plt.imshow(img, cmap='gray')
plt.title('original')
plt.axis('off')

plt.subplot(132)
plt.imshow(rst, cmap='gray')
plt.title('rst')
plt.axis('off')

plt.subplot(133)
plt.imshow(iimg, cmap='gray')
plt.title('iimg')
plt.axis('off')

plt.show()

3. Example of low-pass filtering:

As mentioned earlier, in an image, the low-frequency signal corresponds to the slowly changing grayscale component in the image. For example, in an image of a savannah, low-frequency signals correspond to vast expanses of grassland that tend to be uniform in color. The low-pass filter attenuates high-frequency signals and passes low-frequency signals, and the image will become blurred after low-pass filtering.

For example, in the image below, the left image original is the original image, the middle image result is the result obtained after Fourier transforming the original, and the right image is the low-pass filtered image. Replace all the high-frequency signal values in the Fourier transform result image result with 0 (processed as black), the high-frequency signal is shielded, and only the low-frequency signal is retained, thereby realizing low-pass filtering.
insert image description here

When implementing low-pass filtering, you can specially construct a mask image as shown in the left figure below, and use it to perform an AND operation with the Fourier transform spectrum image of the original image to filter out the high-frequency signals in the spectrum image .

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread('../boat.512.tiff', 0)
dft = cv2.dft(np.float32(img), flags=cv2.DFT_COMPLEX_OUTPUT)
dft_shift = np.fft.fftshift(dft)

rows, cols = img.shape
crow, ccol = int(rows / 2), int(cols / 2)
mask = np.zeros((rows, cols, 2), np.uint8)
mask[crow - 30: crow + 30, ccol - 30: ccol + 30] = 1

f_shift = dft_shift * mask
print(f_shift)

# mask = np.zeros((rows, cols), np.uint8)
# mask[crow - 30: crow + 30, ccol - 30: ccol + 30] = 1
# f_shift = cv2.bitwise_and(dft_shift, dft_shift, mask=mask)
# print(f_shift)

ishift = np.fft.ifftshift(f_shift)
iimg = cv2.idft(ishift)
iimg = cv2.magnitude(iimg[:, :, 0], iimg[:, :, 1])

plt.subplot(121)
plt.imshow(img, cmap='gray')
plt.title('original')
plt.axis('off')

plt.subplot(122)
plt.imshow(iimg, cmap='gray')
plt.title('iimg')
plt.axis('off')

plt.show()