CV: Basics of computer vision, image - to the cv2 python library to learn basic computer vision, image

table of Contents

2, Fourier transform

3, the Fourier transform of the image

1, a frequency-domain transform

2, and time domain transform LPF -HPF

Second, the classic algorithm for image denoising - various filters

0, non-linear filter and linear filter

Gaussian noise and Impulse noise difference

1, the filtering mean - averaging the pixel values

2, median filter - the pixel takes an intermediate value in ascending order: come common salt and pepper noise, speckle noise

3, Gaussian - weighted average of the pixel values of the whole image

4, bilateral filtering - spatial proximity compromise image and the pixel value similarity

An image of the Fourier transform

1, time domain and frequency domain

Time Domain : From our birth, we see the world through time, blossom, illness and death, changing clouds move will occur over time. This time as a way to observe the dynamic world of reference we call time-domain analysis .

Frequency domain : If you see the world from a different point of view, the world has not changed. Why, the reason is, the angle of observation is the frequency domain rather than time domain. Well, the next to introduce what is the frequency domain. Examples of chemical reaction:

Obviously, over time, sodium hydroxide and hydrochloric acid and aqueous sodium chloride eventually become, in the time domain of the angle, the reaction was changed. In the frequency domain it an angle, Na + Na + or Na + Na +, a hydrogen atom or a hydrogen atom, nature has not changed, which is analyzed in the frequency domain angle .
And one of the ways throughout the time domain and frequency domain, that is to say the pass in the Fourier analysis . Fourier analysis can be divided into Fourier series (Fourier Serie) and Fourier transform (Fourier Transformation).

(1), the frequency axis and time domain
frequency domain axis : a function of changes over time;
time domain axis : this function is always function itself does not change.

2, Fourier transform

Fu chiefs tell us that any periodic function, can be regarded as different amplitudes A , a different phase ψ superimposed sine wave .

For example, can be used in front of said sinusoidal wave superimposed on a square wave with a 90 degree angle, the middle portion is gradually approaching the square. Not just rectangular, could think of any other waveforms are possible with such a method are superimposed sine wave.

The nature of the Fourier series is decomposed into a signal of one period of an infinite number of separate (discrete) sine waves, the following formula, OO represents the observed signal, ωω represents a frequency, φφ denotes a phase, AA represents amplitude. But the universe does not seem conversion period, any change in the common things do not seem to be periodic, such as the little sister day trip if it is a cycle, it is not a very good start, so contrary to the laws of nature.

Gangster change Fu in the end doing? Of course, is an analytical tool through time domain and frequency domain, is actually an infinite cycle function is transformed to a time domain continuous non-periodic signals, converted in a frequency domain continuous non-periodic signals.

For Chestnut: For Piano, the
time domain : discrete key signal into a continuous signal is beautiful music.
Frequency Domain : every continuous shaking up will become a continuous signal, into a continuous spectrum from the discrete spectrum. The original superimposed discrete spectrum into a continuous spectrum accumulation, so the integration has become a symbol calculated from the summation symbol.

3, the Fourier transform of the image

Examples of the above-described one-dimensional Fourier transform of the signal, and the signal is continuous. We know that the image is two-dimensional discrete, continuous and discrete can be transformed by Fourier, it is nothing more than a two-dimensional signal in the direction xx yy directions to conduct a one-dimensional Fourier transform. So it seems, you can imagine, it is a time-domain constitute a grid matrix, the horizontal axis from 11 to nn, and the vertical axis is the case, the time-domain constitute all images are considered to be so. The frequency domain, of course, above, will get grid matrix related to the frequency domain.

Discrete Fourier transform two-dimensional, an image of an M × N image f (x, y) into the through discrete Fourier transform F (u, v), then the general formula: F (u, v) is the frequency domain, f (x, y) is a time domain:

Continuous Fourier transform

$F (\ mu \ nu) = \ int _ {- \ infty} ^ {\ infty} \ int _ {- \ infty} ^ {\ infty} f (t, z) ^ {- j * 2 \ pi [\ T + \ nu i]]} dtdz$

$f (t, z) = \ int _ {- \ infty} ^ {\ infty} \ int _ {- \ infty} ^ {\ infty} F (\ mu \ nu) e ^ {j * 2 \ pi [\ mu t + \ nu i]]} d \ mu d \ nu$

Discrete Fourier Transform

$F (\ mu, \ nu) = \ Sigma \ Sigma f (x, y) e ^ {- j * 2 \ pi [\ frac {\ mu x} {m} + \ frac {\ nu y} {n} ]]}$

$f (x, y) = \ frac {1} {MN} \ Sigma \ Sigma F (\ about \ nu) a ^ {j * 2 \ pi [\ frac {\ about x} {my} + \ frac { \ nu} {y n}]]}$

1, a frequency-domain transform

The complex time signal or signals into a spatial frequency component represented by the structure is in the form of frequency domain transform.
Spectrum : frequency domain transform expressed in rectangular coordinates, the resulting pattern is often said spectrum. Spectrum is a generic term amplitude spectrum as well as the specific content, phase spectrum, the power spectrum, energy spectrum, cepstrum video and other types of components. Mathematical principles to achieve transformation of the spectrum is the Fourier transform.
Amplitude spectrum : Introduction to the Fourier spectrum of the variation width of the image, an original image presented amplitude spectrum A change represents: an image pixel into the center of the brightest, then gradually darkened pixels on the edge of the darkest , so that a bright image can be found in the percentage of dark pixels. (That is, the frequency domain amplitude intensity AA).
Numpy a FFT (Fast Fourier Transform) package to complete this work, np.fft.fft2 () provides us with a complex array of frequency conversion, that is, the discrete Fourier transform of an image. If no parameters are passed, the size of the output array will have the same input.

The first argument is the input image, which is a grayscale image.
The second parameter is optional, which determines the size of the output array. If it is larger than the size of the input image, the input image before computing the FFT filled 0. If it is less than the input image, the input image is to be cropped.

You can see more of the white area in the center represents the low-amplitude waves accounted for most of the high-amplitude waves in the minority, but also describes the overall low frequency waves. Since the regulation of the waveform, the desired waves more linear approximate straight line and adjusted. The more low-frequency waves , that is, there are more wave approximates a straight line, that is, the frequency of these waves is very slow, ω is very small.

s1 = np.log (np.abs (fshift) ) # absolute value of: the change in the logarithm of the complex to real object in order to change the number of data to 0-255
using the log function demapping - not mean normalization : and not to take the following normalized form to use, because the elements of the matrix, the vast majority of the pixel values are small, if the following form to normalization, the vast majority of the value is very small, you will get a black image, normalized modeling this kind of thing has always been inside a good control of the degree.

import cv2
import numpy as np
import matplotlib.pyplot as plt
 
img = cv2.imread('F:/File_Python/Resources/face_images/ZLY01.jpg', 0) #直接读为灰度图像
img_fft = np.fft.fft2(img)            #做频率变换
img_fft_fshift = np.fft.fftshift(img_fft)     #转移像素做幅度普
img_fft_fshift_log = np.log(np.abs(img_fft_fshift))     #取绝对值：将复数变化成实数取对数的目的为了将数据变化到0-255


plt.subplot(121)
plt.imshow(img, 'gray')
plt.title('original')
 
plt.subplot(122)
plt.imshow(img_fft_fshift_log,'gray')
plt.title('center')
 
plt.show()

2, and time domain transform LPF -HPF

When domain transform, such as a high-pass filter. To do this, you only need to use a rectangular window size to remove the low-frequency part (the greater the frequency [omega], the more obvious vibration), i.e. the image information retained in the lower most useful. So what is the position of the image frequency is relatively high it, it is obvious boundary. Why, in the non-junction, you can take leisurely wave dithering to display information. At the junction, there is no quick shake, how to display the difference?

1, a high-pass filter (HPF) - for detecting an edge

High Pass Filter, a specific image detection region, and then surrounding pixels according to pixel difference to improve the pixel brightness filter. The results of the convolution matrix, increasing the value of the border, next to the smaller value, that is, able to detect edges .

import cv2
import numpy as np
import matplotlib.pyplot as plt

kernel_9 = np.array([[-1, -1, -1],
                     [-1, 8, -1],
                     [-1, -1, -1]])

kernel_25 = np.array([[-1, -1, -1, -1, -1],
                      [-1, 1, 2, 1, -1],
                      [-1, 2, 4, 2, -1],
                      [-1, 1, 2, 1, -1],
                      [-1, -1, -1, -1, -1]])

img = cv2.imread('F:/File_Python/Resources/face_images/ZLY01.jpg')
ndimg = np.array(img)



k3 = cv2.filter2D(ndimg, -1, kernel_9)      #convolve calculate 
k5 = cv2.filter2D(ndimg, -1, kernel_25)     #the second parameters measns the deepth of passageway.
#such as cv2.CV_8U means every passageway is 8 bit.
#-1 means the passageway of the source file and the object file is equal.



plt.subplot(131)
plt.imshow(img)
plt.title("source image")

plt.subplot(132)
plt.imshow(k3)
plt.title("kernel = 3")

plt.subplot(133)
plt.imshow(k5)
plt.title("kernel = 5")

plt.show()

2, a low pass filter (LPF) - blurring and de-noising

The high pass filter and low pass filter as well as the reverse (the LPF), a high pass filter and low-cut filter contrary, when the interpolation pixel and a surrounding pixel is less than a certain value, the luminance of the pixel smoothing for noise removal and fuzzy .
(1), such as PS software Gaussian blur, blur filter is a common one, belonging to attenuate high frequency signals a low pass filter. The picture shows the intermediate high-pass filtering, the right picture shows the most blurred.

import cv2
import numpy as np
import matplotlib.pyplot as plt

kernel_25h = np.array([[-1, -1, -1, -1, -1],
                      [-1, 1, 2, 1, -1],
                      [-1, 2, 4, 2, -1],
                      [-1, 1, 2, 1, -1],
                      [-1, -1, -1, -1, -1]])

kernel_25l = np.array([[0.04, 0.04, 0.04, 0.04, 0.04],
                      [0.04, 0.04, 0.04, 0.04, 0.04],
                      [0.04, 0.04, 0.04, 0.04, 0.04],
                      [0.04, 0.04, 0.04, 0.04, 0.04],
                      [0.04, 0.04, 0.04, 0.04, 0.04]])

img = cv2.imread('F:/File_Python/Resources/face_images/ZLY01.jpg')
ndimg = np.array(img)

k3 = cv2.filter2D(ndimg, -1, kernel_25h)
k5 = cv2.filter2D(ndimg, -1, kernel_25l)

plt.subplot(131)
plt.imshow(img)
plt.title("source image")

plt.subplot(132)
plt.imshow(k3)
plt.title("kernel5hpf")

plt.subplot(133)
plt.imshow(k5)
plt.title("kernel5lpf")
plt.show()

Second, the classic algorithm for image denoising - various filters

Image filtering, i.e. noise suppression target image under conditions to preserve image detail features, are indispensable image preprocessing operation, the treatment effect is good or bad will directly affect the effectiveness of subsequent image processing and analysis, and reliability. Eliminating noise components in the image filtering operation is called smoothing or image.

1, the filter : one kind is figuratively method: We can think of a window with a filter weighting coefficient, when processing an image using the smoothing filter, put into the window over the image, through the window to see the image we get.
For a filtering application in our lives: microdermabrasion beauty of function. If we face bumpy likened to the noise, then the algorithm is to filter out the noise, so we selfie skin looks very smooth.

2, two object images filtered - Clear + noise extraction features
(1), the feature extraction target image as a characteristic pattern recognition;
(2), to meet the requirements of the image processing, the digitized image to eliminate noise mixed.

3, the filtering process requires two: Lossless + clear image information
(1), the important information can not damage the contour edge image and the like;
(2) the image clarity is good visual effect.

4, two types of object image smoothing - Fuzzy + eliminate noise
smoothing filter is a low-frequency enhanced spatial domain filtering techniques. Smoothing spatial domain, typically using a simple average method, it is to compute the average luminance value of the pixel adjacent to the point. Is directly related to the size of the neighborhood and the effect of smoothing, the better the larger the effect of smoothing the neighborhood, the neighborhood is too large, the smoothing causes greater loss of edge information, so that the output image becomes blurred, and therefore need a reasonable choice o size of the field.

0, non-linear filter and linear filter

Raw data filtering result is a linear filter arithmetic operation , i.e. by arithmetic realization operations, such as averaging filters (average of gradation values of pixels within the template), Gaussian filter (Gaussian weighted average), etc. . Since the linear filter is an arithmetic operation, a fixed template, so the transfer function of the filter can be determined and is the only (i.e., the transfer function of the Fourier transform of the template).
Raw data filtering result is a nonlinear filter logic , i.e. logic operation to achieve such a maximum value filter, minimum value filter, a median filter , etc., by comparing the gradation value of a certain neighborhood size achieved, there is no fixed template and, therefore, no specific transfer function (because there is no template Fourier transform), Further, dilation and erosion are the maximum value, minimum value filter implementation.

Gaussian noise and Impulse noise difference

Gaussian noise refers to Gaussian noise, i.e. noise intensity of a maximum number of points, the farther from this intensity noise is less the number of points, and this rule Gaussian. Gaussian noise is an additive noise , i.e. noise is directly applied to the original image, it can be used to filter out linear filter .
Pepper noise similar to the salt and pepper sprinkled on the image, hence the name, a lot of white spots or black spots noise, such as TV snow noise on the image. Salt and pepper noise may be considered a logical noise, linear filter to filter out bad results, generally adopt median filtering is filtering can get better results.

1, the filtering mean - averaging the pixel values

Within a square picture region (typically 3 * 3), the center of the average value of all pixel values of the pixel points. Average filtering operation is carried out for more than the whole picture. Let's focus will be appreciated that FIG matrix:
(1), defects mean filter - also destroyed denoising image detail : an inherently drawback that it can not well protect the image detail, in the image denoising but also destroys the details of the image, so that the image becomes blurred, can not properly remove the noise points. In particular, salt and pepper noise.

You can see the picture blur and noise is not very effective in removing the blurring algorithm is just a picture of it.

2, median filter - the pixel takes an intermediate value in ascending order: come common salt and pepper noise, speckle noise

Median filter is a nonlinear filter that can maintain a good image edges while noise filtering.
Median filtering principle : all gray pixels to the current pixel in the small window centered in ascending order, taking intermediate sort results as the gradation value of the pixel values.

In a series of numbers in {1,4,6,8,9}, numeral 6 is simply the string of numbers. From this we can be applied to image processing. We still go in the image matrix 3 * 3, which has nine pixels, we will sort 9 pixels, and finally the central point of this assignment matrix for the nine pixel values.

We can see, good salt and pepper noise is smoothed, and it did not mean that blur too serious.

1, add salt and pepper noise to the picture

Pepper noise (salt-and-pepper noise) is a black and white light generated by the image sensor, transmission channel decoding process noise dark spots. The so-called salt and pepper, black pepper is, salt is white, salt and pepper noise is on the image, a random black pixel appears white. A noise salt and pepper noise is caused because the signal pulse intensity, the noise generating algorithm is relatively simple. Often caused by the image cutting, removing salt and pepper noise and pulse interference algorithm is the most common median filtering .

(1), addSaltNoise function, a step to add salt and pepper noise of the digital image, as follows:

Specified signal to noise ratio SNR (which values between [0, 1])
Calculating the number of total pixels SP, noise added to obtain a number of pixels NP = SP * (1-SNR)
Get each pixel position P random noise adding to the (i, j)
Specified pixel value is 0 or 255.
Repeat steps 3 and 4 two pixels is completed for all pixels NP
The output image after adding noise

Original → NSR noise ratio of signal to noise ratio 0.7 0.9 → NSR

2, a salt and pepper noise image to median filtering

Image results, only by median filtering is not enough. Thus the filter adjustment values, i.e. cv2.medianBlur (IMG, 3) function, to 5,7,101 ksize 3, in turn becomes clear, but it also makes the image becomes blurred due to too ksize , the useful information is also replaced with the median value, the error gradually emerged, so the choice of ksize directly affect the picture quality.

3, Gaussian - weighted average of the pixel values of the whole image

Is a linear Gaussian smoothing filter for eliminating Gaussian noise, a noise reduction process is widely used in image processing. Popular speaking, Gaussian filtering is performed on the whole image process of the weighted average value of each pixel, and the other by its own neighborhood pixel values obtained after a weighted average. Specific operation Gaussian filter is: a template (or convolution mask) scanning each pixel in the image, the weighted average of gray value pixels in the neighborhood determined by the value of the template to replace the central pixel of the template.

4, bilateral filtering - spatial proximity compromise image and the pixel value similarity

Bilateral filtering into account not only the pixel information, but also takes into account the pixel position information. It is a non-linear filtering method, a compromise processing and spatial proximity to pixel value similarity combined image, taking into account the spatial and gray similarity information, an edge-preserving achieve denoising.

Wij is the current pixel weights, Pij is the current pixel information Pi is the current pixel neighborhood average; Cij is the current pixel position information, Ci of the current pixel averaging position information, \ Sigma- 1 and \ Sigma2 respectively current pixel information, the current pixel position the standard deviation.

Reference article
Opencv Computer Vision Road (C) - Image Processing (a) Fourier transform