FCN
Image denoising
Recently, it has been found that the classification accuracy of the designed network has not been able to improve. I took out the wrongly classified pictures for comparison, and found that the pictures in the original data set were noisy, so I wrote a fully convolutional self-encoder for image noise reduction.
The structure of the network is as follows:
Description of network structure
2.1 Part 1: Encoder Convolutional Layer
The Encoder convolutional layer sets up three layers of convolution and pooling layers to process the image.
In the first layer of convolution, we use 64 filters of size 3 x 3, strides is 1 by default, and our height and width will not be changed after padding is set to the same, so after the first layer of convolution After multiplying, we get data from the original 28 x 28 x 1 to 28 x 28 x 64.
Then perform max pooling on the convolution result. Here I set the size and stride to be 2 x 2. The pooling operation does not change the depth of the convolution result, so the size after pooling is 14 x 14 x64.
The other convolutional layers are not repeated here. The activation functions of all convolutional layers use ReLU.
After three layers of convolution and pooling operations, the conv3 we get is actually equivalent to the hidden layer of AutoEncoder in the previous part, and the data in this layer has been compressed to a size of 4 x 4 x 32.
At this point, we have completed the convolution operation on the Encoder side, and the data dimension has changed from 28 x 28 x 1 to 4 x 4 x 32.
2.2 Decoder convolutional layer
On the Decoder side, we do not simply perform the convolution operation, but use the combination of Upsample (the Chinese translation can be upsampling) + convolution.
We know that the convolution operation scans each patch in the image through a filter, and then performs nonlinear processing after the weighted summation of the pixel blocks in the patch. For example, if the size of our patch in the original image is 3 x 3 (the common point is that we take one of the 3 x 3 pixel blocks in a picture), then we use a 3 x 3 filter After processing this patch, the patch becomes a pixel block after convolution. In Deconvolution (or transposed convolution) the process is reversed, 1 pixel block is expanded into 3 x 3 pixel blocks.
But Deconvolution has some drawbacks, it will cause checkerboard patterns in the picture, this is because during the process of Deconvolution, there will be a lot of overlap in the filters. In order to solve this problem, someone proposed to use Upsample plus convolutional layer to solve it.
There are two common ways about Upsample, one is nearest neighbor interpolation, and the other is bilinear interpolation.
The operation of Upsample is also encapsulated in TensorFlow. We use resize_nearest_neighbor to resize the result of the Encoder convolution, and then perform convolution processing. After three Upsample operations, we get a data size of 28 x 28 x 64. Finally, we need to convolve this result again, down to the size of our original image.
The last step defines the loss and optimizer.
2.3 Constructing noisy data
Through the above steps, we have constructed the entire convolutional autoencoder model. Since we want to denoise the image through this model, we also need to construct our noise data based on the original data before training.
Let's take a simple example above to see how to add noise. We get the data img of a picture (size is 784), and add the noise factor multiplied by the random number to it, which will change the pixels on the picture. . Next, since each pixel data of the MNIST data is processed into a number between 0-1, we use numpy.clip to perform a clip operation on the image with added noise to ensure that each pixel data is still between 0-1. .
2.4 Training data
2.5 Data Visualization
Summarize:
At first, I just understood the mechanism of full convolution. I thought I had to write the deconvolution code myself. Finally, I found that there are functions that can realize deconvolution in tensorflow. Figure, I feel that this attempt is still very easy.
On the other hand, in this attempt, I got a more nuanced understanding of plotting with matplotlib.