1. Brief description
BP neural network is now a relatively mature network model, because neural network has inherent advantages in digital image processing, especially in image compression. Therefore, I have been studying neural network for a while. At the same time, I studied the principle and process of BP network to achieve image compression, and carried out a simulation experiment on MATLAB. It was found that the designed BP neural network has good generalization ability, and the effect on image compression is still good. good.
BP neural network model architecture and training principles
BP neural network is one of the more mature neural networks currently developed, and it is also a multi-layer feed-forward artificial neural network with powerful nonlinear differentiable functions for weight correction and adjustment, which can withstand rigorous testing. Mathematical logic derivation has been widely and widely recognized in many pattern recognition books and many data compression papers. The main feature of BP neural network algorithm is the forward transmission of input signals and the reverse propagation of errors. BP network The essence of the learning algorithm is to transform the input-output problem of the sample set into a nonlinear optimization problem. It is divided into three layers of network structure, including three layers such as input layer, hidden layer, and output layer. A typical network The structure of the model can be composed of the following parts:
Let's talk about forward propagation first:
First of all, in the BP neural network model, we now assume that it is an L layer, where its input layer is the first layer, and its output layer is the last layer, which is the L layer. The hidden layer is actually from the second layer to the L-1 layer. layer. If it is the Kth neural unit in the output layer, it can be simply expressed as
The superscript is expressed as the number of layers, and the subscript is expressed as the Kth neuron. Similarly, other layers and neurons can also be summarized in this way. In this way, the number of neurons in the L-th layer can be used |L | to represent. That is, the relationship between the input and output of the Jth neuron in the L layer is shown in the figure below:
in
b represents the connection weight between the Jth neuron of the L-th layer and the Jth neuron of the L-1 layer.
And its output Oij is as follows, where f is the activation function
In the neural network, we have many activation functions to choose from, but we still choose to use the sigmood function here. Why do we choose this function? Because I have collected a lot of resources, if we use some other activation functions or give up directly Activation function, but how many times we use the hidden layer, we still get a linear function in the end, but after using the nonlinear sigmoid function, there will be no such problem, and our network can also fit the nonlinear function.
Here we only need to remember the derivation of the sigmoid function, because we will use it in the next backpropagation.
After guidance:
backpropagation
Regarding backpropagation, this part is probably the essence of the entire BP neural network. So what exactly is backpropagation? In fact, backpropagation is to update the weights through the final error. For example, you are a student now , the school made a lot of test papers, and after the exams were sent to you, you found that the results were not good. When the principal saw the report card, the principal was very angry, and the principal said, the grade director, you can find it for me Question, the grade teacher then told the class teacher to find the problem for me,,, until you students, the teacher asked you to change it. This corresponds to the input layer of our neural network until the output layer. It is used here An important rule of is the gradient descent rule.
The gradient descent method, the knowledge of advanced mathematics in college, actually means that you cannot update the weight at the same time, but update the weight in the direction of the gradient to obtain more convergence. The formula is as follows:
Here is a negative gradient direction that needs to be explained, because your learning rate is greater than 0. For the error E, its partial derivative to W must be greater than 0. That is to say, the error increases with the weight Increase, then we need to reduce the error by reducing the weight, so the rate of change of W needs to be less than 0, of course, if we require the maximum value of XX, we can also use a positive gradient.
Intuitively, the weight update of each neuron of the neural network is updated through the partial derivative of the error. Here we introduce a new variable error, and the error of the jth neuron in the l-th layer is defined as
Such a weight update process can be:
What we need to use here is the chain derivation rule, as follows:
According to the derivation formula of the sigmoid function, and the partial derivative of the error, we can get the error term of the neurons of the L layer:
Therefore, the weight update formula of the upper layer and the output of the output layer can be obtained:
Therefore, the weight update formula of the upper layer and the output of the output layer can be obtained:
For the hidden layer, we need to superimpose the influence of each error of his output layer unit when deriving, which is similar to the output layer and will not be described again.
For the error term of the jth neuron of layer L-1, since each neuron is connected to each unit of the output layer, we need to sum over the |L| units of the output layer, thus obtaining The formula for calculating the error term is:
So the formula for the link weight of L-2 and L-1 layers is obtained:
For other layers, we can get the error formula and weight update formula of the neuron unit J in the first layer by generalizing:
For the derivation process, simply speaking, it is a chain derivation. From the above formula, we can see that backpropagation is also derivation, but for the hidden layer and the input layer, the errors generated by them can affect each A final output layer unit, so we need to sum the errors generated by it. The premise of backpropagation is forward propagation, and the formula of forward propagation is believed to be easy for everyone to remember, while backpropagation is actually the chain derivation of the formula of forward propagation, only need to pay attention to the derivation process For the influence of w on the output layer, it is enough to judge when the summation operation is required.
Analysis of image compression process of BP neural network based on MATLAB:
Because the application of BP neural network on MATLAB to compress digital images mainly includes three links: training sample construction, simulation and image reconstruction.
Construction of training samples
Because the performance of my machine is not enough, considering that all pixel data in the entire image needs to be used as the input data of the BP network, in order to control the training scale and training speed of the entire network, the image is divided into blocks, but Considering the correlation and difference between adjacent pixels, the proportion of pixels occupied by a small image should not be too large. Assuming an image is composed of N*N pixels, the entire image is cut into M regular-sized small Image blocks, where each image is composed of n*n pixels, the data of each small image is reconstructed into a column vector, which is used as the training vector of the sample, and then normalized data processing is performed.
Here, the most valued linear function conversion method is used, namely:
Y(k)={X(k)-X(min)}/{X(max)-X(min)}
Among them, X(k) is the data before conversion, Y(k) is the data after conversion, X(min), X(max) are the minimum and maximum values in the entire data set. Through normalization, each The pixel values of the training vectors are normalized to the range [0,1].
2: After creating and training a qualified BP network, use the Sigmoid mentioned above to simulate and compress the normalized processed image data, output the simulation vector, and then restore a complete image data through image reconstruction .
3: Now experiment with a static image, the control size is 128*128, first divide the whole image into 4*4 small images, and then convert each small image into a 16*1 column vector, and unify According to the above-mentioned operations, analyze the sample as the input of the network, create a BP network, and carry out training. This time the compression ratio is 4.
2. Code
%% clean up
clear, clc
close all
%% 载入数据
col=256;
row=256;
I=imread('lena.bmp');
load comp
com.lw=double(com.lw)/63;
com.b=double(com.b)/63;
com.d=double(com.d)/63;
com.lw=com.lw*(maxlw-minlw)+minlw;
com.b=com.b*(maxb-minb)+minb;
com.d=com.d*(maxd-mind)+mind;
%% Rebuild
for i=1:4096
Y(:,i)=com.lw*(com.d(:,i)) +com.b;
end
%% Denormalize
Y=uint8(Y*255);
%% Image block restoration
I1=re_divide(Y,col,4);
%% Calculate performance
fprintf('PSNR :\n ');
psnr=10*log10(255^2*row*col/sum(sum((I-I1).^2))); disp
(psnr)
a= dir();
for i=1:length(a)
if (strcmp(a(i).name,'comp.mat')==1)
si=a(i).bytes;
break;
end
end
fprintf(' rate: \n ');
rate=double(si)/(256*256);
disp(rate)
figure(1)
imshow(I)
title('original image');
figure(2)
imshow(I1)
title( 'Reconstruct image');
3. Running results
