How to realize image recognition with artificial neural network
Artificial Neural Networks (ANN) system has only been born for more than half a century since the end of the 1940s, but because of its advantages such as distributed storage of information, parallel processing and self-learning ability, it has been used in information Processing, pattern recognition, intelligent control and system modeling and other fields have been more and more widely used.
In particular, the Multiple-Layer Feedforward Network (referred to as BP network) based on the Error Back Propagation algorithm can approximate any continuous function with arbitrary precision, so it is widely used in nonlinear modeling , function approximation, pattern classification, etc.
Target recognition is a traditional subject in the field of pattern recognition, because target recognition is not an isolated problem, but a basic problem encountered by most subjects in the field of pattern recognition, and in different subjects, due to specific Different conditions require different solutions, so the research on target recognition still has theoretical and practical significance.
What is discussed here is the problem of sending the image signal sequence formed after the target object to be identified is captured by the imaging head (infrared or visible light, etc.) into the computer, and using the neural network to identify the image.
1. BP neural network BP network is a multi-layer network using Widrow-Hoff learning algorithm and nonlinear differentiable transfer function. A typical BP network uses the gradient descent algorithm, which is specified by the Widrow-Hoff algorithm.
Backpropagation refers to the method of computing gradients for nonlinear multilayer networks. A typical BP network structure is shown in the figure. We represent it with a vector diagram as shown in the figure below.
Among them: for the kth mode pair, the weighted input of the unit j of the output layer is the actual output of the unit and the weighted input of the unit i of the hidden layer is the actual output of the unit. The function f is a differentiable decreasing function. Its algorithm description As follows: (1) Initialize the network and learning parameters, such as setting the network initial weight matrix, learning factors, etc.
(2) Provide a training mode to train the network until the learning requirements are met. (3) Forward propagation process: For a given training pattern input, calculate the output pattern of the network and compare it with the expected pattern. If there is an error, execute (4); otherwise, return to (2).
(4) Backpropagation process: a. Calculate the error of the same layer of units; b. Correct weights and thresholds; The network can implement any nonlinear mapping of input to output.
Increasing the number of hidden layers of the network can reduce the error and improve the accuracy, but it also complicates the network and increases the training time of the network. The improvement of error accuracy can also be achieved by increasing the number of hidden layer nodes. In general, the priority should be given to increasing the number of nodes in the hidden layer.
3. Selection of the number of neurons in the hidden layer When the neural network is used to realize network mapping, the number of neurons in the hidden layer directly affects the learning ability and induction ability of the neural network.
When the number of neurons in the hidden layer is small, the learning time of the network is shorter each time, but the network may not be able to remember all the learning content due to insufficient learning; If the learning time is longer, the storage capacity of the network will increase accordingly, resulting in a decline in the network's inductive ability to unknown inputs, because there is no theoretical guidance for the selection of the number of neurons in the hidden layer, and it is generally determined by experience.
4. Neural Network Image Recognition System The artificial neural network method realizes pattern recognition, which can deal with some complex environmental information, unclear background knowledge, and unclear inference rules. Samples are allowed to have large defects and distortions. The shortcomings of the neural network method The reason is that its model is constantly being enriched and improved, and the number of patterns that can be recognized is not enough. The neural network method allows samples to have large defects and distortions. It runs fast, has good adaptive performance, and has high resolution.
The neural network image recognition system is a kind of neural network pattern recognition system, and the principle is the same. A general neural network image recognition system consists of preprocessing, feature extraction and neural network classifiers. Preprocessing is to delete useless information in the original data, smooth, binarize and normalize the amplitude, etc.
The feature extraction part in the neural network image recognition system does not necessarily exist, so it is divided into two categories: ① With feature extraction part: this type of system is actually a combination of traditional methods and neural network methods. This method can Make full use of human experience to obtain pattern features and neural network classification capabilities to identify target images.
Feature extraction must be able to reflect the characteristics of the entire image. But its anti-interference ability is not as good as that of category 2.
② No feature extraction part: feature extraction is omitted, and the entire image is directly used as the input of the neural network. In this way, the complexity of the neural network structure of the system is greatly increased, and the increase in the dimension of the input mode leads to an increase in the network scale. huge.
Furthermore, the neural network architecture needs to completely remove the effect of mode deformation on its own. However, the anti-interference performance of the network is good, and the recognition rate is high. When the BP network is used for classification, it must first select various types of samples for training, and the number of each type of samples should be approximately equal.
The reason is that on the one hand, it prevents the trained network from being too sensitive to categories with many samples, but not sensitive to categories with few samples. On the other hand, it can greatly improve the training speed and prevent the network from falling into local minimum points.
Since the BP network does not have the ability of invariant recognition, to make the network invariant to the translation, rotation, and expansion of the pattern, it is necessary to select samples of various possible situations as much as possible.
For example, it is necessary to select representative samples of different postures, different orientations, different angles, and different backgrounds, so as to ensure a high recognition rate of the network.
The first thing to construct a neural network classifier is to choose an appropriate network structure: the input of the neural network classifier is the feature vector of the image; the output node of the neural network classifier should be the number of categories. The number of hidden layers should be selected properly, and the number of neurons in each layer should be appropriate. At present, there are many network structures that use one hidden layer.
Then choose an appropriate learning algorithm, so as to have a good recognition effect.
In the learning stage, a large number of samples should be used for training and learning. Through a large number of samples, the connection weights of each layer of the neural network should be corrected so that the samples can be correctly identified. This is like memorizing numbers. The network The neurons in the network are like human brain cells, and the change of weights is like the change of the interaction of human brain cells. The neural network is like people memorizing numbers in sample learning, and the network weight adjustment when learning samples is quite For people to remember the image of each number, the network weight is the content remembered by the network. The network learning stage is the same as the repeated learning process of people from not knowing numbers to knowing numbers.
The neural network memorizes the image according to the entire feature vector. As long as most of the features conform to the samples that have been learned, it can be identified as the same category. Therefore, the neural network classifier can still correctly identify when the sample has large noise.
In the image recognition stage, as long as the lattice vector of the image is used as the input of the neural network classifier, after the calculation of the network, the output of the classifier is the recognition result. V. Simulation experiment 1. Experiment object In this experiment, MATLAB is used to complete the training of neural network and image recognition simulation.
Select the target image in BMP format with ten numbers from 0 to 9 from the experimental database. The size of the image is 16×8 pixels, each target image is added with 10%, 20%, 30%, 40%, 50% random noise respectively, and a total of 60 image samples are generated.
Divide the sample into two parts, one for training and the other for testing. In the experiment, 40 samples are used for training and 20 samples are used for testing. Random noise is generated by calling the function randn(m,n).
2. Network structure This experiment uses a three-layer BP network, and the number of neurons in the input layer is equal to the number of pixels in the sample image, which is 16×8. 24 neurons are selected for the hidden layer, which is an ideal number of hidden layer nodes tested in the experiment.
The number of neurons in the output layer is the number of patterns to be recognized. In this example, there are 10 patterns, so choose 10 neurons in the output layer, and 10 neurons correspond to 10 patterns one by one.
3. Network training and simulation based on MATLAB language to establish and initialize the network% =================S1 = 24;% The number of neurons in the hidden layer S1 is selected as 24[R,Q] = size(numdata);[S2,Q] = size(targets);F = numdata;P=double(F);net = newff(minmax(P),[S1 S2],{'logsig''logsig'} ,'trainingda','learngdm') where numdata is the training sample matrix with a size of 128×40, and targets is the corresponding target output matrix with a size of 10×40.
newff(PR,[S1 S2…SN],{TF1 TF2…TFN}, BTF,BLF,PF) is a function for building an N-layer forward BP network in the MATLAB function library. The argument PR of the function represents the network input vector Value range matrix [Pmin max]; S1~SN is the number of neurons in each layer; TF1~TFN is used to specify the transfer function of neurons in each layer; BTF is used to specify the training function of the network; BLF is used to specify the weight and the learning function of the threshold; PF is used to specify the performance function of the network, and the default value is 'mse'.
set training parameters net.performFcn = 'sse'; % sum of squared error performance function = 0.1; % sum of squared error target = 20; % progress display frequency net.trainParam.epochs = 5000; % max training steps = 0.95; % momentum Constant network training net=init(net);% initialize network [net,tr] = train(net,P,T);% network training simulates the trained network D=sim(net,P);A = sim (net,B); B is the test sample vector set, a 128×20 lattice.
D is the recognition result of the training sample by the network, and A is the network recognition result of the test sample. Experimental results show that the recognition rate of the network for both training samples and test samples is 100%. The picture shows the recognition results of the network after adding 50% random noise to the five numbers of 64579.
6. Summary From the above experiments, it can be seen that it is feasible to use neural network recognition. The example given is just a simple digital recognition experiment. If you want to recognize complex target images in network mode, you need to reduce the network scale. Increase recognition ability, the principle is the same.
Google AI Writing Project: Neural Network Pseudo-Original
CNN neural network for image classification (Matlab)
You have to see what good copywriting your images are . If it is a color number, first convert it to grayscale. Train the network with MNIST. If it's a variety of subjects, train with colored imageNET. If your data size is large enough to be comparable to the dataset, then train the network directly on your data.
After training on a popular data set, you need to fix the convolutional pooling layer, and only train the parameters of the fully connected layer behind, using your own data set. CNN first is to adjust the network structure, several layers of convolution, several layers of pooling, the size of the convolution template, etc.
Instead, adjust the parameters, weight scale, learning rate, reg, etc. on the determined structure.
You use CNN for image classification, nothing more than using CNN as a means of learning features. You can think of the network as two parts. The front convolutional layer learns basic-medium-high-level features of images, and the latter fully connected layer corresponds to ordinary neural networks. Do classification.
If you need to learn, first you go to the UFLDL tutorial. Then instead of asking others, cs231n, first of all, have you read the imageNet dataset? For the method of mixing popular data sets with your own data to train models. It's okay if the two data sets are very similar.
But for popular data sets, the amount of labeled data is generally not too large. If it is 1:1000, 1:100, then you may not add your own data, and you can get the model completely trained with the data set. A good result.
If there is some difference between my own data and the data set, then I think my own data is being added to the data set as noise when it is mixed together.
CNN believes that images are locally relevant, and the method of deceiving CNN is mainly due to the fact that natural images are distributed in a manifold structure, and the trained model requires this manifold assumption, while artificially synthesized images are due to the addition of unnatural noise. It does not meet the model assumptions, so it can seriously interfere with the classification results with noise that is difficult to distinguish with the naked eye.
If the difference between the two is too large, the data set is one distribution, and your data is another. Put them together for training. I haven't tried it, but I don't think the result will be very good. At this time, the data set can only be used to train the feature extraction ability of cnn. Then use the fully connected layer for classification, and adjust the scale according to your data volume.
Classification of Feedforward Neural Networks
The single-layer feedforward neural network is the simplest type of artificial neural network, which only includes one output layer, and the value (output value) of the node on the output layer is directly obtained by multiplying the input value by the weight value.
Take out one of the units for discussion, the transformation relationship from input to output is the above formula, is the input feature vector, is the connection weight to, and the output is the classification result according to different features.
A multilayer feedforward neural network has an input layer, one or more hidden layers in between, and an output layer. The input and output transformation relationship in the multi-layer perceptron network is that each layer is equivalent to a single-layer feed-forward neural network. For example, for the first layer, it forms a one-dimensional hyperplane.
It performs a linear classification of the input patterns of this layer, but due to the combination of multiple layers, a more complex classification of the input patterns can be achieved in the end.
What are the three categories of neural network algorithms?
The three categories of neural network algorithms are: 1. Feedforward neural network: This is the most common type of neural network in practical applications. The first layer is the input and the last layer is the output. If there are multiple hidden layers, we call it a "deep" neural network. They compute a series of transformations that change the similarity of samples.
The activity of neurons in each layer is a non-linear function of the activity of the previous layer. 2. Recurrent Networks: Recurrent networks have oriented cycles in their connection graph, which means you can follow the arrows back to where you started. They can have complex dynamics, making them difficult to train. They are more biologically realistic.
The purpose of recurrent networks is to process sequential data. In the traditional neural network model, from the input layer to the hidden layer to the output layer, the layers are fully connected, and the nodes between each layer are unconnected. But this ordinary neural network is powerless for many problems.
Recurrent neural network, that is, the current output of a sequence is also related to the previous output.
The specific manifestation is that the network will remember the previous information and apply it to the calculation of the current output, that is, the nodes between the hidden layers are no longer connected but connected, and the input of the hidden layer not only includes the output of the input layer Also includes the output of the hidden layer at the previous moment.
3. Symmetrically connected networks: Symmetrically connected networks are a bit like recurrent networks, but the connections between units are symmetrical (they have the same weight in both directions). Symmetrically connected networks are easier to analyze than recurrent networks. There are more constraints in this network because they obey the law of the energy function.
Symmetrically connected networks without hidden units are called "Hopfield networks". A network with symmetrical connections of hidden units is called a Boltzmann machine.
Extended information: Application and development: Psychologists and cognitive scientists study neural networks for the purpose of exploring the mechanism of human brain processing, storing and searching information, clarifying the mechanism of human brain function, and establishing the microstructure theory of human cognitive process.
Experts in biology, medicine, and brain science try to promote the development of brain science to a quantitative, precise, and theoretical system through the study of neural networks, and also hope for new breakthroughs in clinical medicine; the purpose of information processing and computer scientists to study this issue is to Seek new ways to solve a large number of problems that cannot be solved or are extremely difficult to solve, and construct a new generation of computers that are closer to the functions of the human brain.