7.4.1 Full connection and partial connection
In 1968, biologists Professor David Hunter Hubel and
Professor Torsten N. Wiesel made an important discovery when studying how animals process visual information
. They found that the cerebral cortex of animals processes information hierarchically and hierarchically.
There are several different types of cells in the primary visual cortex of the brain , and these different types of cells are responsible for different levels of visual perception
.
The research results of the two scholars have important inspiration for the field of neural networks. It turns out that when we think
, the neurons in the brain are not in a "fully connected" way, that is to say, there is no need to activate
all the cells in the brain to think about one thing. So can the artificial neural network also use
the "local activation" mode of neurons like the brain? In this way, the complexity of the neural network can be greatly simplified.
Radial basis function neural network is one of the representatives. Radial Basis Function (RBF) neural network is a kind of feedforward neural network with
excellent performance . Like other forward neural networks based on BP algorithm, it can realize the approximation to any nonlinear function. One point is the commonality of all neural networks, the approximation ability depends on the number of neurons in the hidden layer. The RBF neural network consists of an input layer, a hidden layer, and an output layer. The transformation from the input space to the hidden layer space is nonlinear, while the transformation from the hidden layer space to the output layer space is linear, as shown in Figure 7-13.
From the network structure point of view, there are obvious differences between RBF neural network and BP neural network. Mainly in the
following three aspects:
(1) BP neural network can contain multiple hidden layers, but RBF has only one hidden layer.
(2) The BP neural network implements weight connection, and the transfer function of the hidden layer unit generally chooses a nonlinear
function. In the RBF neural network, there is a direct connection between the input layer and the hidden layer, and the
weight connection is implemented from the hidden layer to the output layer. The transfer function of the hidden layer unit of the RBF neural network is generally a centrally symmetric Gaussian function
.
(3) BP neural network is a global approximation network, and each parameter in the network has an impact on the output result
. Every time a new sample is input, the weights of all neurons in the network must be updated, so the learning speed is
relatively slow; the RBF neural network is a local near network, that is to say,
there are only a few in a certain local area of the network input space. Connecting neurons affects the output of the network. As shown in Figure 7-14, each
time the input is made, only the neurons that are closer to the input sample vector will become active, and the corresponding weights will be
updated , and the other weights will remain unchanged. This is determined by the Gaussian distribution function determined by its characteristics.
The BP neural network is like, when we see an image of a cat, the
neurons in the brain responsible for vision, smell, taste, and sense are all activated. The global response of the brain, after integrating all sensory calculations,
it is judged that this is a picture of a cat; the RBF neural network is like when we see a picture of a cat
, only the neurons responsible for vision are activated in the brain, and the other senses of smell , taste, and? neurons
are not activated. The brain responds locally, and the amount of calculation is small, so the calculation speed of the model is of course much faster than that of the BP neural network
.
7.4.2 Changing the activation function
The main reason for the above differences is that the activation functions of the hidden layer neurons in the two networks are different.
The basic idea of the RBF network is to use the Gaussian function as the basic structure of hidden layer neurons, and then
directly map the input to the hidden layer space without connecting through weights. When the center point of the RBF is determined,
the mapping relationship is also determined. The mapping from the hidden layer to the output layer is linear, that is, the output of the network is the
linear weighted sum of the outputs of all neurons in the hidden layer. In the RBF neural network, the function of the hidden layer is to
map the data from the low-dimensional space to the high-dimensional space. When the linearly inseparable data set in the low-dimensional space is converted to the high-dimensional space, a linearly
separable hyperplane can be found. For example, when the data that cannot be distinguished linearly in the two-dimensional plane is mapped
to the three-dimensional space, a plane distinction can be found, as shown in Figure 7-15.
The mapping from input to output in RBF network is nonlinear, while the output is
linear to adjustable parameters. This design allows the weights of the network to be directly solved by linear equations, thereby increasing the learning speed
and avoiding local minimum problems.
Why does the global? Near network become local? Near network after using the Gaussian function as the activation function
? Let's take a look at the image characteristics of the Gaussian function. The image of the Gaussian function conforms to the normal distribution, and the function
image is attenuated on both sides and radially symmetrical, as shown in Figure 7-16.
The hidden layer neurons of the RBF neural network use the distance between the input sample and the center vector (such as the Euclidean distance
) as the independent variable of the function.
The farther the input of a neuron is from the center of the activation function, the less activated the neuron is . This sentence can be understood as: the neurons that are closer to the input sample
will be mapped to a larger value in the role of the Gaussian formula, and the neurons can be activated at this time.
Neurons that are far away from the input sample are mapped to a value that is almost 0, that is, the neuron is not activated
, and there is no weight value that is not updated during backpropagation. From a global perspective, only a
part of the neurons in the network update the weights, which is the so-called local learning. It can be seen that for the RBF neural network
, the core problem is to determine the central parameters of the hidden layer neurons. The commonly used method is to directly select the central parameters from a given training sample set
according to a certain method, or obtain them by clustering.
The advantage of the RBF network is that the network can determine the corresponding topology structure according to specific problems, and has
the characteristics of self-learning, self-organization and self-adaptation. It has consistent closeness to nonlinear continuous functions,
fast learning speed, can carry out large-scale data fusion, and can process data at high speed in parallel. At present, RBF neural
network has been successfully applied
in different fields such as nonlinear function approximation, time series analysis, pattern recognition, image processing, control and fault diagnosis.
At present, the artificial neural network has achieved world-renowned achievements in some specific fields, but its learning
and training? It is often a difficult process. In order to obtain the best results, it is often necessary to repeat the experiment many
times. Therefore, the diversified application of neural networks and the mining of scenarios are inseparable from the joint
efforts of product managers and engineers. With the industry's in-depth research on neural networks and the improvement of hardware computing capabilities, it is believed that in the near future
In the near future, neural networks must have wider applications.