1. Biological implications
Neurons are structurally composed of four parts: cell body, dendrites, axons and synapses.
The cell body is the main body of a neuron and consists of three parts: the nucleus, the cytoplasm and the cell membrane. The outside of the cell body is the cell membrane, which
The cell fluid inside and outside the membrane is separated. Since the cell membrane has different permeability to different ions in the cell fluid, there are ions inside and outside the membrane.
There is a difference in subconcentration, resulting in a resting potential of negative inside and positive outside. This potential difference is called the membrane potential.
Dendrites, nerve fibers with many projections extending outward from the cell body. Responsible for receiving input signals from other neurons.
Acts as the input end of the cell body.
Axon, the longest process extending from the cell body is called an axon. Axons are longer and thinner than dendrites. Axons are also called nerve fibers.
There are many thin branches at the end called nerve endings. Each nerve ending can send signals in all directions, which is equivalent to the cell body.
output terminal.
Synapse, the communication link between one neuron's axon and the cell body or dendrites of another neuron.
This connection is equivalent to the input/output interface (I/0) between neurons and is called a synapse.
Synapses cause changes in the membrane potential of nerve cells, and potential changes can be accumulated. A single neuron can interact with as many as
The axon terminals of thousands of other neurons form synaptic connections that receive impulse input from each axon.
These inputs can reach different parts of the neuron. Different input parts have different weights on the neuron.
Different input sites, the membrane potential of the nerve cell is the sum of the potentials generated by all its synapses. When the membrane potential of the nerve cell rises
When it is high enough to exceed a threshold, a pulse will be generated, so that the summed membrane potential directly affects the excitatory pulse of the nerve cell.
number.
The information of neurons is a pulse train with the same width and amplitude. If a nerve cell is excited, the frequency of the pulse train output by its axon will be
The rate is high; if a certain nerve cell is inhibited, the frequency of the pulse train output by its axon will be low, or even there will be no pulse output.
Therefore, synapses can be divided into two types: excitatory and inhibitory. Excitatory synapses may cause excitation of the next nerve cell, and inhibitory synapses
Sexual synapses inhibit the next nerve cell. Pulse transmission is in the forward direction and reverse propagation is not allowed. In addition, synaptic transmission of information requires
There needs to be a certain delay.
In summary, we can summarize the assumed characteristics of biological neural networks:
Each neuron is an information processing unit with multiple inputs and single output;
Neuron input is divided into two types: excitatory input and inhibitory input;
Neurons have spatial integration properties and threshold properties;
There is a fixed time lag between neuron input and output, which mainly depends on synaptic delay.
2.MP model
In 1943, psychologist W. McCulloch and mathematician W. Pitts jointly proposed the MP model.
Named after both of them (McCulloch-Pitts).
In order to make the modeling simpler and facilitate formal expression, we ignore complex factors such as time integration effect and refractory period.
elements, and treat the synaptic delay and strength of neurons as constants. The following is a schematic diagram of an MP model.
Combined with the schematic diagram of the MP model, for a certain neuron j (be careful not to confuse it with a variable, j here only serves as a standard
(recognize the function of a certain neuron), it may receive many input signals at the same time, represented by xi. As mentioned earlier, due to biological neuron
Neurons have different synaptic properties and synaptic strengths, so they have different effects on neurons. We use the weight wij to represent its positive
The negative simulates the excitation and inhibition of protrusions in biological neurons, and its size represents the different connection strengths of protrusions. Due to additivity, I
They accumulate and integrate all input signals, which is equivalent to the membrane potential in biological neurons.
Whether a neuron is activated or not depends on a certain threshold level, that is, the neuron is activated only when the sum of its inputs exceeds the threshold thea.
The neuron must be alive to fire impulses, otherwise the neuron will not produce an output signal. The entire process can be represented by the following function:
From this, we can summarize the six characteristics of the MP model:
Each neuron is an information processing unit with multiple inputs and single output. Neuron inputs are divided into two types: excitatory input and inhibitory input.
type;
Neurons have spatial integration properties and threshold properties;
There is a fixed time lag between neuron input and output, which mainly depends on synaptic delay;
Temporal integration effects and refractory periods are ignored;
The neuron itself is time-invariant, that is, its synaptic delay and synaptic strength are constant.
3. Neuron model
Figures, numerical formulas, and the text explaining the figures and numerical formulas will use the following symbols:
Scalar: lowercase italic letters.
Vector: lowercase bold letters, such as a, b, c.
Matrix: uppercase black monolithic letters, such as A, B, C.
Single input neuron model:
Neurons are calculated as follows: 
For example. If w=3, p=2, b=1.5, then
Transfer function:
Hard limit transfer function:
Linear transfer function:
Logarithmic transfer function:
Characteristics of sigmiod function:
Value range a∈(0,1); nonlinear, monotonic; infinitely differentiable; linear function can be approximated when |n| is small; linear function can be approximated when |n| is large
threshold function.
Multiple input neurons:
Weight matrix: Typically, a neuron has more than one input. A neuron with R inputs is shown in the figure below. its input
P1, P2, ....., PR respectively correspond to the elements W1,1, W1,2, ...., W1,R of the weight matrix w.
The neuron has a bias value b that is summed with the weighted sum of all inputs to form the net input n:
The expression can be written in matrix form:
The output of the neuron can be written as:
Weight subscript: The first subscript of the weight matrix element subscript represents the weight; the target neuron number specified by the corresponding connection, the second
The subscript indicates the source neuron number of the corresponding connection of the weight.
Simplified notation:
In this case, the neuron's output a is a scalar, and if the network has multiple neurons, the network output is a
vector.
Multiple neurons operating in parallel:
Each element of the input vector p is connected to each neuron through a weight matrix w.
Layers of neurons:
If the output of a certain layer is the output of the network, then the layer is called the output layer, and the other layers are called hidden layers.
Delays and integrators:
Recurrent Neural Network:
A recurrent network is a network with feedback in which part of its output is connected to its input, a type of discrete-time recurrent network.
How to choose a network structure? The description of the application problem is very helpful in defining the structure of the network from the following aspects:
The number of inputs to the network = the number of inputs to the application problem;
The number of output layer neurons = the number of outputs of the application problem:
The choice of transfer function for the output layer depends at least in part on the output description of the application problem.