"Principles of Artificial Neural Networks" Reading Notes (2)-Basics of Artificial Neural Networks

Summary of all notes: "Principles of Artificial Neural Networks"-Summary of Reading Notes

1. Biological Nervous System

The structure of biological neurons

The function of biological neurons

• Spatio-temporal integration function
• Excited and inhibited state
• Pulse and potential conversion
• Nerve fiber conduction velocity
• Synaptic delay and refractory period
• Learning, forgetting and fatigue

2. Artificial neuron model

Formal description of artificial neurons

The function of artificial neurons

• Weighting: Process each input signal to determine its strength
• Sum: Determine the combined effect of all input signals
• Transfer characteristics: determine its output

Artificial neuron model

Transfer function

Three, MP model

Fourth, the interconnection structure of artificial neural networks

• There is no interconnected hierarchical structure within the layer without feedback.
  Simplex hierarchical structure: input layer, hidden layer, output layer (BP model)
• No interconnected hierarchy within the feedback layer
• Non-feedback interconnection hierarchical structure
  self-organizing neural network
• Interconnected hierarchical structure within the feedback layer.
  Mutual combination structure. There may be connections between any two neurons in the network (Hopfield network, Boltzmann machine)

Five, artificial neural network learning

The learning method of artificial neural network

The learning process of artificial neural networks

• Network connection weight adjustment process

Learning method of artificial neural network (evaluation standard)

• Guided learning: the evaluation criteria are provided by the outside (expected output)
• Unsupervised learning: the evaluation criteria are provided by the network itself
• Indoctrination learning: getting the right to connect to the network through design
  

Basic neural network learning rules

• Hubb learning rules
  adjust neurons $i$ to neuronjjConnection weight between$j$$W_{ij}$The method is: if the $i$ neuron and$If\; j$ neurons are in an excited state at the same time, the connection between them should be strengthened.
  The Hubb learning rules are consistent with the "conditioned reflex" theory, which has been confirmed by the nerve cell theory.
• Error correction learning rules are
  applied to artificial neural network models that use guided learning methods.
  Adjust the connection rights according to the external feedback (desired output) from the output node, so that the actual output of the network output node is consistent with the expected output of the outside.
  A large number of training samples are required for learning and training, so the convergence speed during training is slow.
  Some error correction learning rules have the problem of "local minimum", which cannot guarantee to converge to the global minimum.
• Relevant learning rules
  only adjust the connection weight based on the activation level of interconnected neurons.
  It is often used in the artificial neural network model that can realize self-associative memory, and is used to realize rote learning of special memory state.
• Random learning rules The
  random learning rules use the relationship between random process, probability and energy to adjust the connection weight.
  The network does not adjust the connection weights based on a certain deterministic algorithm, but processes it according to a certain probability distribution.
  It is possible to make the network avoid falling into the local minimum point and reach the global minimum point.
• Winner is King Learning Rule
  A competitive learning rule, applied to an artificial neural network model that uses an unsupervised learning method.
  In the competition layer of the neural network, for a specific input model $X$ , only the neuronjjwith the largest weighted sum$j$ can win.
  The result of the winner is king learning rule adjusting the input connection weight of the winning neuron is to make it closer to the current input mode$X$。
• Inner-satellite and extraterrestrial learning rules
  Inner-satellite learning rules are used to train inner-satellite nodes to respond to a specific input pattern $The\; X-$ .
  Alien learning rules are used to adjust and alien node$J$ connected alien output connection right$T_j$To make it close to the desired output mode $And$。

The next chapter Portal: "Principles of Artificial Neural Networks" Reading Notes (3)-Early Adaptive Neural Networks