Summary of all notes: "Principles of Artificial Neural Networks"-Summary of Reading Notes

1. The concept of artificial neural network

Artificial neural network is an information processing system inspired by the biological brain, based on imitating the structure and function of the biological brain, and using mathematical and physical methods for research. It is composed of many very simple processing units that work in parallel according to a certain method. Computer systems that are connected to each other and dynamically respond to external input information depending on their status. It is a simple simulation of the brain, with high-speed processing capabilities and certain intelligence characteristics. Its function depends on the network structure, connection strength, and processing methods of each unit.

Second, the development history of artificial neural networks

The development of artificial neural networks has gone through four periods: in 1943, the MP model was proposed to mark the rise of artificial neural networks; in 1969, the publication of the book "Perceptron" marked the development of artificial neural networks into a period of depression; The Hopfield model proposed in 1982 and the error back-propagation model proposed in 1986 marked the prosperous period of artificial neural networks; in 1987, after the first International Conference on Artificial Neural Networks was held, the research of artificial neural networks entered a climax period.

Third, the characteristics of artificial neural networks

Artificial neural network is a new type of information processing system based on the structure and function of the human brain. It has parallel structure and parallel processing capabilities, knowledge can be distributed and stored, has good fault tolerance, high nonlinearity and computational inaccuracy, and Self-learning, self-organization and self-adaptive characteristics.

Brief description of characteristics

Inherent parallel structure and parallel processing characteristics
The computing functions of artificial neural networks are distributed on multiple processing units, and the processing units in the same layer operate in parallel at the same time. Information processing in neural networks is carried out in parallel and hierarchically in a large number of units, which not only has a high computing speed, but also far exceeds the serial processing speed of traditional computers. Supplement: Concurrency: Processing many things at the same time (half of the meal, and then eating) Parallel: processing many things at the same time (both eating and calling) The key is to see whether they can be at the same time. Serial: Process data sequentially.
Distributed storage characteristics of knowledge
A neural network can store a variety of information, and the connection weight of each neuron stores part of the variety of information. When a neural network wants to obtain distributed storage knowledge, it must use the associative memory method similar to humans or animals, that is, when a neural network obtains an input excitation, it must search for the stored knowledge that best matches the input Knowledge as the solution.
Good fault tolerance
When the input is some fuzzy, deformed and other imperfect data and information, the artificial neural network can restore the complete memory through association, so as to realize the correct recognition of incomplete input information. Because it is distributed storage, when some neurons are damaged, it will not have a significant impact on the performance of the entire system.
The highly nonlinear and computationally imprecise
artificial neural network is a highly parallel nonlinear system. The parallelism of its structure and the distributed storage of knowledge make the storage and processing of information exhibit spatially distributed and temporally parallel characteristic. Neural networks can process continuous analog signals and inaccurate and incomplete fuzzy information, which makes neural networks usually give satisfactory solutions rather than accurate solutions.
Self-learning, self-organization, and self-adaptation
Self-adaptation is the ability of a system to change its performance to adapt to changes in the environment. It usually includes both self-learning and self-organizing characteristics. Self-learning means that when the external environment changes, after a period of training or perception, the neural network can produce the desired output for a given input.
Self-organization means that the neural network can adjust the connection weight by itself through training, that is, adjust the synaptic connection between neurons to make it plastic, so as to gradually build a neural network adapted to different information processing requirements.

Fourth, the information processing capabilities of artificial neural networks

Storage capacity and computing capacity are two basic issues in modern computer science.

Storage capacity depends on different neural network models, neural network topology and network connection weight design methods.

Computational ability refers to the ability of the neural network to obtain the corresponding output at the output end after a given input at the input end of the neural network.

Five, the function of artificial neural network

Artificial neural network is a new type of intelligent information processing system, which starts from simulating the structure of the human brain's biological nervous system, and then simulating the functions of the human brain. Artificial neural network has the functions of nonlinear mapping, pattern recognition, classification and clustering, associative memory, optimized calculation, and knowledge acquisition and representation .

The optimization algorithm
refers to finding a set of parameter combinations under known constraints, so that the objective function determined by the combination reaches the minimum (or maximum).
Knowledge Acquisition and Representation
The knowledge acquisition capabilities of neural networks enable it to automatically extract characteristics from input data and discover laws without any prior knowledge, and construct the network through a self-organizing process to make it suitable for expressing the discovered laws .

Seven, the main research directions of artificial neural networks

Theoretical research
An important aspect of theoretical research is the study of basic theories, which mainly study the mechanism of human thinking and intelligence from the perspective of neurophysiology and cognitive science, laying a foundation for the modeling of cognitive information processing.
The realization technology research is carried
out from two aspects: software simulation and hardware realization.
Application Research
Explore how to use neural networks to solve practical problems in various application fields.

8. Artificial Neural Network and Artificial Intelligence

Artificial neural networks focus on the working mechanism of the human brain and adopt a bottom-up approach to research and realize intelligence; while artificial intelligence adopts a top-down approach and realizes intelligence by simulating the functions of the human brain.
self-study ability. The self-learning ability of artificial neural network is very strong. Generally, the more data collected for learning, the more complete and accurate the learning. On the other hand, the artificial neural network model has dynamic self-organization capabilities for different network structures (requires a technology: dynamic self-programming)
Speed and real-time processing capabilities.

Insert picture description here

Nine, artificial neural network and traditional computing

Although artificial neural network technology cannot fully replace traditional computing, it can complement it in some aspects. As a calculation model inspired by human brain functions, artificial neural network is closer to the structure of the human brain than traditional calculation models in form. It is expected that it has stronger computing power to solve some problems that are difficult to solve with traditional calculation methods.

Insert picture description here

The next chapter of the portal: "Principles of Artificial Neural Networks" Reading Notes (2)-Basics of Artificial Neural Networks

"Principles of Artificial Neural Networks" Reading Notes (1)-Introduction