"Principles of Artificial Neural Networks" Reading Notes (9)-Realization of Artificial Neural Networks

Summary of all notes: "Principles of Artificial Neural Networks"-Summary of Reading Notes

1. Overview of Neural Network Implementation Technology

Neural network realization degree

• Zero-level realization: The specific realization of neural network is not involved, and the neural network only exists in the theoretical model;
• First-level realization: limited to the realization of software simulation on the von Neumann architecture computer;
• Secondary realization: use $p$ physical processing units are implemented bynnA network composed of$n$ neurons, where$0<p<n$；
• Three-level realization: the physical processing units and connections correspond to the neurons and connections in the neural network model to be realized one-to-one.

Materials used in the realization of the optic neural network

• Virtual Realization Based on Traditional Computer
  Research on neural computer architecture, that is, how to effectively organize various functional components on the basis of existing technical conditions to support neural network simulation to the greatest extent.
  The use of software increases the versatility and flexibility of implementation

• Software simulation on traditional computers;
• Neural network parallel multi-machine system;
• Neural computing accelerator.

• Physical realization based on direct hardware devices (full hardware realization)
  The communication between the physical processing unit and the processing unit corresponds to the neurons and connections in the neural network model of the specific problem, each neuron and each connection has a one-to-one correspondence The corresponding physical device.
  Advantages: fast processing speed, easy to meet real-time requirements.
  Disadvantages: lack of versatility, flexibility and programmability.

2. Virtual Realization of Neural Network

Software simulation based on traditional computer

Use any high-level language (such as C/C++ language) to simulate and implement a specific neural network model on a general-purpose computer.

• Manual coding can describe the neural calculation process in depth and detail, and implement flexible control.

General purpose neural network model library

• There are some neural network models that have been established in advance, from which researchers choose neural network models, parameters and learning rules suitable for specific problems to simulate and solve them.

Develop a neural network software development environment that supports general neural network simulation.

Neural Network Parallel Multi-Computer System

Point of departure

• The use of multiple processors enables each processor to share part of the calculation tasks, increase the calculation speed, and at the same time reflect the essential characteristics of large-scale parallelism of neural computing.

Implement four levels of neural computing in parallel

• Neural network computing level
• Neural computing step level
• Neuron computing level
• Micro task computing level

Neural Computing Accelerator

Features of neural computing

• Operational concentration
• Parallel processing
• Huge storage

Neural computing accelerator is mainly designed for neural computing characteristics. Based on existing computer technology, it can quickly complete multiplication and addition operations, and can effectively use memory to quickly store and access neural network information.

Three, the physical realization of the neural network

VLSI realization of neural network

• Realization of analog VLSI of neural network
• Digital VLSI Realization of Neural Network
• Realization of digital-analog hybrid VLSI of neural network

Optical realization of neural network

Pure optical method

• A typical pure optical device is a hologram, which can realize associative memory addressed by content;
• Although pure optical devices can realize high-density connections without mutual influence and have good parallelism, it is not easy to realize the nonlinear transfer characteristics of neurons.

Optical hybrid method

Molecular realization of neural network

Molecular device

• Use conductive polymers and other compounds
• Using biopolymers and native proteins

The next chapter Portal: "Principles of Artificial Neural Networks" Reading Notes (10)-The Development of Artificial Neural Networks