1. Neuron - the basic unit of thinking
We know that on the one hand, computers are amazing for their speed, but they are powerless for object recognition and complex tasks. On the other hand, humans are not as fast as computers, but they can do many extremely complex things. A fly has about 100,000 neurons to complete complex tasks such as flying, finding food, and avoiding natural enemies. A nematode has only 302 neurons, but it can perform quite useful tasks such as squirming, but these tasks are difficult for a fast computer to complete.
Here is a graph of neurons:
Consists of dendrites , cell body , axon , myelin , and synapses at the axon terminal .
Pictures from the Internet
The picture below shows the neuron connection diagram of Caenorhabditis elegans, that is, the neural network of the nematode, which shows that its structure is quite complicated.
Caenorhabditis elegans, picture from Nature, 2019, doi:10.1038/s41586-019-1352-7.
Biologists have found that electrical and chemical signals are transmitted between neurons. For example, when you get an injection, the needle will hurt you, and the signal from the needle is transmitted through the skin to the brain nerves. The pain factor of needling only activates nociceptors , which are converted into pain signals and transmitted to the dorsal horn of the spinal cord through the central process of dorsal root ganglion neurons , where the projection neurons form an ascending conduction pathway, which passes through the brainstem, thalamus, etc. The secondary relay reaches the cortex to produce pain sensation. That is to say, the pain sensation can only be activated when the degree of needle pricking reaches a certain threshold .
The picture below is a comparison between biological neurons and artificial neurons . Are there many similarities?
For example: a dendrite is like an input , an axon is like an output port , each terminal synapse of the axon is like an output , and a neuron cell body is like an activation function , because the signal input to the cell body only reaches a certain level Threshold cell bodies are only processed, in addition to neuron cell bodies may have other processing.
The picture comes from the Internet
Inspired by neurons, people invented artificial neurons . As shown in the figure below, the basic idea is: the signals of all input neurons are superimposed at the same time according to a certain weight, and then the output of the neuron will be triggered only when the superimposed sum signal exceeds a certain threshold.
Describe an artificial neuron with the formula:
1.output is the output of the neuron;
2.x1——xn is the n input of the neuron;
3.w1——wn is the weight of the neuron corresponding to n inputs;
4.b is the offset affected by the outside world;
5.f is the activation function, which is a function that can control the output;
So in essence, an artificial neuron is an n-element linear function plus a nonlinear processing.
Image source network
2. Neural Networks – Where Intelligence Happens
The following picture shows the color map of the neural network of the human brain and the schematic diagram of the human brain. Hundreds of millions of neurons form a huge neural network with various connections, so that people are at the top of the biological wisdom.
Image source network
Artificial neural networks are produced by simulating biological or human neural networks. The following is a 7- layer artificial neural network and a 3-layer artificial neural network.
人工神经元连接到一起形成了人工神经网络，信号从神经网络的最左侧流向最右侧，即信号从输入层输入，穿过中间层，最终流向输出层。
前面说过，每一个神经元相当便于一个n元的线性函数加一个非线性的处理，那么人工神经网络就是多层的多个n元的线性函数加一个非线性的处理的多次叠加与非线性处理，这样想的话人工神经网络确实是一个很复杂的东西，让人很难理解，不过我们不用想那么多，就这么个东西却有很大的用处。
我们以最简单的二层人工神经网络来理解，如下图所示，第一层有3个人工神经元，第二层有2个人工神经元。
信号流向从第一层流向第二层最后输出，可见第一层的输出对于第二层就是输入，所以
第二层的第一个神经元的输出可以写为：
类似第二层的第二个神经元输出可以写为：
其中，w1为第一层第一个神经元与第二层第一个神经元之间的连接权重，其余权重以相同的方式理解。为了便于记忆：我们把第1层第i个神经元与第二层第j个神经元的连接权重用
表示，把第一层第i个神经元的输出用
表示，第二层第j个神经元的输出用
表示,那么下面的二层神经网络输出可以表示为：
仔细观察这两个表达式，忽略掉g激活函数，感觉有些像线性代数里的线性方程组。
我们知道线性方程组可以用矩阵来描述，所以二层神经网络的输出是否可以用矩阵来描述呢？答案是肯定的。
使用矩阵描述二层神经网络如下：
简写为：
Inspired by the two-layer neural network, can more layers of neural networks be described by matrices? Obviously it's also possible. Because 3 layers is an extension of 2 layers, and more layers are an extension of 3 layers, matrix multiplication can represent a multilayer neural network .
Due to the space problem, the subsequent neural network algorithm and parameter content will be explained in the second part, and the third part will describe the python implementation code of the neural network and the training and testing situation.
Handwritten neural network source code download address: https://download.csdn.net/download/xipengbozai/19781210?spm=1001.2014.3001.5501
Reprinted from https://blog.csdn.net/xipengbozai/article/details/118115444