Introductory Learning of Neural Networks - The Construction of Neural Networks
Completion date: 2018.04.24
construct a neural network
Each biological neuron in the organism has a different function. When these different types of neurons are connected according to a certain structure, it becomes a neural network. We imitate the human brain and regard the artificial neural network as a kind of computing model, which is composed of a large number of nodes (or neurons) connected to each other. Each node represents a specific output function. Each connection between two nodes represents a weighted value of the signal passing through the connection, called the weight, which is equivalent to the memory of the artificial neural network. The output of the network varies depending on how the network is connected, the weight values and the excitation function.
1. Linear inseparability
We have used perceptrons to solve logical AND and logical OR operations, but found that perceptrons cannot solve the problem of logical XOR, because logical XOR is linearly inseparable.
So what is linear inseparability?
Linearly separable means that two kinds of things can be distinguished by a straight line.
Linear inseparability means that two kinds of things cannot be distinguished by a straight line.
As shown in the figure, AND and OR are linearly separable, while XOR is linearly inseparable.
Therefore, to solve the problem of XOR, one more layer needs to be added after the perceptron (neural network).
2. Multilayer Neural Network
(1) Perceptron : The perceptron is a two-layer neural network, one layer is the input layer (we simplify the above to input stimuli, that is, the input node does not participate in the calculation), and the other layer has a computing unit, which can be established through supervised learning. The ability to discriminate patterns.
(2) Multi-layer neural network (also called feed-forward neural network) : The layer composed of the input signal source, hidden layer and output is called a multi-layer neural network. An important feature is that the output of the upper layer can only be the lower layer. The input of one layer cannot be linked across layers.
Features: Each neuron of the feedforward network receives the input of the previous stage and outputs it to the next stage without feedback.
Node: Input Node, Output Node.
(3) Computing unit : It can have any input, but only one output, and the output can be coupled to the inputs of any number of other nodes.
(4) Layer : visible layer - input and output nodes; hidden layer - middle layer.
3. Solve the XOR problem (solve the linear inseparability) - use the BP algorithm to adjust the parameters
Using a two-layer neural network can memorize the XOR operation, as shown in the figure:
The neuron on the upper side of the first layer has a net input: 2p1 + 2p2 - 1
The relationship between its net input/output and p1, p2 is shown in the figure:
| p1/p2 | 0 | 1 |
|---|---|---|
| 0 | -1 step(-1)=0 | 1 step(1)=1 |
| 1 | 1 step(1)=1 | 3 step(3)=1 |
The neuron on the lower side of the first layer has a net input: -2p1 - 2p2 + 3.
The relationship between its net input/output and p1 and p2 is shown in the figure:
| p1/p2 | 0 | 1 |
|---|---|---|
| 0 | 3 step(3)=1 | 1 step(1)=1 |
| 1 | 1 step(1)=1 | -1 step(-1)=0 |
Finally, the data of the two neurons is integrated by the output neuron, and the logical AND operation is used here, so that the memory of the logical XOR can be realized.