[Example screenshot]
File: n459.com/file/25127180-478966531
The following are irrelevant:
-------------------------------------------Dividing line----- ----------------------------------------
We put a picture in the article at that time. This picture is a picture of a multilayer perceptron. Take a look at it, it is the picture below.
This picture is not a problem at first glance, but when you think about it, it will feel a little strange. It seems that the picture of the neural network we see in our impression is also like this. In that case, what is the difference between them?
The most obvious difference on the surface is the name. This is a picture of a neural network. We found that there are also three layers, but the names of each layer are input layer, intermediate layer (hidden layer) and output layer. We generally name the input layer and output layer separately, and the layers in between are called hidden layers or intermediate layers. Of course, like the perceptron, it is also possible to name the number of layers. For example, the input layer in the figure below is called layer 0, the middle layer is called layer 1, and the final output layer is called layer 2.
We generally do not regard the output layer as an effective neural network, so the network in the following figure is called a two-layer neural network instead of a three-layer neural network.
In addition to the different names, the key difference is the activation function. To illustrate this point, let's take a look at the signal transmission in the neural network.
Signal transmission
The figure below is a neural network diagram I found randomly. We can see that the first node of the input is set to 1. This is done to facilitate the introduction of the offset, but we usually do not draw the offset deliberately when drawing pictures. Let's take the following picture as an example to see how signals are transmitted in neural networks.
Let's take an example, we can try to write the expression, it has a total of three inputs, which are 1,, and then we can also see the weight corresponding to each input, so we can finally write:
It's not over here, each layer in the neural network will have a corresponding activation function. Under normal circumstances, the activation function in the same layer of the network is the same, we call it h, so the final output of this node is not just obtained, but.
We are already familiar with the activation function. We have introduced it many times before. There are probably the following commonly used ones: Relu, Sigmoid, tanh, softmax, and some derived variants. Under normal circumstances, we usually use Relu before the output layer. If the model is a classification model, we will use Sigmoid or softmax at the end. If it is a regression model, we will not use any activation function.
We are already familiar with Sigmoid. If we regard the LR model as a single-layer neural network, then Sigmoid is its activation function. Sigmoid is applied to a single output node in a two-class scene. If the output value is greater than 0.5, it is true, otherwise it is false. In some probability estimation scenarios, the output value can also be considered to represent the probability of an event.
Corresponding to it is the softmax function, which is used in multi-classification problems. The number of nodes it uses is not 1, but k. Here k represents the number of categories in the multi-category scene. Let's take k=3 as an example, look at the following figure:
There are three nodes in the graph. For each node, its formula can be written as:
In fact, the calculation method is the same as Sigmoid, except that a weight is calculated in the end. Finally, we will choose the largest among these k nodes as the final classification result.
Code Implementation
Finally, let's try to write the code of the neural network. Since we have not introduced the training method of the neural network yet, we can only realize the part of its prediction. After we have introduced the backpropagation algorithm, let's make up the process of model training.
If backpropagation is not considered, the code of the entire algorithm is actually very simple, as long as students who are familiar with Python syntax can understand it.
import numpy as np
def relu(x):
return np.where(x > 0, x, 0)
def sigmoid(x):
return 1 / (1 + np.exp(-x))
class NeuralNetwork():
def init(self):
self.params = {}
self.params[‘W1’] = np.random.rand(2, 3)
self.params[‘b1’] = np.random.rand(1, 3)
self.params[‘W2’] = np.random.rand(3, 2)
self.params[‘b2’] = np.random.rand(1, 2)
self.params[‘W3’] = np.random.rand(2, 1)
self.params[‘b3’] = np.random.rand(1, 1)
def forward(self, x):
a1 = np.dot(x, self.params['W1']) + self.params['b1']
z1 = relu(a1)
a2 = np.dot(z1, self.params['W2']) + self.params['b2']
z2 = relu(a2)
a3 = np.dot(z2, self.params['W3']) + self.params['b3']
return np.where(sigmoid(a3) > 0.5, 1, 0)
if name == " main ":
nn =
NeuralNetwork () print(nn.forward(np.array([3, 2]))) In the
next article, we will discuss the training method of neural network, which is famous Backpropagation algorithm to see how it works in a neural network.