[Basic knowledge of perceptron and neural network (introduction)]

Basic knowledge of perceptron and neural network (introduction)

Look at the definition first:

Artificial Neural Networks (ANNs for short), also referred to as Neural Networks (NNs) for short, insert a picture description here
or a connection model (Connection Model), which imitates the behavioral characteristics of animal neural networks and performs distributed parallel Algorithmic mathematical models of information processing. This kind of network depends on the complexity of the system, and achieves the purpose of processing information by adjusting the interconnection relationship between a large number of internal nodes.

• The basic unit of human neural network is——Neurons
• The basic unit of artificial intelligence neural network is——Perceptron
  

---

What is a perceptron?

Look at the definition

formula first, as shown in the figure below,

the perceptron is composed oflinear functionandactivation functionComposition, we give an example as follows:

We have a set of two-dimensional data of height and weight, punctuated in the Cartesian coordinate system. As the data increases, we can see that the data is roughly divided into two parts, the "thin" one in the upper left corner and the "fat" one in the lower right corner. Then can we model the data and find a straight line y = kx + b in the plane to distinguish these data? If there is such a straight line or what we calllinear functionAfter being fitted, we only need to have a set of height and weight data in the future and bring it in to judge whether the function value is >0 or <0, and we can know whether it is "fat" or "thin". This constitutes the most basic perceptron. By constructing the linear functionax+by+c=0, bring in the two-dimensional data, judge the positive and negative, and then classify the two-dimensional space of the Cartesian coordinate system. As shown in the figure above, the left side of the linear function is "thin", and the right side is "fat". fn = (W ^T T) we callactivation function, which is used to classify the results judged by the linear function.

give another example

Also construct a linear function in two-dimensional spaceax+bx+c=0, then useactivation functionSeparate the prices.

In the same way,
our previous two examples are to divide the space in the plane, transform the actual real problem into an abstract mathematical problem, and project the real problem into the space for classification and solution.
But the actual problem may be more complicated than this, such as the three-dimensional problem in the figure below:

When the input data is three-dimensional data, the linear function constructs a three-dimensional space, and the activation function classifies the data in the space;

If you do not have a good foundation in linear algebra, you may not understand this formula well. The explanation is as follows:

W ^T matrix is ​​the parameter matrix of a linear function. UseW^T*XThe form of the inner product of two matrices represents an N-dimensional linear function.

So what happens when the perceptron dimension increases? Not just two-dimensional, three-dimensional?
For example, there are many similar ones:

But we have found a problem so far. One is that our linear function currently solves only linear classification problems (straight line or plane), and cannot handle more complex classifications, and the activation function judgment is only (+1 or -1) that is ("Yes " or "no"), and the real situation is not simply black and white. For example, if we input housing price data, we hope to conclude that this data is more likely to be the housing price of "Beijing" or "Changzhi".

Therefore, to solve this problem, we introduced theA new activation function, this is a super evolution of the perceptron !
**As shown in the figure below:** We have introduced a new activation functionsigmoid

1. The sigmoid function is key to understanding how neural networks learn complex problems, solvingnonlinear separable problem
2. The sigmoid function can also be used ascompression function, because its domain is the set of all real numbers, and its range is (0,1). So if the input to the function is a very large negative number or a very large positive number, the output will always be between 0 and 1. The same goes for any number between -∞ and +∞.
3. The sigmoid function is used as the neural networkactivation function. To review what an activation function neuron is, the diagram below shows the role an activation function neuron plays in a layer of a neural network. The weighted sum of the inputs is passed through an activation function, and this output is used as the input to the next layer.

The sigmoid function formula

solves nonlinear separable problems

So having said so much, what does it have to do with neural networks?
Let's continue with the example:

Is it a bit like a neural network?
Project the input N-dimensional X data into the corresponding high-dimensional space for classification!

Then we discovered the essence of machine learning:y=σ(wx+b)

y=σ(wx+b), a complex problem in the real world is projected into a high-dimensional space through nonlinear transformation for classification and discrimination! (Take something that seems unreasonable, and make it reasonable by some means (training model))

What is the essence of nonlinear transformation?

By changing the position coordinates in space, any point can be found in the dimensional space. Through some means, an unreasonable point (unreasonable position) becomes reasonable

Examples are as follows:

Neural Networks

Input data, classify layer by layer, and finally judge!