Machine Learning 0003 Simple Neurons
We have been exposed to mathematics since preschool. We learned one-dimensional linear equations in elementary school. We started to learn quadratic functions of linear functions and quadratic equations of multivariate equations in junior high school. We learned more about equations and functions in high school and university, especially in advanced mathematics. , which covers more in-depth knowledge of functions and equations. Maybe every thinking person has thought: what is the use of learning these? In fact, mathematics is useful to solve all kinds of problems in life: grocery shopping, optimization problems, games... For some programming languages, everything is an object. For mathematics: everything is a function. Free fall s=0.5gt*t, uniform motion s=vt, price=unit price*quantity There are many more, we have to believe that mathematics can solve any problem. A few days ago, my colleague played a very complicated game, and one level was difficult. After watching this game, I found that it can be abstracted into a "graph" (this graph is a concept in graph theory, which is a very important branch of mathematics), and then I passed this level smoothly.
If I knew that there is a lot of mathematical knowledge used in neural networks, I would definitely study advanced numbers and probability theory, and I would hate it when it was time to use them. After reading my description above, you may be confused and still don't understand what I want to do. As mentioned above, all problems are function problems, and we can solve any problem through functions. But what does this have to do with neural networks, neurons, see the following explanation:
We all know the problem of chickens and rabbits in the same cage (if you don’t know, look here: chickens and rabbits share a cage with a total of 10 heads and 30 legs, how many chickens are there?), is a very classic math problem, for many primary school students This is still a difficult problem, and here we use simple neurons to solve this problem that plagues some elementary school students.
There are many normal practices, here is only one:
Let the chicken have x and the rabbit have y
x+y=10
2x+4y=30
Solve x=5, y=5
A neuron solution:
1. What is a neuron
I drew a simple ugly picture, let's take a look:
This is a simple neuron. 3, 12, 5 are the input data, there are some numbers in the neuron these numbers w0, w1, w2...wn, where w0 is the bias and w1 to wn are the weights. Only need to multiply the corresponding input and the corresponding weight and add w0, f(x) is to process the result, which is called the activation function. The activation function determines the type of neuron.
Assumption
The data is A: a1, a2, a3...an
The weights are W: w1, w2, w3...wn
Bias is w0
The output of the neuron is:
f(a1*w1+a2*w2+...+ai*wi+...an*wn + w0)
Simple as that.
We have the activation function f(x)=x here, and there are other types of activation functions, f(x)=max(0,x) f(x)=1/(1+e^-x), etc., theoretically Any differentiable function above can be used as an activation function.
2. Now the problem is converted to
Now our task is to find what the value of w0, w1, w2 should be
3. Calculate Weights and Bias
(1) First randomly initialize the weights and biases to several numbers close to 0
w0 = 0.3
w1=-0.2
w2=0.1
(2) Calculation results
Generate new head count, foot count and chicken count
res=Number of heads*w1+Number of feet*w2+w0
e=number of chickens-res
e represents the difference between the output and the actual result
(3) Update weights and biases
There are 3 values of e:
Note: ai represents the ith input of the neuron
<1> e>0, indicating that the output of the neuron is too small, and the output needs to be increased. At this time, the change of wi is
if ai>0
wi=e*ai+wi At this time, e and ai are positive numbers, and the increase of wi*ai will increase the output of the neuron
if ai<0
wi=e*ai+wi ai is less than 0, e is greater than 0, wi decreases, wi*ai increases
If ai=0, the weight has no effect on the result
wi=e*ai+wi ai=0 -----> wi=wi
<2> e=0, indicating that there is no difference between the output of the neuron and the actual result, and there is no need to update the weights
wi = e * ai + wi Among them e = 0 -----> wi = wi
<3> e<0, it means that the output of the neuron is too large and the output needs to be reduced. At this time, the change of wi is
if ai>0
wi = e * ai + wi e <0, ai> 0, e * ai <0, wi
if ai<0
wi = e * ai + wi e <0, ai <0, e * ai> 0, wi meeting enlargement, wi * ai meeting small
If ai=0, the weight has no effect on the result
wi=e*ai+wi ai=0 -----> wi=wi
To sum up the above wi=e*ai+wi .
For the processing of w0, we need to add an a0
a1*w1+a2*w2+...+ai*wi+...an*wn + w0
From the above formula, it becomes a1*w1+a2*w2+...+ai*wi+...an*wn + a0*w0, obviously a0=1
This way w0=e*a0+w0
(4) Repeat (2)(3)(4) until the result is very close to the correct answer
4. The following is python's implementation of the above algorithm
#chicken and rabbit in the same cage
import random as rd
#(1) Randomly initialize neuron weights
w0 = 0.3
w1=-0.2
w2=0.1
for i in range(10000):
#Here is the loop 2, 3 operation
#(2)Calculation result
#<1> Generate the correct number of heads, legs and chickens
#rabbit number
rabbit=rd.randint(0,10)
#number of chickens
chickens = rd.randint (0,10)
# number of heads
heads=rabbit+chickens
#number of legs
legs=rabbit*4+chickens*2
#<2> Calculate the difference between the neuron output result and the correct result
a1=heads
a2=legs
res=a1*w1+a2*w2+w0
e=chickens-res
#Here is limited to e to avoid too much influence on the weight
if e>1.0:
e=1.0
elif e<-1.0 :
e=-1.0;
e*=0.001
#(3) Update weights and biases
w1=e*a1+w1
w2=e*a2+w2
a0 = 1
w0=e*a0+w0
#The training has been completed, let's start using it
#chicken and rabbit share a cage with 10 heads and 30 legs. How many chickens are there?
heads=10
legs=30
a1=heads
a2=legs
res=a1*w1+a2*w2+w0
print(res,w0,w1,w2)
#Sometimes run output
5.073595116899521 0.11500408173399584 1.991363016946072 -0.4985013044765065
4.96734930028768 0.11104094502477757 1.9890304126201197 -0.5011331923646098
4.991091991393172 0.10861962962525538 1.9875744051914601 -0.4997757230048895 We just need to round the result, res=5
ok