Given point set Points (x, Y) , assume that the regression equation we require is of the form f (x) = w * x + b a linear equation form, then each x can correspond in the case of w and b are determined estimate a value F (X) , namely linear regression determined the most appropriate w and b, can find out that the function of the trend prediction regression equation.
Thinking about the first stroke:
Suppose a variable LOSS (meaning loss), LOSS = F (X) -Y = X + W * by . So LOSS meaning is the real difference equations in your seeking out the regression equation. LOSS smaller, the regression equation closer to the real equation.
Thinking generally divided into two steps:
1. obtaining each of w, LOSS the value of B
2. seek out a minimum value w0 LOSS, b0
this time w0, w b0 is the most appropriate regression equation, b
We now know how to seek out LOSS In the case of known w and b, so the first step has been resolved, the next step is how to find out what time the minimum LOSS.
This time had to introduce gradient concept, intended to point gradient direction function increased .
If we invert the gradient, then you can find the minimum .
As shown
in Figure gradients of these red dots pointing in the direction of the next to fall
so if the use of minimum gradient to find it?
If the figure is a function of x, y
then we know that, if f '(x) <0, let x to the right, if f' (x)> 0, let x left
so will be able to find y = f ( x) the minimum
Similarly, if we are to find LOSS minimum on w, b are
then respectively w and b derivative of
the derivative is ∂LOSS / ∂w and ∂LOSS / ∂b respectively
which ∂LOSS / ∂w sigma] 2 = (W XI-B + Yi) XI
∂LOSS / ∂b = sigma] 2 (W XI-Yi + B)
And w b and w as follows and that the operation can move towards the minimum in the direction b LOSS
w '= w - (∂LOSS / ∂w)
b' = b - (∂LOSS / ∂b)
But this time there is a problem, and that is likely to be very large range, in that case is likely to make very accurate equation when w and b further above operations.
In this, we turn changes the equation in the following format:
W '= W - lr * (∂LOSS / ∂w)
B' = B - lr * (∂LOSS / ∂b)
Meaning wherein lr is ** (learn rate ) ratio learning **, general initially set to 0.001 or 0.005 , after their own debugging will see how LOSS fall until satisfactory value.
Code is as follows: 
the final result is output 
wherein the first output is assumed that w and b 0 is the initial value of 5565 LOSS
second output after the return of 0.7037 w and b LOSS size 37
at this time has LOSS It dropped from 147 to about 5565, deviation reduced 40-fold!
