Detailed mathematical derivation of regression of a linear function

Hypothetical premise: All data have the same distribution (conform to the same function)
function to be fitted: $$y^{\ast }=kx+bError$$
function (L2 norm):$$L(y,y^{\ast })=(y^{\ast }-y{)}^2=(kx+b-y{)}^{2.}$$
a$a=b-y$ , the function is:
$$L(w)=k^{2x}{\_}^2+2akx+a^2$$
The lowest point of the L(w) function is
$$\frac{4x{\_}^{2a}{\_}^2-4a^{2x}{\_}^2}{4x{\_}^2}=0$$
This simple function can directly calculate the result:
$$w=-\frac{2ax}{2x{\_}^2}=-\frac{a}{x}$$
Therefore, there is a w that makes the L function take 0, and the error is the smallest at this time.
Similarly, the solution to b is the same
$$L(b)=b^2+2(kx-y)b+(kx-y{)}^2$$
The lowest point of the L(b) function is
$$\frac{4(kx-y{)}^2-4(kx-y{)}^2}{4}=0$$
This simple function can directly calculate the result:
$$b=-\frac{2(kx-y)}{2}=y-kx$$
But according to complex functions, it is often difficult to judge its extreme points, so consider gradient descent.
Partial derivative of L with respect to k:
$$\frac{d(kx+a{)}^2}{dk}=\frac{d(k^{2x}{\_}^2+2akx+a^2)}{dk}=2x{\_}^{2}k+2ax$$
In the same way, the partial derivative of L with respect to b:
$$\frac{d(b+kx-y{)}^2}{db}=2b+2(kx-y)$$
For any k, its partial derivative can be obtained. If its partial derivative>0, the formula for updating k is (η is the learning rate) k ∗ = k −$$k^{\ast }=k-theDk=k-\eta (2x^{2}k+2ax)$$
Similarly, the update formula of b can be obtained as follows:
$$k^{\ast }=k-\eta ▽k=k-h(2b+2(kx-y\left)\right)$$
Then a large amount of learning is carried out at an appropriate learning rate, and the function will slowly approach the optimal solution.
This is the idea of ​​using linear function fitting. If you increase the parameters, increase the order, etc., the situation of model fitting will become more complicated, and of course the effect will be better.