Analysis of the situation where the batchsize of multiple linear regression is N

I introduced the case of unary linear regression batchsize=1 and N, now let's discuss the scalar yy label in multiple linear $y$ , there are M attributes, use{ x 1 , ⋯ , xi } i ∈ M \{x_1,\cdots,x_i\} i \in M{ x1​,⋯,xi​}i∈M means that there are M parameters in this way, use$w_1,\cdots ,w_i$express. The specific representation is as follows:
$$y=b+w_1{x}_1+\cdots +w_i{x}_i$$
In order to express it more concisely with a vector, the parameter $b$ as$w_0$, so that the expression can be written as:
$$\begin{array}{r}y=w_0+w_1{x}_1+\cdots +w_i{x}_i\end{array}$$
A vector can be used to represent w = { w 0 , ⋯ , wi } T \boldsymbol{w}=\{w_0,\cdots,w_i\}^Tw={ w0​,⋯,wi​}T， x = { 1 , x 1 , ⋯   , x i } T \boldsymbol{x}=\{1,x_1,\cdots,x_i\}^T x={ 1,x1​,⋯,xi​}T , such that$y={\mathbf{w}}^{T}x$.
For the sake of simplicity, we still consider the case where the batchsize is 1, then the loss function$L$ is expressed as:
$$\begin{array}{rl}L& =\frac{1}{2}(y-y^{\ast }{)}^2\\ & =\frac{1}{2}(w_0+w_1{x}_1^{\ast }+\cdots +w_i{x}_i^{\ast }-y^{\ast }{)}^2\end{array}$$
Loss function $L$ versus$The\; components\; of\; w$ are partial derivatives:
$$\frac{\partial L}{\partial w_0}=(w_0+w_1{x}_1^{\ast }+\cdots +w_i{x}_i^{\ast }-y^{\ast })\ast 1$$
$$\frac{\partial L}{\partial w_1}=(w_0+w_1{x}_1^{\ast }+\cdots +w_i{x}_i^{\ast }-y^{\ast })x_1^{\ast }$$
…
$$\frac{\partial L}{\partial w_i}=(w_0+w_1{x}_1^{\ast }+\cdots +w_i{x}_i^{\ast }-y^{\ast })x_i^{\ast }$$
So the loss function about $\mathbf{w}$的梯度为：
∇ L = { ∂ L ∂ w 0 , ⋯   , ∂ L ∂ w i } T = ( w T x ∗ − y ∗ ) x ∗ \begin{aligned} \nabla L&=\{\frac{\partial{L}}{\partial{w_0}},\cdots,\frac{\partial{L}}{\partial{w_i}}\}^T\\ &=(\boldsymbol{w}^T\boldsymbol{x}^*-y^*)\boldsymbol{x}^* \end{aligned} ∇L​={ ∂w0​∂L​,⋯,∂wi​∂L​}T=(wTx∗−y∗)x∗​
Set the step size step, and the parameter update method is as follows:
$${\mathbf{w}}_{new}=w-step\ast \nabla L$$
Next consider the case where the batchsize is N, then the loss function$L$可表示为：
$$\begin{array}{rl}L& =\sum _{j=1}^N\frac{1}{2}(y^j-y^{j\ast }{)}^2\\ & =\sum _{j=1}^N\frac{1}{2}(w_0+w_1{x}_1^{j\ast }+\cdots +w_i{x}_i^{j\ast }-y^{j\ast }{)}^2\\ & =\sum _{j=1}^N\frac{1}{2}({{\mathbf{x}}^{\ast }}^T\mathbf{w}-y^{j\ast }{)}^2\\ & =\frac{1}{2}(A\mathbf{w}-{\mathbf{y}}^{\ast }{)}^T(Aw-y^{\ast })\end{array}$$

Loss function $L$ versus$The\; components\; of\; w$ are partial derivatives respectively:
$$\begin{array}{rl}\frac{\partial L}{\partial w_0}& =\sum _{j=1}^N(w_0+w_1{x}_1^{j\ast }+\cdots +w_i{x}_i^{j\ast }-y^{j\ast })\ast 1\\ & =\sum _{j=1}^N{w}_0+\sum _{j=1}^N{w}_1{x}_1^{j\ast }+\cdots +\sum _{j=1}^N{w}_i{x}_i^{j\ast }-\sum _{j=1}^N{y}^{j\ast }\\ & =w_0{\mathbf{e}}^{That\text{'}s}it+w_1{e}^T{\mathbf{x}}_1^{\ast }+\cdots +w_i{e}^T{x}_i^{\ast }-e^T{\mathbf{y}}^{\ast }\\ & =e^T(w_{0}e+w_1{x}_1^{\ast }+\cdots +w_i{x}_i^{\ast }-y^{\ast })\\ & =e^T([\mathbf{e},x_1^{\ast },\cdots ,x_i^{\ast }]\mathbf{w}-y^{\ast })\\ & =e^T(Aw-y^{\ast })\end{array}$$
$$\begin{array}{rl}\frac{\partial L}{\partial w_1}& =\sum _{j=1}^N(w_0+w_1{x}_1^{j\ast }+\cdots +w_i{x}_i^{j\ast }-y^{j\ast })x_1^{j\ast }\\ & =\sum _{j=1}^N{w}_0{x}_1^{j\ast }+\sum _{j=1}^N{w}_1{x}_1^{j\ast }{x}_1^{j\ast }+\cdots +\sum _{j=1}^N{w}_i{x}_i^{j\ast }{x}_1^{j\ast }-\sum _{j=1}^N{y}^{j\ast }{x}_1^{j\ast }\\ & =w_0{x}_1^{\ast \mathbf{T}}\mathbf{e}+w_1{x_1}^{\ast T}{\mathbf{x}}_1^{\ast }+\cdots +w_i{x_1}^{\ast T}{x}_i^{\ast }-{x_1}^{\ast T}{\mathbf{y}}^{\ast }\\ & ={x_1}^{\ast T}(w_{0}e+w_1{x}_1^{\ast }+\cdots +w_i{x}_i^{\ast }-y^{\ast })\\ & ={x_1}^{\ast T}([\mathbf{e},x_1^{\ast },\cdots ,x_i^{\ast }]\mathbf{w}-y^{\ast })\\ & ={x_1}^{\ast T}(A\mathbf{w}-y^{\ast })\end{array}$$
The same method can be used to find the partial derivatives of other components
$$\frac{\partial L}{\partial w_i}={{\mathbf{x}}_i}^{\ast T}(A\mathbf{w}-{\mathbf{y}}^{\ast })$$
其中$A=[\mathbf{e},x_1^{\ast },\cdots ,x_{\mathbf{i}}^{\ast }]$ For the x-value matrix of each batch, a column of all 1s is added to the first column. The loss function is about$\mathbf{w}$的梯度为：
∇ L = { ∂ L ∂ w 0 , ⋯   , ∂ L ∂ w i } T = [ e T , x 1 ∗ T , ⋯   , x i ∗ T ] T ( A w − y ∗ ) = A T ( A w − y ∗ ) \begin{aligned} \nabla L&=\{\frac{\partial{L}}{\partial{w_0}},\cdots,\frac{\partial{L}}{\partial{w_i}}\}^T\\ &=[\boldsymbol{e}^T,\boldsymbol{x_1^*}^T,\cdots,\boldsymbol{x_i^*}^T]^T(A\boldsymbol{w}-\boldsymbol{y^*})\\ &=A^T(A\boldsymbol{w}-\boldsymbol{y^*}) \end{aligned} ∇L​={ ∂w0​∂L​,⋯,∂wi​∂L​}T=[eT,x1∗​T,⋯,xi∗​T]T (Aw−y∗)=AT (Aw−y∗)​
Set the step size step, and the parameter update method is as follows:
$${\mathbf{w}}_{new}=w-step\ast \nabla L$$
Using matrix and vector forms can easily implement multiple linear regression with numpy:

x = np.array([0.1,1.2,2.1,3.8,4.1,5.4,6.2,7.1,8.2,9.3,10.4,11.2,12.3,13.8,14.9,15.5,16.2,17.1,18.5,19.2,0.1,1.2,2.1,3.8,4.1,5.4,6.2,7.1,8.2,9.3,10.4,11.2,12.3,13.8,14.9,15.5,16.2,17.1,18.5,19.2])
y = np.array([5.7,8.8,10.8,11.4,13.1,16.6,17.3,19.4,21.8,23.1,25.1,29.2,29.9,31.8,32.3,36.5,39.1,38.4,44.2,43.4])
x = x.reshape(2,int(len(x)/2)).T
x = np.insert(arr=x,values=[1],obj=0,axis=1)
y = y.reshape(1,len(y)).T

The regression process is as follows:

# 设定步长
step=0.001
# 存储每轮损失的loss数组
loss_list=[]
# 定义epoch
epoch=500
# 定义batch_size
batch_size=12
# 定义单位列向量e
e=np.ones(batch_size).reshape(batch_size,1)
# 定义参数w和b并初始化
w=np.zeros(3).reshape(3,1)
#梯度下降回归
for i in range(epoch) :
    #计算当前输入x和标签y的索引，由于x和y数组长度一致，因此通过i整除x的长度即可获得当前索引
    index = i % int(len(x)/batch_size)
    # 当前轮次的x列向量值为：
    cx=x[index*batch_size:(index+1)*batch_size]
    # 当前轮次的y列向量值为：
    cy=y[index*batch_size:(index+1)*batch_size]
    # 计算当前loss
    loss_list.append(float(1/2*(cx.dot(w)-cy).T.dot(cx.dot(w)-cy)))
    # 计算参数w的梯度
    grad_w = cx.T.dot(cx.dot(w)-cy)
    # 更新w的值
    w -= step*grad_w
print(loss_list)
plt.plot(loss_list)
plt.show()
print(w)