【机器学习总结】向量、矩阵求导公式

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/Hemk340200600/article/details/87969811

关于向量求导用到的公式实在是太多了…经常公式推着推着就被卡住，这里一次性做个总结吧。

0.引言

  正文中，元素使用字母a，b，c等表示，向量使用小写的 $x，y，z$等表示，并且默认是列向量，矩阵使用大写的A,B,C进行表示。

1.向量对元素求导

• 行向量对元素求导
  $\frac{\partial x^T}{\partial a}= \begin{bmatrix} \frac{\partial x_1}{\partial a} , \frac{\partial x_2}{\partial a},... ,\frac{\partial x_n}{\partial a} \end{bmatrix}$
• 列向量对元素求导
  $\frac{\partial x}{\partial a}= \begin{bmatrix} \frac{\partial x_1}{\partial a} \\ \frac{\partial x_2}{\partial a}\\... \\\frac{\partial x_n}{\partial a} \end{bmatrix}$

2.向量对向量求导

• 行向量对列向量求导
  $\frac{\partial y^T}{\partial x}= \begin{bmatrix} \frac{\partial y_1}{\partial x_1} , \frac{\partial y_2}{\partial x_1},... ,\frac{\partial y_n}{\partial x_1} \\ \frac{\partial y_1}{\partial x_2} , \frac{\partial y_2}{\partial x_2},... ,\frac{\partial y_n}{\partial x_2} \\ ...\\ \frac{\partial y_1}{\partial x_n} , \frac{\partial y_2}{\partial x_n},... ,\frac{\partial y_n}{\partial x_n} \end{bmatrix}$
• 列向量对行向量求导
  $\frac{\partial y}{\partial x^T}= \begin{bmatrix} \frac{\partial y_1}{\partial x_1} , \frac{\partial y_1}{\partial x_2},... ,\frac{\partial y_1}{\partial x_n} \\ \frac{\partial y_2}{\partial x_1} , \frac{\partial y_2}{\partial x_2},... ,\frac{\partial y_2}{\partial x_n} \\ ...\\ \frac{\partial y_n}{\partial x_1} , \frac{\partial y_n}{\partial x_2},... ,\frac{\partial y_n}{\partial x_n} \end{bmatrix}$
• 行向量对行向量求导
  $\frac{\partial y^T}{\partial x^T}= \begin{bmatrix} \frac{\partial y_T}{\partial x_1} , \frac{\partial y_T}{\partial x_2},... ,\frac{\partial y_T}{\partial x_n} \end{bmatrix}$
• 列向量对列向量求导
  $\frac{\partial y}{\partial x}= \begin{bmatrix} \frac{\partial y_1}{\partial x} \\ \frac{\partial y_2}{\partial x}\\... \\\frac{\partial y_n}{\partial x} \end{bmatrix}$

3.矩阵对向量求导

• 矩阵对行向量求导
  $\frac{\partial A}{\partial x^T}= \begin{bmatrix} \frac{\partial A}{\partial x_1} , \frac{\partial A}{\partial x_2},... ,\frac{\partial A}{\partial x_n} \end{bmatrix}$
• 矩阵对列向量求导
  $\frac{\partial A}{\partial x}= \begin{bmatrix} \frac{\partial A_{11}}{\partial x} , \frac{\partial A_{12}}{\partial x},... ,\frac{\partial A_{1n}}{\partial x}\\...\\\ \frac{\partial A_{n1}}{\partial x} , \frac{\partial A_{n2}}{\partial x},... ,\frac{\partial A_{nn}}{\partial x} \end{bmatrix}$

4.矩阵复合向量的求导

• $\frac{d}{dx}x^TA=A$
• $\frac{d}{dx^T}Ax=A$
• $\frac{d}{dx}xA=A^T$
• $\frac{d}{dx}Ax=A^T$
• $\frac{d}{dx}x^T=I$
• $\frac{d}{dx^T}x=I$
• $\frac{d}{dx}x^Ty=\frac{d}{dx}y^Tx=y$
• $\frac{d}{dx}x^TAy=xy^T$
• $\frac{d}{dx}x^TAx=xx^T$
• $\frac{d}{dA}x^TA^Ty=yx^T$
• $\frac{d}{dx}x^TAx=(A+A^T)x=2Ax$（当A为对称矩阵时第二个等式成立）
• $\frac{d}{dA}x^Tx=2x$