理解自动求导
- 例子
def f(x):
a = x * x
b = x * a
c = a + b
return c基于图理解
代码实现
def df(x):
# forward pass
a = x * x
b = x * a
c = a + b
# backward pass
g_c = 1
# c = a + b
g_a = g_c * 1
g_b = g_c * 1
# b = x * a
g_x = g_b * a
g_a += g_b * x
# a = x * x
g_x += g_a * 2x
return g_x- 进一步, 如果前后两节点都是向量, 相应的导数改为Jacobi矩阵, 导数改为梯度.