When I was working on a convolutional neural network, I suddenly found that I didn't know how the neural network was trained, and my face was full of black lines. I took this opportunity to review it.
First put an explanation from a know-it-all. https://www.zhihu.com/question/27239198?rf=24827633
The chain rule is learned when you are a freshman. However, if you directly use the chain rule to calculate redundancy, the complex network cannot accept it. The feature of the bp algorithm is to solve the redundant calculation by reverse calculation. The remaining calculation problem.