版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/weixingstudio/article/details/51773524
从上一篇原创的文章到现在，已经有一年多了，目前终于有一些新的总结分享个大家。
本文主要讲了RNN神经网络的基本原理，并给出了RNN神经网络参数更新的详细推导过程（back propagation），对于想知道RNN的参数是如果推导的，可以仔细阅读本文。
因为时间有限，下面的总结难免有疏漏之处，请大家指正。
本文结合了一个非常经典的RNN的例子代码，进行了详细的说明，RNN的代码以及注释请见：https://github.com/weixsong/min-char-rnn
并且，本文给出了验证为什么在output layer采用sigmoid激活函数的时候应该采用cross entropy error作为cost function。
本文目录：
1.Introduction
2.Simple RNN Theory
3. Using RNN to predict next character
4. Loss Function
4.1 Sum of Squared error (Quadratic error)
4.2 Cross Entropy Error
5. Forward Propagation
6. Quadratic Error VS Cross Entropy Error
6.1 Derivative of error with regard to the output of output layer
6.2 Derivative of error with regard to the input of output layer
7. Error Measure of RNN for Character Prediction
8. Back Propagation
8.1 compute the derivative of error with regard to the output of outputlayer
8.2 compute the derivative of error with regard to the input of outputlayer
8.3 compute the derivative of error with regard to the weight betweenhidden layer and output layer
8.4 compute the derivative of error with regard to the output of hiddenlayer
8.5 Compute the derivative of error with regard to the input of hiddenlayer
8.6 compute the derivative of error with regard to the weight between inputlayer and hidden layer
8.7 compute the derivative of error with regard to the weight betweenhidden layer and hidden layer
