The traditional neural network model, between the nodes in the hidden layer is not shown, connected as in FIG.
Recurrent Neural Networks and between the node connecting the hidden layer, the data sequence is mainly used for classification, prediction process. Connecting means need to receive information, such networks usually used for sequence data processing.
Input the output of the hidden layer only comprises an input layer further comprises an output timing of the hidden layer, i.e., the time information memory is performed before the network will, and applied to the calculation of the current output. RNN sequence structure may have the following process:
The first-many, e.g. speech tagging, input sentence, each word corresponding to the output speech.
The second is a many-such as passage of emotion marked.
The third-many, non-sequential input and output sync, for example in machine translation of one language into another language is output.
The fourth-to-many, for example, an input image, and generates and outputs a text, this text is used to describe the content.
RNN basic structure shown below:
From left to right are three hidden layer, can be folded, he said arrow with a circle represents a connection from the hidden layer.
RNN basic calculation process:
RNN parameters share
In traditional neural networks, the parameters of each layer are not shared. In the RNN in each step shared parameters U, V, W, i.e., the calculation of the output O T +. 1 will be used parameter U, V, W, the calculation of these parameters and values, and output O T when the same , i.e., the figure three U, three V, W of the three values are the same. Share Parameter Description RNN every step of doing the same thing, just enter different. Thus greatly reducing the need to learn the parameters of the network, thereby increasing efficiency.