What is RNN (Recurrent Neural Network)
Recurrent Neural Network (Recurrent Neural Network), when recognizing images, each input image is isolated. Recognizing this image as an apple will not affect the recognition of the next image as a pear.
But for language, order is very important. "I eat apples" and "apples eat me", the change of word order expresses completely different meanings. The order also provides certain information. For example, "eat" is followed by a noun that represents food with a high probability.
In order to capture this association of data, people found RNN, a network that attaches great importance to sequence information.
Sequence is the context of data. The basic structure of RNN is still neural network, but it has a small box to record the state of the network when data is input. When inputting data next time, the network must consider the information stored in the small box. Information, as the data is input again and again, the stored information is constantly updated, and the information in the box is called the hidden state .
Like the example given at the beginning, the most common application area of RNN is natural language processing.
-
Machine translation is to find the same sequence of meanings expressed in different languages.
-
Poetry generation is based on a theme and outputs a logical sequence of words according to certain rules. Change the type of information at both ends, input a picture and output a sentence, just look at the picture and talk.
-
Speech can also be regarded as a sequence of sound signals in chronological order, and speech recognition and speech generation are also within the capabilities of RNN.
-
The stock price can also be regarded as a time-affected sequence, and the establishment of many quantitative trading models is based on this cognition.
However, RNN still has flaws that cannot be ignored. The earlier the data is input, the less impact it will occupy in the hidden state. That is to say, if a sentence is very long, RNN will forget what it said at the beginning.
So there is an improved version of RNN, LSTM-long short-term memory model.