Understand RNN in two minutes

This article refers to Mr. Li Hongyi's online class to give my own understanding.

1. RNN

The advantage of RNN is that it has a memory function, which can affect the output at this moment according to the input at the previous moment. The moments in this sentence can be seen in a broad sense. Different words in a sentence are equivalent to different moments, and different frames of a picture can also be regarded as different moments.

I think this picture is the best picture to understand RNN.
To be clear: (I think you can understand RNN by knowing these points)
1. $x_1、x_2、x_3$are three words in a sentence, $y_1、y_2、y_3$is the output obtained by inputting these three words into the network.
2. Each green block is the same network, not three different networks. The three green blocks represent the same network at three moments.
3. The memory block will store $x_1$The memory information output in the green block $a_1$, this memory information works together with the next word to output memory $a_2$和$y_2$.
4. The green block contains many neurons but is ignored as a block.
5. The green block is called the hidden layer, $a_i$give $y_i$The biggest difference is that the former does not pass the activation function.
6. The weight W of the memory block is constant in each word, and is only updated once in each sentence.

Then through these 6 points, we can know that RNN is a network with memory, and different words are output in sequence after cycling in this network .

I believe that this minimalist diagram can be understood through the above five points. In the same network , the vectors in Input are input sequentially , and each input is accompanied by an output and a memory. This memory will affect the output value of the next vector.