RNN usually cannot use dropout, because the weight of RNN has a cumulative effect. If dropout is used, it will destroy the learning process of RNN.
However, Google Brain published an article researching this in 15 years: recurrent neural network regularization
They used dropout in the acyclic phase to improve the overfitting phenomenon
The paper uses dropout in two places. The dotted line in the figure uses dropout, and the solid line is not used.
There is no cumulative effect of weights in the non-cyclic phase, which will not destroy the learning process of RNN
In order to see more clearly, draw a picture:
The dropout and recurrent_dropout in the figure are both dropout, but the job is in a different place, and keras also sets parameters for it:
model.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2))
and we can also use dropout in the non-RNN stage, such as this:
model.add(..)
model.add(LSTM(10))
model.add(Dropout(0.5))The above code means that dropout can be used after executing LSTM
