LSTM and GRU presentation

1, ordinary RNN

　　Neural network processing sequence data.

2、LSTM

　　LSTM full name is long short-term memory short and long term memory, is a special RNN, mainly to solve the long training sequence during the gradient disappears and gradient explosion.

　　Compared to ordinary RNN, LSTM be able to perform better in the longer sequence.

LSTM inside there are three main stages:

(1) Forget stage;

　　For a node passed in input selectively forgotten. "Forget not important, important to remember."

By gating forget achieved.

(2) select the memory stage;

　　Will enter this phase are selectively "memory." What are the key focuses has been recorded, and what is not mind some of the less.

(3) output stage.

3, summary