Reference
Long short-term memory (Long short-term memory, LSTM) is a special RNN, mainly to solve the problem of gradient disappearance and gradient explosion in the training process of long sequences.
Basic input and output of LSTM
As shown in the figure, different from ordinary RNN, LSTM has two transfer states, namely ct and htc^t and h^tct andht , where c changes slowly, and h can change greatly.
zi, zf, zo, z four states z^i,z^f,z^o,z four states withi,withf,withthe ,z four th state condition
The calculation method is as shown in the figure above, where xt and ht − 1 are joined to each other to form a higher-dimensional vector and then with various W. Calculate x^t and h^{t-1} each other to form a higher-dimensional vector and then the same. W calculationxt andhT - . 1 relative tothe crosspiececonnectedtoathdimensionofmorehigherintheamountsand thenwithvariouskinds ofWdollarscalculated
Calculation
Perform calculations as shown
