Encoder Decoder Architecture
Subsequent natural language processing is basically implemented using this architecture
Seq2Seq was first used for machine translation, and now Bert is used more. Seq2Seq is an Encoder-Decoder architecture. The encoder of the Seq2Seq model uses Rnn, and the decoder is also Rnn. The encoder passes the last hidden state to the decoder.
Bidirectional Rnn is often used in the encoder (look forward to a sentence and look in the reverse direction), the decoder needs to predict, the decoder does not need bidirectional, and the decoder is also an Rnn output.
Seq2Seq training and inference are not the same
