Author: Zen and the Art of Computer Programming
1 Introduction
overview
Natural language processing (NLP) tasks usually include text classification, sentiment analysis, named entity recognition, etc. Traditional machine learning models such as random forests and support vector machines (SVM) classify or predict by extracting features, but these models often have the following shortcomings:
- Model training time is long
- Requires a lot of data preprocessing
- Model capacity is huge
In order to overcome the above problems, Hochreiter et al. proposed a new method in 2017 - long short-term memory neural network (LSTM), which is an efficient and differentiable way for sequence data, which can learn temporal dependence and maintain state. Therefore, we will build an LSTM neural network from scratch, and practice its training and application in Python language. This article will take the most basic case - digital sequence generation as an example, and teach readers step by step how to build an LSTM neural network and use this model to generate digital sequences.
text
1. Background introduction
LSTM is a method for better learning and prediction of sequence data. It can process data of any length and can handle long-term and short-term correlations at the same time. Dependencies at the sentence level are common in human language processing, such as "quick brown" in "The quick brown fox jumps over the lazy dog" will affect the following verb "jumps", while "the laziest guy in town" does not have this dependency. Therefore, LSTM-based models are expected to play a role in many fields, such as autonomous driving, speech synthesis, machine translation, etc.
2. Explanation of basic concepts and terms
1. Time t (time step)
A time series is a continuous collection of time series data, and each data item is called a time series.