In nlp engineering, text preprocessing process generally comprises the steps of: obtaining an original text, word, text cleaning, standardization, feature extraction, modeling. Below we will step by step, respectively, for each step of the method commonly used and popular libraries are introduced.
In English on the word, due to the special nature of the language led to the idea of the word will be different. In most cases, the direct use of the English word spaces can be carried out.