In natural language processing, word of the corpus is one of the most basic and critical step. In the commonly used word python package has jieba, snownlp, thulac, pkuseg like. Specific reference can be used in official documents.
jieba: https://github.com/fxsjy/jieba
pkuseg:https://github.com/lancopku/pkuseg-python
snownlp:https://github.com/isnowfy/snownlp
Thulach: https://github.com/thunlp/THULAC