Awesome-Repositories-for-Text-Modeling
| repo | paper |
|---|---|
| miracleyoo/DPCNN-TextCNN-Pytorch-Inception | Deep Pyramid Convolutional Neural Networks for Text Categorization |
| Cheneng/DPCNN | Deep Pyramid Convolutional Neural Networks for Text Categorization |
| conv-RNN | A Hybrid Framework for Text Modeling with Convolutional RNN |
| AC-LSTM | |
| - FastText;- BasicCNN (KimCNN,MultiLayerCNN, Multi-perspective CNN);- InceptionCNN;- LSTM (BILSTM, StackLSTM);- LSTM with Attention (Self Attention / Quantum Attention);- Hybrids between CNN and RNN (RCNN, C-LSTM);- Transformer - Attention is all you need;- ConS2S;- Capsule;- Quantum-inspired NN | |
| Double ConvNet;C-LSTM | |
| CNN | Convolutional Neural Networks for Sentence Classification |
| CNN, RNN (LSTM and GRU) | |
| fastText;TextCNN;TextRNN;RCNN;Hierarchical Attention Network;seq2seq with attention;Transformer("Attend Is All You Need");Dynamic Memory Network;EntityNetwork:tracking state of the world; | |
| JerrikEph/Capsule4TextClassification | Information Aggregation via Dynamic Routing for Sequence Encoding |
| SVM | Short Text Classification with One Model for All Languages |
| VDCNN | Very Deep Convolutional Neural Network for Text Classification |
| Sent2Vec (Skip-Thoughts) | |
| Dialogue act tagging classification. | Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks |
| Universal Language Model Fine-tuning (ULMFiT) | Universal Language Model Fine-tuning for Text Classification |
| cvangysel/SERT | Structural Regularities in Text-based Entity Vector Spaces |
| klout/opendata | Actionable and Political Text Classification using Word Embeddings and LSTM |
| jacoxu/STC2 | Self-Taught Convolutional Neural Networks for Short Text Clustering |
| guoyinwang/LEAM | Joint Embedding of Words and Labels for Text Classification |
| abhyudaynj/LSTM-CRF-models | Structured prediction models for RNN based sequence labeling in clinical text |
| A Context-aware Natural Language Generator for Dialogue Systems | UFAL-DSG/tgen |
| hugochan/KATE | KATE: K-Competitive Autoencoder for Text |
| harvardnlp/sa-vae | Improved Variational Autoencoders for Text Modeling using Dilated Convolutions |
| sordonia/hred-qs | A Hierarchical Recurrent Encoder-Decoder For Generative Context-Aware Query Suggestion |
| lanwuwei/SPM_toolkit | Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering |
| salesforce/awd-lstm-lm | |
| cyvius96/adgpm | Rethinking Knowledge Graph Propagation for Zero-Shot Learning |
| alexandres/lexvec | Enhancing the LexVec Distributed Word Representation Model UsingPositional Contexts and External Memory |
| brmson/dataset-sts | Sentence Pair Scoring: Towards Unified Framework for Text Comprehension |
| summanlp/textrank | Variations of the Similarity Function of TextRank for Automated Summarization |
| yufengm/SelfAttentive:binary | A Structured Self-Attentive Sentence Embedding |
| ExplorerFreda/Structured-Self-Attentive-Sentence-Embedding:multi-class | A Structured Self-Attentive Sentence Embedding |
| pandeykartikey/Hierarchical-Attention-Network | Hierarchical Attention Networks for Document Classification |
0x00 中文分词
1)FoolNLTK
slogan：可能不是最快的开源中文分词，但很可能是最准的开源中文分词
2)CWS_Dict
论文"Neural Networks Incorporating Dictionaries for Chinese Word Segmentation", AAAI 2018 源码
3)multi-criteria-cws
多标准中文分词的简单解决方案
4)jieba
结巴中文分词
5)NLPIR-team/NLPIR
NLPIR是一套专门针对原始文本集进行处理和加工的软件，提供了中间件处理效果的可视化展示，也可以作为小规模数据的处理加工工具。
0X01 命名实体识别
1)anaGO
2)golden-horse
3)LTP
4)NeuroNER
使用神经网络实现的命名实体识别，简单易用并达到state-of-the-art的效果。
5)shiyybua/NER
基于tensorflow深度学习的中文的命名实体识别
6)nltk
0x02 词性标注POS-tagging
1)anaGO
2)LTP
3)NLPIR
4)jieba
仓库地址
“结巴”中文分词：做最好的 Python 中文分词组件
5)nltk
0X03 关键词抽取KEYWORD EXTRACTION(FOR DOCUMENT(S))
1)NLPIR(FOR SHROT TEXT AS WELL)
2)jieba
TextRank-based
TFIDF-based
4) aneesha/RAKE
0x04 依存句法DEPENDENCY PARCING
1) LTP
2) nltk
0x05 自然语言理解NLU(SEMANTIC ANALYSIS)
1)anaGO(SRL)
2)RasaHQ/rasa_nlu
将自然语言转换为结构化数据
仓库地址
Rasa NLU (Natural Language Understanding) is a tool for understanding what is being said in short pieces of text. For example, taking a short message like:
"I'm looking for a Mexican restaurant in the center of town"
And returning structured data like:
intent: search_restaurant
entities:
- cuisine : Mexican
- location : center3)LTP
4)is13
Investigation of Recurrent Neural Network Architectures and Learning Methods for Spoken Language Understanding
0x06 字词匹配WORDS MATCHING
1)NLPIR(KeyScanner)
2)AC AUTOMATON (PYTHON:esmre ahocorasick)
0X07 实体链接ENTITY LINKING
1)yahoo/FEL
快速的实体链接工具集，实现mention连接到Wikipedia。
2)dalab/pboh-entity-linking(JAVA)
论文"Probabilistic Bag-Of-Hyperlinks Model for Entity Linking"的源码。
3)CN-DBpedia API
CN-DBpedia提供全套API，并且免费开放使用。
4)songjs1993/Entity-Linking
Entity Linking，识别给定文本中出现的命名实体（Named Entity），并映射到特定的知识库中唯一的实体。包括命名实体识别、消歧等工作。
5)semanticize/semanticizer
The Semanticizer是2012年由Daan Odijk开发的用于语义连接的应用。
0x08 自动文摘AUTOMATIC SUMMARIZATION
1)miso-belica/sumy
针对文本文档和HTML的自动文摘python模块。 https://pypi.python.org/pypi/sumy
0x09 主题建模TOPIC MODELING
1)baidu/Familia
百度开源的Familia 开源项目包含文档主题推断工具、语义匹配计算工具以及基于工业级语料训练的三种主题模型：Latent Dirichlet Allocation(LDA)、SentenceLDA 和Topical Word Embedding(TWE)。
2) Microsoft/LightLDA
微软开源的方便快捷轻量的大规模主题建模系统 http://www.dmtk.io
0x0A 文本分类TEXT CLASSIFICATION
1)yoonkim/CNN_sentence
经典论文Convolutional Neural Networks for Sentence Classification (EMNLP 2014). 的大牛作者的源代码。
2)DocumentClassification
使用TensorFlow实现的简单的用以文本分类的CNN模型。
3)lc222/text_classification_AI100
主要用于文本分类，其中涉及CHI选择特征词，TFIDF计算权重，朴素贝叶斯、决策树、SVM、XGBoost等算法， 实现传统的文本分类并取得了不错的效果。
4) dennybritz/cnn-text-classification-tf
TensorFlow搭建cnn文本分类模型
5)gaussic/text-classification-cnn-rnn
使用卷积神经网络以及循环神经网络进行中文文本分类.基于TensorFlow在中文数据集上的简化实现，使用了字符级CNN和RNN对中文文本进行分类，达到了较好的效果。
6)清华大学THUCTC
THUCTC(THU Chinese Text Classification)是由清华大学自然语言处理实验室推出的中文文本分类工具包，能够自动高效地实现用户自定义的文本分类语料的训练、评测、分类功能。
7)jiegzhan/multi-class-text-classification-cnn-rnn
使用TensorFlow构建的用于多类别分类的 CNN, RNN (GRU and LSTM)模型。
0x0B 问答系统QA\CHATBOT
1)brmson/yodaqa
YodaQA 是一个开源的事实型问答系统，他能够利用即时的信息抽取从数据库和非结构化的文本语料中抽取答案。在Freebase和DBPpedia上表现优异。
2)ChatterBot
ChatterBot 是一个用于构建chatbot的集成了机器学习的对话引擎，。http://chatterbot.readthedocs.io
3)Conchylicultor/DeepQA
使用TensorFlow实现论文A Neural Conversational Model 中的google的聊天机器人（seq2seq方法）。
4)macournoyer/neuralconvo
使用Torch实现论文A Neural Conversational Model 中的google的聊天机器人（seq2seq方法）。
5) fateleak/chatbot-zh-torch7
4)的中文情境下的实现。
6)alfredfrancis/ai-chatbot-framework
使用python的chatbot框架。
7)zake7749/Chatbot
基于向量匹配的情景式聊天机器人。
8)qhduan/Seq2Seq_Chatbot_QA
使用TensorFlow实现的Sequence to Sequence的聊天机器人模型。