1. Preliminary exploration of large language models
Introduction to ChatGLM3
ChatGLM3-6B is a pre-trained language model based on Transformer, which was jointly trained and released by Tsinghua University KEG Laboratory and Zhipu AI Company in 2023. The basic principle of this model is to pre-train a large amount of unlabeled text data and then use it for various downstream tasks, such as text classification, named entity recognition, sentiment analysis, etc.
The core function of ChatGLM3-6B is language modeling, which is to build a language model by predicting the next word or character. The model adopts the Transformer structure, which is a deep neural network structure based on a self-attention mechanism that can effectively capture long-term dependencies in text.
The ChatGLM3-6B model has a variety of pre-trained tasks, such as text classification, named entity recognition, sentiment analysis, etc. During the pre-training process, the model will learn various language knowledge and patterns, so that it can better complete various downstream tasks.
The limitation of the ChatGLM3-6B model is that it can only process already trained models and cannot be directly used on new, unlabeled data. In addition, since the pre-trained model is based on unlabeled data, it may not fully capture some specific language knowledge and patterns.
Overall, ChatGLM3-6B is a powerful language model that performs well on a variety of text-related tasks. Its core function is a self-attention mechanism based on the Transformer structure, which can capture long-term dependencies in text. At the same time, it also has a variety of pre-training tasks to better complete various downstream tasks. However, its limitation is that it can only process already trained models and cannot be directly used on new, unlabeled data.
2. Introduction to LangChain and its core components
Introduction to LangChain
LangChain is an open source framework that allows developers working in artificial intelligence to combine large language models such as GPT-4 with external computing and data sources. The framework is currently available as a Python or JavaScript package.
Suppose, you want to know something specific from your own data or file (it can be a book, a pdf file, a database containing proprietary information). LangChain can connect GPT-4 to this external data and even let LangChain help you take the action you want to take, such as sending an email.
Practical courses:
1. Realize the loading and parsing of pdf and jpg format documents
File location: document_loader.ipynb
2. To implement a chat application based on ChatGLM3+LangChain, a Gradio interface is required
File location: chat.ipynb
3. Implementing local knowledge base Q&A based on LangChain+ChatGLM3 requires the Gradio interface.
Supports local files in four formats: txt, md, pdf, and jpg.
File location: langchain_chatglm3_V3.ipynb
The final Gradio interface Q&A is as follows:
Generate text directly:
Streaming chat mode generates text
