[Large language model] Quickly master LangChain and ChatGLM in 15 minutes

LangChain is a powerful framework that simplifies the process of building advanced language model applications.

Introduction to LangChain

LangChain is a powerful framework designed to help developers build end-to-end applications using language models . It provides a set of tools, components, and interfaces that simplify the process of creating applications powered by large language models (LLM) and chat models . LangChain can easily manage interactions with language models, link multiple components together, and integrate additional resources such as APIs and databases.
The main functions of LangChain are : call the language model, connect different data sources to the interaction of the language model, and allow the language model to interact with the operating environment.

Core Concepts in LangChain

1. Components and Chains

**Components are modular building blocks that can be combined to create powerful applications. A Chain is a series of Components (or other Chains) grouped together to accomplish a specific task. **For example, a Chain might include a Prompt Template, a Language Model, and an Output Parser that work together to process user input, generate a response, and process the output.

2. Prompt Templates and Values

The Prompt Template is responsible for creating the PromptValue, which is what is ultimately passed to the language model . Prompt Template helps convert user input and other dynamic information into a format suitable for language models. PromptValues ​​are classes with methods that convert to the exact type of input each model type expects (such as text or chat messages).

3. Example Selectors

Example Selectors are useful when you want to dynamically include examples in Prompts. They take user input and return a list of examples to use in prompts , making them more powerful and context-specific.

4. Output Parsers

Output Parsers are responsible for building language model responses into a more useful format . They implement two main methods: one for providing formatting instructions and one for parsing the language model's response into a structured format . This makes it easier to process the output data in your application.

5. Indexes and Retrievers

Index is a way of organizing documents to make it easier for language models to interact with them . A retriever is an interface for taking relevant documents and combining them with a language model. LangChain provides tools and functions for working with different types of indexes and retrievers, such as vector databases and text splitters.

6. Chat Message History

LangChain mainly interacts with the language model through the chat interface . The ChatMessageHistory class is responsible for remembering all previous chat interaction data, which can then be passed back to the model, aggregated, or otherwise combined . This helps maintain context and improves the model's understanding of the conversation.

7. Agents and Tookits

Agents are entities that drive decision-making in LangChain . They have access to a set of tools and can decide which tool to invoke based on user input. Tookits are a set of tools that, when used together, accomplish a specific task . The agent executor is responsible for running the agent with the appropriate tools.

What is LangChain Agent?
LangChain Agent is the entity in the framework that drives decision making . It has access to a set of tools and can decide which tool to invoke based on user input . Proxies help build complex applications that require adaptive and context-specific responses. They are especially useful when there are unknown interaction chains that depend on user input and other factors.

What is the LangChain model?
The LangChain model is an abstraction representing the different types of models used in the framework. The models in LangChain are mainly divided into three categories:

1. LLM (Large Language Model): These models take text strings as input and return text strings as output. They are the backbone of many language model applications.
2. Chat Model: The chat model is backed by the language model, but with a more structured API. They take a list of chat messages as input and return chat messages. This makes it easy to manage conversation history and maintain context.
3. Text Embedding Models: These models take text as input and return a list of floats representing the text embedding. These embeddings can be used for tasks such as document retrieval, clustering, and similarity comparison.

The code structure of LangChain

The repo of langchain: https://github.com/hwchase17/langchain

can see that there are more than one LLM that LangChain can connect to, and there are also many agents and tools:

1. Modules provided in LangChain

• Modules: supported module types and integrations
• Prompt: prompt word management, optimization and serialization
• Memory: Memory refers to the state that persists between chain/proxy invocations.
• Indexes: Language models become even more powerful when combined with application-specific data - this module contains interfaces and integrations for loading, querying, and updating external data.
• Chain: A chain is a structured sequence of calls (to LLM or other utilities).
• Agents: An agent is a chain in which the LLM, given a high-level instruction and a set of tools, iteratively decides on an action, executes the action, and observes the results until the high-level instruction is completed.
• Callbacks: Callbacks allow recording and streaming of intermediate steps of any chain, making it easy to observe, debug and evaluate the internals of your application.

2. Application scenarios of LangChain

• Documentation Q&A: A common LangChain use case. Answer questions on specific documents, using only the information in those documents to construct answers.
• Personal Assistant: One of the main use cases of LangChain. Personal assistants need to act, remember interactions, and understand your data
• Query tabular data: use the language model to query library table type structured data (CSV, SQL, DataFrame, etc.)
• Interacting with APIs: Having language models interact with APIs is very powerful, it allows them to access up-to-date information, and allows them to take action.
• Information extraction: extracting structured information from text.
• Document Summary: Compressing Longer Documents, a Data Augmentation Generation.

3. Main features of LangChain

LangChain aims to support developers in six main areas:

• LLMs and Prompts : LangChain makes it easy to manage prompts, optimize them, and create a common interface for all LLMs. Additionally, it includes some handy utilities for working with LLMs.
• Chain : These are sequences of calls to LLM or other utilities. LangChain provides a standard interface for the chain, integrates with various tools, and provides an end-to-end chain for popular applications.
• Data-enhanced generation : LangChain enables chains to interact with external data sources to collect data for generation steps. For example, it can help summarize long texts or answer questions using a specific data source.
• Agents : Agents let the LLM make decisions about actions, take those actions, check the results, and move on until the job is done. LangChain provides a standard interface for proxies, a variety of proxies to choose from, and end-to-end proxy examples.
• Memory : LangChain has a standard memory interface that helps maintain state between chain or proxy calls. It also provides a range of memory implementations and examples of chains or agents that use memory.
• Evaluation : It is difficult to evaluate generative models with traditional metrics. That's why LangChain provides hints and chains to help developers evaluate their models using LLM themselves.

Example of using LangChain

LangChain supports a large number of use cases, such as:

• Document-Specific Question Answering : Answers questions given documents, using the information in those documents to create answers.
• Chatbots : Build chatbots that can leverage the power of LLMs to generate text.
• Agents : Develop agents that can decide on actions, take those actions, observe the results, and continue execution to completion.

Build an end-to-end language model application with LangChain

There are two core concepts in LangChain: agentandtool

Install LangChain :

pip install openai

Environment Setup
Now since LangChain often needs to integrate with model providers, data stores, APIs, etc., we will set up our environment. In this example, OpenAI 's API will be used, so their SDK needs to be installed:

pip install openai

Set environment variables:

import os
import openai
import warnings

warnings.filterwarnings('ignore')
# Set the OpenAI API key
os.environ['OPENAI_API_KEY'] = "sk-xxxxxxxxxxxxxxx"  # get api_key from openai official website
openai.api_key = os.getenv("OPENAI_API_KEY")

Building language model applications:
After LLM has installed LangChain and set up the environment, it can start building language model applications. LangChain provides a bunch of modules that can be used to create language model applications. These modules can be combined for more complex applications, or used individually for simpler applications.

Building language model applications: Chat Model
In addition to LLM, chat models can also be used. These are variants of language models that use a language model under the hood but have a different interface. The chat model uses chat messages as input and output instead of the "text in, text out" API. The use of the chat model API is still relatively new, so everyone is still figuring out the best abstraction to use.

To complete a chat, one or more messages need to be passed to the chat model. LangChain currently supports AIMessage, HumanMessage, SystemMessage and ChatMessage types . Mainly use HumanMessage, AIMessage and SystemMessage. For example:

from langchain.chat_models import ChatOpenAI
from langchain.schema import (AIMessage, HumanMessage, SystemMessage)

In the example above, a message is sent to OpenAI. Of course, it is also possible to pass multiple messages:

it is also possible to generategenerate completions for multiple sets of messages. This will return an LLMResult with an additional message parameter:

token usage and other information can also be extracted from the LLMResult:

result.llm_output['token_usage']
# -> {'prompt_tokens': 69, 'completion_tokens': 19, 'total_tokens': 88}

For chat models, MessagePromptTemplatetemplates can also be used by using . MessagePromptTemplatesCan be created from one or more ChatPromptTemplate. The method format_prompt of the ChatPromptTemplate returns a PromptValue which can be converted to a string or a Message object, depending on whether the formatted value is to be used as input to the LLM or chat model. For example:

from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (ChatPromptTemplate, SystemMessagePromptTemplate, HumanMessagePromptTemplate)

Agents can also be used with chat models. Initialize using AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTIONas proxy type Agent:

from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI

Install serpapithe package and set SERPAPI_API_KEYenvironment variables:

!pip install google-search-results
# Set the Search API key
os.environ['SERPAPI_API_KEY'] = "xxxxxxxxxxxxxxxxx"

In this example, the agent will interactively perform searches and calculations to provide the final answer.

Finally, explore using memory with chains and agents initialized with the chat model . The main difference between this and Memory for LLMs is that you can keep previous messages as their own unique memory objects instead of compressing them into a string.

from langchain.prompts import (
    ChatPromptTemplate, 
    MessagesPlaceholder, 
    SystemMessagePromptTemplate, 
    HumanMessagePromptTemplate
)
from langchain.chains import ConversationChain
from langchain.chat_models import ChatOpenAI
from langchain.memory import ConversationBufferMemory

prompt = ChatPromptTemplate.from_messages([
    SystemMessagePromptTemplate.from_template("The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know."),
    MessagesPlaceholder(variable_name="history"),
    HumanMessagePromptTemplate.from_template("{input}")
])
llm = ChatOpenAI(temperature=0)
memory = ConversationBufferMemory(return_messages=True)
conversation = ConversationChain(memory=memory, prompt=prompt, llm=llm)

A ConversationChain is used in this example to maintain a conversation context across multiple interactions with the AI.

How to implement question answering based on local knowledge?

Implementation principle based on single document question answering

1. load local documentation
2. Document Splitting
3. Match text against a question
4. build prompt
5. LLM generates answers

Implementation principle of question answering based on local knowledge base

From the perspective of document processing, the implementation process is as follows:

Code:

Introduction to LangChain-ChatGLM

LangChain-ChatGLM is a local knowledge base Q&A implementation based on ChatGLM and other big oracle models, that is, a Q&A application based on local knowledge bases implemented using the idea of ​​langchain. A knowledge base question answering solution that works offline.

Project address: Implementation of large language model applications such as ChatGLM based on local knowledge base

Project Features:

• Relying on open source models such as ChatGLM, it can be deployed offline
• Based on langchain implementation, it can quickly realize access to multiple data sources
• In terms of sentence segmentation and document reading, it is optimized for Chinese usage scenarios
• Support PDF, txt, md, docx and other file types to access, with command line demo, webui and vue front-end.

Project structure:

• models: The interface class and implementation class of LLM, providing streaming output support for open source models.
• loader: the implementation class of the document loader
• textsplitter: the implementation class of text segmentation
• chains: work link implementation, such as chains/local_doc_qa implements question and answer based on local documents
• content: used to store the uploaded original file
• vector_store: used to store the vector library file, that is, the local knowledge base ontology
• configs: configuration file storage

The direction of project optimization:

1. Model fine-tuning: Fine-tuning LLM and embedding based on professional field data
2. Document processing: After the text is segmented, summarize each segment separately, and match based on the semantics of the summarized content
3. With the help of different model capabilities: when code needs to be generated in text2sql and text2cpyher scenarios, different model capabilities can be used

Relevant information

1. The Complete Guide to LangChain: Building Powerful Applications with Large Language Models
2. Register Serpapi