Large Model - Introduction to LangChain Open Source Framework
2023 can be said to be the first year of the development of AI large language models. With the release of OpenAI's ChatGPT and GPT-4, the development wave of artificial intelligence large language models has been ignited. Major technology companies have launched their own large language model products. Various countries have promoted the development of large language models as an important breakthrough in artificial intelligence technology, and have carried out business and technical layouts one after another. However, the existing pre-training large models also have certain technical defects, such as only general knowledge representation, knowledge is only up to the date of training, and it is impossible to acquire new knowledge. However, the update training of large models requires a huge amount of resources and costs, and it is even more unaffordable for enterprises or individuals at the application level of large models. Therefore, in this context, the LangChain open source framework emerged, which is used to combine external data with the LLM big language model, so that LLM can perform intelligent question and answer based on external data knowledge. This is the AI big language model built by the LangChain open source framework. A new paradigm for AI application development under the model. This article focuses on the basic content such as the concept of LangChain, business architecture, core modules and application scenarios, so that everyone can have a deeper understanding of LangChain.
1. The concept of LangChain open source framework
LangChain is a framework for developing applications driven by language models, designed to help developers build end-to-end applications using language models. It provides a set of tools, components, and interfaces that simplify the process of creating applications powered by large language models (LLM) and chat models. LangChain can easily manage the interaction with the language model, link multiple components together, and connect to other data sources, that is, LangChain is an open source framework that can combine large language models with external data and calculations, allowing large language models to be Data generation based on external data sources.
A powerful and differentiated application will not only call the language model through the API, but also connect the language model to other data sources for data awareness (such as Pinecone), and also allow the language model to interact with its environment (such as accessing the Internet to obtain data ), and LangChain can accomplish these tasks very well. Currently, the LangChain open source framework provides Python and JavaScript packages.
2. LangChain open source framework business architecture
Different from the emerging business innovation route of large models, LangChain has chosen an innovative path of application development framework based on large models. It uses the excellent multi-modal data understanding and generation capabilities of large models, combined with external computing and data sources to provide The framework for application development, and the open source code will make its attention soar in March 2023 with the release of GPT-4.
LangChain defines a set of application development framework based on large models, and proposes that all major models can be implemented in accordance with the framework definition. It is also constantly improving its ability to support large models. Its update iterations are very fast, and the new content is constantly appearing. The following figure describes the business architecture of LangChain. The internal core business includes six parts: Models, Prompts, Agents, Indexes, Chains, and Memory.
The core business principle of LangChain is to first use the Indexes index module to load documents, segment text, and vectorize external data sources (such as text documents or databases) and store them in the vector storage database VectorStore; Retrieve relevant information similar to the Promp in the query; then pass the query and the retrieved similar information to the LLM; finally, the LLM understands and generates data information according to the input, and completes the processing of subsequent Actions through Agents. In this way, developers can easily develop AI applications for specific industries, specific purposes, and specific data sources based on large models based on the LangChain development framework. It expands the application field and space of large models.
LangChain believes that AI application development must be based on the large LLM model. LangChain provides very convenient interfaces for accessing various LLMs through Models, including GPT and GLM series models, which can be called through LangChain's Models interface; at the same time, it simplifies the difficulty of prompting Rich prompt flexibility is also a must-have capability for future AI application development. LangChain provides Prompts to provide developers with the function of flexible custom prompt templates; in order to better understand human intentions, the internal execution process of AI tools should be Similar to the pipeline assembly line, this process is called Pipeline in the field of machine learning, and Chains in the definition of LangChain; the ability of the large language model to understand the context is the basis of natural language understanding, and AI can remember human prompts and Its own operation will be the key to improving the user experience. LangChain implements this memory function through the Memory module; and the future development of AI in various industries is inseparable from the interaction of external tools and data. LangChain uses the Agents module to set Agents use these tools to expand the functions of AI tools themselves. The Indexes module provides the indexing function of local files and external data, so that AI tools can manage local files or external data more conveniently. The functions of LangChain are still being improved and updated continuously. With the continuous application of LangChain in the future, LangChain will become more and more perfect and stronger.
3. LangChain open source framework core module
Based on the above business structure, LangChain's internal core business includes six parts: Models, Prompts, Agents, Indexes, Chains, Memory. They are described as follows:
3.1. Models component
Models component: the interactive component of the model connected to LLM, which is used to complete business interaction with different types of models. LangChain divides the model into three types of models: LLMS, Chat Model, and Text Embedding, and completes the business interaction of the three models through different types and operations. .
Large-scale language models (LLMs) refer to commercial large-scale language models with language understanding and generation capabilities, which take text strings as input and return text strings as output. The LLM class is designed in LangChain for interface interaction with large language models. This class aims to provide standard interfaces for LLM providers, such as OpenAI, Coherent, and Hugging Face.
Taking the OpenAI LLM wrapper as an example, its usage is as follows:
from LangChain.llms import OpenAI
llm = OpenAI(model_name = 'text-davinci-003',n=2,best_of = 2)
llm("Please introduce yourself")
llm.get_num_tokens(question)
That's right, it's the most basic function. You can call it directly, pass in a string and return a string, generate text, and count the number of tokens required for calculation. More broadly, it can also be called with an input list, getting a more complete response than just text. Such as: llm.generate([“Tell me a joke”, “Tell me a poem”]*15)
Chat Model chat model is a variant of the language model, the chat model is based on the language model, which uses the language model internally, Instead of text strings as input and output, chat messages list as input and output, they provide a more structured API. One or more messages can be passed through the chat model. LangChain currently supports four message types: AIMessage, HumanMessage, SystemMessage and ChatMessage.
chat = ChatOpenAI(temperature=0)
chat([HumanMessage(content="Translate this sentence from English to French. I love programming.")])
AIMessage(content="J'aime programmer.", additional_kwargs={})
SystemMessage : System message is a tool used to set the model, which can be used to specify the specific environment and background of the model, such as role-playing, etc.; HumanMessage
: Human message is user information, information given by humans, Such as asking questions; using the Chat Model model, you have to put system messages and human messages in a list, and then use it as the input of the Chat Model model.
AIMessage
: It is the message output by AI, which can be an answer to the question. Parameters for any role
In most cases, we only need to deal with HumanMessage, AIMessage and SystemMessage message types. In addition, the chat model supports multiple messages as input, such as the following examples of system messages and user messages
messages = [
SystemMessage(content="You are a helpful assistant that translates English to French."),
HumanMessage(content="I love programming.")
]
chat(messages)
AIMessage(content="J'aime programmer.", additional_kwargs={})
You can also use generate to generate multiple sets of messages.
batch_messages = [
[
SystemMessage(content="You are a helpful assistant that translates English to French."),
HumanMessage(content="I love programming.")
],
[
SystemMessage(content="You are a helpful assistant that translates English to French."),
HumanMessage(content="I love artificial intelligence.")
],
]
result = chat.generate(batch_messages)
The text embedding model text-embedding-model is to represent the text as a vector, so that operations such as semantic search can be performed on the text in the vector space, that is, to find the most similar text fragments in the vector space. And these are realized through the Embedding class in LangChain.
The Embedding class is a class for interacting with text embeddings. This class aims to provide a standard interface for providers (there are many embedded providers such as OpenAI, Cohere, Hugging Face, etc.). The Embedding class in LangChain exposes two methods: embed_documents and embed_query. The biggest difference is that the two methods have different interfaces: one works for multiple documents, while the other works for a single document. Besides that, another reason for having these two separate methods is that some embedding providers have different embedding methods for the document being searched than for the query itself.
3.2. Prompts prompt project
Prompt refers to the input of the model. This input is rarely hard-coded, but is built from a specific template component. This template component is the PromptTemplate prompt template, which can provide a prompt template as input. Template Refers to the specific format and blueprint in which we want answers. LangChain provides pre-designed prompt templates that can be used to generate prompts for different types of tasks. When the preset templates cannot meet the requirements, a custom prompt template can also be used.
In LangChain, we can set the prompt template as needed and connect it with the main chain for output prediction. In addition, LangChain also provides the function of outputting a parser for further refining the results. The role of the output parser is to instruct how the output of the model should be formatted, and to parse the output into the desired format.
LangChain provides several classes and functions that make it easy to construct and process hints.
LLM Prompt Templates (LLM Prompt Templates)
The LLM language model takes a text string as input, and this text string is usually called a prompt. Sometimes it's a hard-coded string, but most of the time the hint is a combination of a template, some examples, and user input.
The LLM prompt template is a repeatable method for generating prompts for the LLM language model, which includes a text string template that can receive some input parameters from the user to generate prompts.
LLM prompt templates generally include: instructions for the language model, a small set of examples to help the language model generate better responses, and questions for the language model.
The creation of LLM prompt templates can be done using the PromptTemplate class, a simple example is as follows:
from LangChain import PromptTemplate
template = " Please help me to find a new name for my company that makes {product}?"
prompt = PromptTemplate(
input_variables=["product"],
template=template,
)
prompt.format(product="super phone")
#-> Please help me to find a new name for my company that makes super phone?
Chat Prompt Template (Chat Prompt Template)
Chat Models Chat Models take a list of chat messages as input, and this list of chat messages is usually called a prompt. Chat messages are different from the original text strings that users enter into the LLM language model. Chat messages need to be attached to corresponding roles, such as SystemMessage, HumanMessage, and AIMessage corresponding to system, human, and AI roles. The chat model should more closely follow the instructions of the system chat messages.
The creation of chat prompt templates can use chat-related prompt templates instead of PromptTemplate to fully exploit the potential of the underlying chat model. LangChain provides different types of MessagePromptTemplate. The three most commonly used types, AIMessagePromptTemplate, SystemMessagePromptTemplate and HumanMessagePromptTemplate, are used to create AI messages, system messages and human messages respectively.
SystemMessagePromptTemplate ---- Create a message template associated with a system role
AIMessagePromptTemplate ---- Create a message template associated with an AI role
HumanMessagePromptTemplate ---- Create a message template associated with a human role
The chat model supports sending with any role In the case of chat messages, a ChatMessagePromptTemplate can also be used, allowing the user to specify a role name.
Example Selectors (Example Selectors)
Example selectors refer to the design of appropriate examples or prompt texts to guide the model to generate the desired output when using a large language model for generation tasks. For example, guide the model to understand the content and format that should be generated, help control the context of model generation, constrain or limit the content generated by the model, and assist in generating responses to specific questions or situations.
The function of the example selector is to guide the model to generate the desired content, control the context, constrain the output, etc. by designing appropriate example texts, thereby improving the accuracy and controllability of the generation task. It plays the role of guidance, guidance and constraints when using large language models, making the generation of models more in line with expectations and requirements.
If there are a large number of examples in use, it is necessary to select the examples included in the prompt. The ExampleSelector class in LangChain is responsible for performing this task. Here are some examples of hints
#These are a lot of examples of a pretend task of creating antonyms.
examples = [
{"input": "happy", "output": "sad"},
{"input": "tall", "output": "short"},
{"input": "energetic", "output": "lethargic"},
{"input": "sunny", "output": "gloomy"},
{"input": "windy", "output": "calm"},
]
Often times it is useful to include examples in the prompt. These examples can be hardcoded, but the effect is more powerful if they are chosen dynamically.
Output Parsers (Output Parsers)
Output Parsers refer to the components that parse and process the results generated by the model. Its main function is to parse the text generated by the model, extract useful information and perform subsequent processing. Such as parsing the text generated by the model, extracting useful information, identifying entities, classifying and filtering results, and post-processing the generated text to make the generated results easier to understand and use. It plays a role in parsing and processing results when interacting with large language models, enhancing the application and usability of the models.
The language model outputs text. But many times, you may want to get more structured information than text. This is what the output parser does. That is, the output parser is a class that helps structure the language model response, and the main class provided in LangChain is PydanticOutputParser.
3.3. Chains
Usually we can use a separate LLM to solve the problem, but for more complex applications, we need to link between LLMs or with other systems to complete the task, which is usually called linked LLM. Chaining allows combining multiple components across models or systems to create a single, coherent application. As an example, we create a chain that takes user input, formats the input via a PromptTemplate template, and passes it to the LLM language model. Multiple chains can also be combined, or chains can be combined with other system components to build more complex chains and achieve more powerful functions. LangChain provides standard interfaces as well as common implementations for chains, has extensive integrations with other tools, and provides end-to-end chains for common applications.
In the above example, accepting user input through template formatting and passing it to the language model can be realized through a simple chain class, such as the LLMChain class in LangChain. It's a simple chain that takes a prompt template, formats it with user input, and returns a response from the LLM. Of course LLMChain can also be used for chat models.
from LangChain.prompts import PromptTemplate
from LangChain.llms import OpenAI
from LangChain.chains import LLMChain
llm = OpenAI(temperature=0.9)
prompt = PromptTemplate(
input_variables=["product"],
template=" Please help me to find a new name for my company that makes {product}?",
)
chain = LLMChain(llm=llm, prompt=prompt)
print(chain.run("super phone"))
LangChain provides many ready-made links, but sometimes it may be necessary to create a custom link for a specific application. In addition, there is also the concept of sequence chain in LangChain, that is, a chain that executes links in a predefined order. For example, SimpleSequentialChain is the simplest type of sequence chain, in which each step has an input/output, and the output of one step is the next step. input of. There are many advanced usages of chains in LangChain.
3.4. Agents proxy
Usually, a user’s problem may require multiple logic processing of the application to complete related tasks, and it may be dynamic, requiring different actions depending on the user’s input, or executing different actions depending on the output of the LLM. Therefore, the application program not only needs to predetermine LLM and other tool call chains, but may also need to generate different chains according to different user inputs. Using a proxy can make LLM access tools more direct and efficient. Tools provide unlimited possibilities. With a tool, LLM can search the network, perform mathematical calculations, run codes, and other related functions.
Agents in LangChain use LLM to determine which actions to take and in which order, look at the observations, and repeat until the task is completed. The LangChain library provides a large number of preset tools, and also allows modifying existing tools or creating new ones. When proxies are used correctly, they can be very powerful. In LangChain, the concept of "agent" is used to access a series of tools in these types of chains to complete tasks. Based on user input, the agent can decide whether to invoke any of these tools. Here is a simple example:
#加载要使用的语言模型来控制代理人
llm = OpenAI(temperature=0)；
#加载一些需要使用的工具，如serpapi和llm-math，llm-math工具需要使用LLM
tools = load_tools(["serpapi", "llm-math"], llm=llm)
#使用工具、语言模型和代理人类型初始化一个代理人。
agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
#控制代理执行
agent.run("Who is Vin Diesel's girlfriend? What is her current age raised to the 0.5 power?")
以上示例需要处理三个过程，首先查询Vin Diesel's girlfriend，然后在查询她的年龄，再计算指数关系。可看到代理的工作过程。
3.5. Indexes
Indexes refers to the way documents are structured so that LLM can best interact with them. LangChain can combine external data and LLM to understand and generate natural language. The external data can be local documents, databases and other resources. These data are segmented and vectorized and stored in the vector storage database, and then the vector is retrieved through the user's prompt. Similar information in the database is passed to the large language model for generation and execution of Action. Among them, the Indexex index runs through the process of retrieving similar related information in the vector database and plays a pivotal role.
Indexes schemas and functions for processing external data and making it ready for interaction with language models, including functions such as document loaders, vector stores, text splitters, and retrievers. This module contains utility functions for working with documents, different types of indexes, and the ability to use these indexes in chains.
The most common way to use an index in a chain is as a "retrieve" step. This step refers to accepting the user's query and returning the most relevant documents. Most of the time when we talk about indexing and retrieval, we are talking about indexing and retrieving unstructured data such as text documents. The index and retrieval types supported by LangChain are currently mainly focused on vector databases
Document loader
The document loader solves how to load documents from various source data. The combination of language models and external text data is the strength of LangChain. The first step in doing this is to load the data into a "document", this module can make this easy, relying on a powerful tool, the Unstructured Python package, which will be able to convert files (such as: text, PowerPoint , images, HTML, PDF, etc.) to text data.
Text Splitter
A text splitter is a function that solves the problem of splitting text. When dealing with long texts, it is necessary to split the text into chunks. Split the text into small, semantically meaningful chunks, start combining those smaller chunks into one larger chunk, until it reaches a certain size, make that chunk its own chunk of text, and start creating a new one blocks with some overlap to maintain context between blocks of text. As simple as it sounds, there is a lot of potential complexity here. It is necessary to keep small text blocks, but also to keep semantically related text ends together, and to be able to make contextual associations.
The text splitter supported by LangChain is RecursiveCharacterTextSplitter. This text splitter expects a list of characters. It tries to create chunks based on the first character split, but if any chunk is too large, it moves to the next character, and so on. By default, the characters it tries to split on are [" ", "\n", " ", ""]. LangChain also supports many text splitters, such as: character text splitter, Hugging Face length function, Latex text splitter, Python code text splitter, etc.
Vector storage (VectorStores)
Vector storage is one of the important components of Indexes index construction. A key part of dealing with vector storage is creating the vectors to put into it, which is usually created through embedding. LangChain implements the embedding class OpenAIEmbeddings for openAI.
Retriever interface (Retrievers)
The retriever interface is a general interface that makes it easy to combine documents and language models. There is a get_relevant_documents method exposed in LangChain that takes a query (string) and returns a list of documents.
LangChain uses the Indexex index to process external data and store it in the vector storage database after vectorization processing by the document loader and text segmenter. Finally, the similar vector information obtained by calling the retriever interface through the user's question is passed to the LLM for processing.
3.6. Memory storage
Memory is a concept that maintains state throughout the interaction between the user and the language model. Embodied in the interactive chat message process between the user and the language model, this involves ingesting, capturing, converting and extracting knowledge from a series of chat messages. Memory maintains state between Chains/Agents calls. By default, Chains and Agents are stateless, which means that they process each incoming query independently, but in some applications, such as: chatbots, It is very important to remember previous interactions, both in the short and long term. The concept of "Memory" is to achieve this.
LangChain provides two ways to use the Memory component. One is to provide auxiliary tools for managing and manipulating previous chat messages to extract information from message sequences; the other is to use it in Chains. Memory can return multiple pieces of information, such as the latest N messages or a summary of all previous messages, etc. The returned information can be a string or a list of messages.
LangChain provides methods and interfaces for extracting memory information from chat records, buffer memory, and Chains, such as the ChatMessageHistory class, an ultra-lightweight wrapper that provides some convenient methods to save human messages and AI messages, and then extract them from Get them; another example is the ConversationBufferMemory class, which is a wrapper for ChatMessageHistory, used to extract messages in variables, and so on.
4. Application scenarios of the LangChain open source framework
Different from the emerging business innovation route of large models, LangChain has chosen an innovative path of application development framework based on large models. It uses the excellent multi-modal data understanding and generation capabilities of large models, combined with external computing and data sources to provide The framework for application development, and the open source code will make its attention soar in March 2023 with the release of GPT-4. The reason is that LangChain has started the exploration of application development models based on large models. LangChain is a development tool, development protocol and development paradigm for AI applications. As Mr. Kai-fu Lee described, when AI enters the 2.0 era, all applications will be rewritten. This also reflects the development direction of AI, and the LangChain open source framework will greatly assist and promote the rapid implementation of this trend. LangChain believes that future AI applications based on LLM are based on its six core elements Models, Prompts, The patchwork and stacking of Chains, Agents, Memory, and Indexes, so what kind of application scenarios will there be through the combination of LangChain elements?
The personal assistant application
LangChain enables AI applications not only to answer questions, but also to take actions, which opens up the possibility of countless practical applications, and any affairs involving personal assistants will become simple, such as booking hotels, booking air tickets, Pay taxes etc.
Analysis and processing of local data
LangChain can combine external data and LLM large model, and propose problems to be solved through screening and retrieval of external data, and not only have answers but also actions to guide problem solving and complete tasks. External data can be local files, local databases, etc., so that LangChain can be used to analyze local data, and the results of data analysis can be further processed to take action, including the work that needs to be done through coding is still competent. Specifically, such as financial statement analysis, customer relationship data processing, purchase orders, material inventory management and so on.
Customized industry robots
The LLM large language model has general knowledge capabilities, and LLM in the general knowledge field can give better answers, but its capabilities in the industry are limited, and it is even more confused about real-time updated business content. Through LangChain, the local industry knowledge database can be connected to LLM for intelligent processing, and LLM’s ability to understand general knowledge can be used to solve the problems and processing of business knowledge in the industry field, such as quickly completing industry intelligent customer service, industry chat robots, It is also possible to customize the design of industrial robots.
Creation of local knowledge base
LangChain can complete the construction of local knowledge base, and can quickly build and upgrade the local knowledge base into a domain knowledge base model with natural language understanding and generation capabilities. It can empower small and medium-sized enterprises to meet the internal business knowledge base model construction without large-scale neural network training and construction. With the continuous development of Tsinghua University's GLM series of open source models and the continuous upgrading of support capabilities, the use of LangChain + ChatGLM-6B to build local knowledge bases has been practiced and applied in various small and medium-sized enterprises.
LangChain is a development protocol and framework for AI applications. It builds a development framework for AI applications based on the LLM large language model. Its core capabilities are constantly being improved and updated. With the continuous emergence and upgrading of modal models, the application scenarios of LangChain will become more and more abundant, and innovative businesses will continue to emerge. It is expected that LangChain will have better performance in the future.
5 Conclusion
LangChain has chosen an innovative way of developing an application development framework based on a large model in an innovative way different from the development route of the large model. By leveraging the excellent multi-modal data understanding and generation capabilities of the large model, combined with external computing and data sources Provide a framework for application program development, and expect to define the AI application development paradigm based on the LLM model through a unified function module name and development specification, and LangChain also provides a series of APIs to realize these functions, allowing AI based on the LangChain open source framework Application development quickly landed. This article focuses on the basic business introduction of the LangChain open source framework, involving the concept of LangChain, business architecture, core element functions and the exploration of application scenarios. Looking forward to being helpful to everyone.