With the release of OpenAI's groundbreaking GPT-3 in 2020, we have witnessed the popularity of LLM steadily increasing, and it is still gradually heating up. These powerful artificial intelligence models bring new possibilities to natural language processing applications, allowing developers to create more complex chatbots, question-answering systems, summarization tools, and other products that resemble human interactions.
LangChain emerged as a multifunctional framework designed to help developers fully realize the potential of LLMs in various applications. Based on the core concept of "chaining" different components, LangChain simplifies the working process with LLMs such as GPT-3/4, Bloom, Huggingface, etc., allowing developers to seamlessly build advanced LLM-based applications.
1. What is Langchain?
LangChain is an innovative framework and a development method for language model-driven applications. For the concept and use of application frameworks, please refer to " Full Stack Cognition: Application Framework" and "Exploring the Embedded Application Framework (EAF)". LangChain is a large model-based application development framework and an open source Python library designed to make it easier to build LLM-based applications by:
Provides common interfaces to many different base models,
Provides a framework for managing prompts, and
Provides a central interface to long-term memory capabilities, external data, and other agents for processing tasks that LLM cannot handle (such as calculations or searches).
By incorporating advanced principles, LangChain is redefining the limits of what can be achieved through traditional APIs. Additionally, LangChain applications are proactive, enabling language models to easily interact and adapt to their environment. Langchain consists of several modules. As its name suggests, connecting different modules together is the main purpose of Langchain. The idea here is to link each module in a chain and eventually use that chain to call all modules at once.
2. Core concepts in LangChain
LangChain simplifies the management of prompt words, provides optimization capabilities, provides a common interface for all LLMs, and includes common programs for processing LLMs. LangChain provides a standard interface for chained calls, enabling developers to create call sequences that extend beyond a single LLM call. LangChain also provides developers with the ability to create chains that integrate with external data sources, a feature that makes it possible to generate text based on specific data rather than the generic data used to train language models. Moreover, LangChain provides developers with a standard interface that enables LLM to make informed decisions about which actions to take and when to take those actions based on the output of LLM. Memorization is a key concept in LangChain as it involves retaining state between invocations of the chain/proxy. LangChain also provides a standard memory interface, a series of memory implementations, and examples of chains/agents using memories.
2.1. Model
A large language model (LLM) refers to a model that consists of a neural network with numerous parameters and is trained on a large amount of unlabeled text. Many technology giants and academic organizations have their own LLM, such as: OpenAI's GPT-3/4, Google's LaMDA/PaLM, Meta AI's LLaMA, Baidu's Wenxin, Alibaba's Qianwen, iFlytek's Spark, Tsinghua University GLM and so on. With Langchain, it becomes easier to interact with application results and large language models.
LangChain easily integrates and uses different language models for enhancing the functionality of applications, connecting to most APIs available for third-party LLMs. It has ~40 API connections to public LLM, chat and embedding models. LangChain also provides asynchronous support for LLM through the asyncio library, and also provides asynchronous support for calling multiple LLMs at the same time. We can call LLM asynchronously using the agenerate method, and we can also write a custom LLM wrapper. Each large model has its own advantages, token usage times, and use cases. For more details, you can go to the official website of the relevant large model to read more information.
2.2. Prompt 电影
LangChain allows efficient management, optimization and serialization of prompts, allowing developers to build dynamic prompts using templates. It can adapt to different LLM types depending on the context window size and the input variables used as context (conversation history, search results, previous answers, etc.). This helps generate more accurate and contextually relevant responses from the language model.
Prompts are input we provide to the LLM system to improve our answers, making them more accurate or better suited to our specific use case. Many times, we may want to obtain more specific and structured information than plain text. Many target detection and classification algorithms based on contrastive pre-training and zero-shot learning take Prompt as a valid result input. For example, OpenAI's CLIP and META's Grounding DINO both use Prompt as the input for prediction.
In Langchain, the Prompt template can be set according to the answer we want, and then linked to the main chain for output prediction, and there is also the function of an output parser for result refinement. The output parser is responsible for indicating the format of the model output and parsing the output into the required format, retrying if necessary.
A template refers to a specific format or blueprint of what we want the answers to be. LangChain provides pre-designed prompt templates that can generate prompts for different types of tasks. However, in some cases, the preset templates may not suffice and a custom prompt template can be used.
2.3. Memory ability
LangChain provides a standard interface and a series of implementations for memory capabilities, providing LLM with access to conversation history. It facilitates persistence of state across invocations of a chain or agent, enhancing the model's knowledge recall capabilities.
LangChain runs in stateless mode by default, which means each incoming query is processed independently. However, for some applications, such as chatbots, it is important to preserve previous interactions, both in the short and long term. This is where the concept of "memory capacity" comes into play. To track user interactions with the language model, LangChain’s memory capabilities involve converting chat message sequences into ChatMessages and ingesting, capturing, transforming and extracting knowledge from them. There are many different memory types in LangChain, each with its own unique way of handling message sequences. When using memory capabilities, there are stand-alone functions that extract information from message sequences, and how to use this type of memory in a chain. LangChain's memory capability can return multiple pieces of information, such as the last N messages or a summary of all previous messages. The returned information can be a string or a message list.
LangChain provides two forms of memory capability components. First, auxiliary tools for managing and manipulating previous chat messages are provided. These tools are designed to be modular and usable, adaptable to various use cases. Second, LangChain provides an easy way to integrate these commonly used programs into the chain, making them highly adaptable.
2.4. Index
Indexing refers to the way documents are structured in a way that LLM optimally interacts with them. To enhance the capabilities of language models, LangChain effectively uses LLM with users' text data. It contains practical functions for processing documents, different types of indexes, and examples of using these indexes in the chain. Indexing and searching data sources are provided. best practices.
LangChain provides three document loaders:
Transform loader
Public dataset or service loader
Proprietary dataset or service loader
Transformation loaders convert data from a specific format into a document format, for example there are converters for CSV and SQL. Most of the time these loaders input data from files and sometimes from URLs. The main driver for many of these converters is the Unstructured module. This package can convert many types of files (text, PowerPoint, images, HTML, PDF, etc.) into text data. For datasets and data sources created in the public domain, we do not require any access rights to use queries to search and download the required documents. For datasets and services that are not in the public domain, proprietary datasets or service loaders are mainly used to convert data in specific formats for applications or cloud services, and we need access tokens and other parameters to access these datasets and services.
Typically, these documents are stored as embeddings in a vector database so that they can be indexed and searched.
2.5. 链
A chain is a series of calls to a language model or other commonly used program. LangChain provides a standard interface to the chain, as well as many chains that are well integrated with common applications.
Chains are the result of logically connecting one or more large language models (LLMs), providing a way to combine various components into a unified application. For example, you can create a chain, receive input from the user, format it using a prompt template, and then send the formatted reply to LLM. You can also generate more complex chains by integrating multiple chains with other components. . LLMChain is considered one of the most widely used methods for querying LLM objects. It formats the input key values provided and the key values that need to be remembered according to the prompt template, and then sends the formatted string to LLM, and LLM generates the returned output. After calling a language model, a series of steps can be taken and a series of calls to the model made. This is even more valuable when you want to use the output of one call as input to another call. In this series of chains, each individual chain has an input and an output, and the output of one step is used as the input of the next step.
2.6. Deputy
Agents enable language models to make decisions, take actions, observe results, and repeat the process until the goal is achieved. LangChain provides standard interfaces for proxies, alternative proxies, and examples of end-to-end proxies.
Some applications may require not only a predetermined sequence of LLM/other tool calls, but an undefined sequence that relies on user input. Such sequences include an "agent" that has access to a range of tools. Based on user input, the agent can determine which of these tools should be called and what the input to that tool should be. The tool is then called with this input and an observation is recorded. The history of tools, tool inputs, and observations is passed back to the agent, which decides what steps to take next. This process is repeated until the agent decides that the tool is no longer needed and responds directly to the user.
3. Build applications using LangChain
We may be experiencing a "Linux moment for AI," where developers must choose between proprietary or open source base models based on the tradeoff between performance and cost.
3.1 Build development environment
First, create a Python-based virtual environment. A virtual environment is an isolated Python environment that allows you to install packages and dependencies specific to a specific project without interfering with the system-wide Python installation or other projects. This isolation helps maintain consistency and avoid potential conflicts between different project requirements.
Then, install LangChain, for example: pip install langchain.
Finally, select a model or models and install the corresponding software package, taking openai as an example, pip install openai. Further, access permissions must be configured, such as obtaining an API key from OpenAI.
3.2 Use of large models
LangChain provides an LLM class specifically designed to interact with various language model providers (such as OpenAI, Hugging Face, etc.). This class provides a standard interface for all LLM types. When using the basic capabilities of large models, you can import the package first and directly call the LLM instance to generate text based on the input of the question.
import os
from langchain.llms import OpenAI
os.environ["OPENAI_API_KEY"] = ""
llm = OpenAI(model_name="text-ada-001", n=2, best_of=2)
result = llm("给我讲个笑话")
print(result)If you want to get multiple responses, you can call the generate() method of the LLM instance. The generate() method requires a prompt list as input, and LLM generates a response for each prompt in the list.
In the scenario of using OpenAI LLM, the return result of llm.generate contains program-specific information, especially the usage status of tokens, such as completion_tokens, < a i=2>total_tokens, statistics of prompt_tokens.
3.3 Build chain services
Building a typical chained service mainly includes the following four parts:
- Combining LLM with prompt templates
- By combining the first The output of the LLM serves as input to a second LLM, combining multiple LLMs together in sequence (see this section)
- Combining LLMs with external data, e.g. for question answering
- Combine LLM with long-term memory, e.g. for chat history
If you have already created a Prompt template, you can apply the Prompt template to the large model through LangChain:
from langchain.chains import LLMChain
chain = LLMChain(llm = llm,
prompt = prompt)
chain.run(my_query)If we want to use the output of the first LLM as the input of the second LLM, we can use SimpleSequentialChain:
from langchain.chains import LLMChain, SimpleSequentialChain
# Define the first chain as in the previous example
# ...
# Create a second chain with a prompt template and an LLM
second_prompt = PromptTemplate(
input_variables=["company_name"],
template="Write a business domain for the following company: {company_name}",
)
chain_two = LLMChain(llm=llm, prompt=second_prompt)
# Combine the first and the second chain
overall_chain = SimpleSequentialChain(chains=[chain, chain_two], verbose=True)
# Run the chain specifying only the input variable for the first chain.
catchphrase = overall_chain.run(my_query)One limitation of LLMs is that they lack contextual information (e.g., not having access to some specific documents or emails), we can solve this problem by giving LLMs access to specific external data. LangChain provides various loaders for different types of documents. For example, load all PDF files under a certain path in my local area——
from langchain.document_loaders import DirectoryLoader
loader = DirectoryLoader(
'./Abel/ePapers/llm',# my local directory
glob='**/*.pdf',# only get pdfs
show_progress=True
)
papers_llm = loader.load()
papers_llmAfter the external data is ready as a "document", it can be indexed in a vector database using a text embedding model. Popular vector databases include Pinecone, Weaviate and Milvus, Faiss which already requires no API key, and more.
# pip install faiss-cpu
from langchain.vectorstores import FAISS
# create the vectorestore to use as the index
db = FAISS.from_documents(documents, embeddings)Once the document is stored in an embedded form in a vector database, we can perform various operations on this external data, such as using it for question and answer tasks using information retrievers:
from langchain.chains import RetrievalQA
retriever = db.as_retriever()
qa = RetrievalQA.from_chain_type(
llm=llm,
chain_type="stuff",
retriever=retriever,
return_source_documents=True)
query = "What am I never going to do?"
result = qa({"query": query})
print(result['result'])For applications like chatbots, being able to remember information from previous conversations is crucial. But by default, LLM does not have any long-term memory unless the user manually enters the chat history. LangChain provides several ways to handle chat history by:
Keep all conversations,
Keep the latest k conversations,
Summarize the conversation. For example, we use ConversationChain to provide conversation history information to this application.
from langchain import ConversationChain
conversation = ConversationChain(llm=llm, verbose=True)
conversation.predict(input="Alice has a parrot.")
conversation.predict(input="Bob has two cats.")
conversation.predict(input="How many pets do Alice and Bob have?")In addition to historical information, LLM still has some limitations, such as the inability to access specific knowledge not included in the training data, and the data may also become outdated quickly (for example, GPT-4 was trained on data before September 2021 ), and they're not good at math. Therefore, we need to use agents to make decisions based on the output of LLM, deciding which tools to use to complete the task. For example, by building a proxy, use Wikipedia to find Barack Obama's date of birth, and then use a calculator to calculate his age in 2022.
# pip install wikipedia
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
tools = load_tools(["wikipedia", "llm-math"], llm=llm)
agent = initialize_agent(tools,
llm,
agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
verbose=True)
agent.run("When was Barack Obama born? How old was he in 2022?")In the process of using LangChain, there are some chains that do not require LLM, mainly the preprocessing conversion chain of Prompt prompts, such as removing extra spaces before entering it into LLM. Another reference is https://python.langchain.com/en /latest/modules/chains/generic/transformation.html.
4. Typical use cases based on LangChain
As an advanced language model application development framework, LangChain empowers developers to create various intelligent language applications based on the underlying language model. Common use cases are as follows:
Autonomous Agents: LangChain supports the development of autonomous agents, such as AutoGPT and BabyAGI, which are long-running agents that perform multiple steps to achieve a goal.
Agent Simulation: LangChain facilitates the creation of sandbox environments where agents can interact with each other or react to events, providing insights into their long-term memory capabilities.
Personal Assistant: LangChain is ideal for building a personal assistant that can perform actions, remember interactions and access your data to provide personalized assistance.
Q&A: LangChain excels at answering questions in specific documents, using the information in those documents to construct accurate and relevant answers.
Chatbot: Leveraging the text generation capabilities of language models, LangChain empowers the creation of engaging chatbots.
Querying tabular data: LangChain provides guidance on using language models to query data stored in tabular formats (such as CSV files, SQL databases, or data frames).
Code Understanding: LangChain assists in using language models to query and understand source code from platforms such as GitHub.
Interacting with APIs: LangChain enables language models to interact with APIs, providing them with up-to-date information and the ability to take action based on real-time data.
Extraction: LangChain helps extract structured information from unstructured text, simplifying data analysis and interpretation.
Summary: LangChain supports summarizing longer documents into concise, easily digestible chunks of information, making it a powerful tool for data enhancement.
Evaluation: Since generative models are difficult to evaluate using traditional metrics, LangChain provides hints and chains to assist in the evaluation process using the language model itself.
5. Summary
LangChain gives developers the ability to combine LLM with other sources of computing and knowledge to build applications. Using LangChain, developers can use a framework that abstracts the core building blocks of LLM applications. Explore LangChain's capabilities and try out its individual components and you'll find the possibilities are nearly endless. The LangChain framework provides a flexible and modular approach to language generation, allowing the creation of custom solutions tailored to the specific needs of the user.
However, LangChain is suspected of complicating simple problems. There may be better options waiting for us to discover and explore. So, stay curious and keep learning, the world of LLM and generative AI is still evolving rapidly, presenting both opportunities and challenges.
[Reference materials and related reading]
LangChain Chinese website: https://www.langchain.asia/
LangChain documentation: https://python.langchain.com/docs/get_started/introduction
LangChain Blog: https://blog.langchain.dev/
https://minimaxir.com/2023/07/langchain-problem/
Qcon2023: The growth of technical people in the era of large models (Simplified)
Thesis study notes: Reinforcement learning applied to OS scheduling