Use LangChain to build large language model applications

Use LangChain to build large language model applications

Since the release of ChatGPT, Large Language Models (LLMs) have gained popularity. While you may not have the money and computing resources to train your own large language model from scratch, you can still use pretrained large language models to build cool stuff like:

• A personal assistant that can interact with the outside world based on your data
• Chatbots customized for your purposes
• Analyze or summarize your documentation or code

Big language models are changing the way we build AI products .

Leveraging APIs and hinting engineering, big language models are changing the way we build AI-driven products. A new technical name " LLMOps " was born from this - LangChain is one of the most popular tools.

What is LangChain?

LangChain is a framework designed to help you easily build large language model applications. It provides the following functions:

• Provide a unified interface for a variety of different basic models (see Models )
• Framework to help manage prompts (see Prompts )
• A centralized set of interfaces for working with long-term memory (see Memory ), external data (see Indexes ), other LLMs (see Chains ), and other proxies for tasks that LLMs cannot handle (e.g. computation or search).

Because LangChain has so many different functions, it might be difficult to understand what it does at first. So in this article I will introduce the (current) six key modules of LangChain so that you can better understand its functionality.

Environment build

To continue with this tutorial, you need to have langchainthe Python package installed and the associated API key required.

Install LangChain

Before installing langchainthe package , make sure your Python version is ≥ 3.8.1 and < 4.0.

langchainThe easiest way to install Python packages is via pip installation.

pip install langchain

After the installation is complete, you can import langchainthe package , which outputs langchainthe version number of the package. If you can see the version number in the control output, it means that langchainthe package was installed successfully.

>>> import langchain
>>> langchain.__version__
'0.0.154'

⚠ NOTE: For this tutorial, I am using langchainversion 0.0.154. The LangChain project is very active and the repository code is updated frequently; so please make sure you are using the same or compatible version as mine.

API key

Building applications with LLM requires you to provide an API key to call services, and some APIs have associated fees.

LLM provider (required) - You first need to choose a large language model provider and obtain its API key. We are currently experiencing a "Linux moment for AI" where developers must make trade-offs between performance and cost, choosing between a proprietary or open source base model.

Proprietary models are closed-source base models owned by companies with large teams of experts and large AI budgets. They are usually larger than open source models and thus perform better, but their APIs are also expensive. Typical examples of proprietary model providers include OpenAI , co:here , AI21 Labs , and Anthropic .

Most of the LangChain tutorials use OpenAI, but please note that the OpenAI API is not free, and even the GPT-4 API is expensive, please refer to "GPT-4 API Interface Call and Price Analysis" for details . To obtain an OpenAI API Key, you need an OpenAI account, and then click "Create new secret key" on the API keys page to create an API Key (see the figure below).

After you have the API Key, you can save the Key to an environment variable for use by each module of the code:

import os
os.environ["OPENAI_API_KEY"] = ... # 填入OpenAI Secret Key

Open source models are typically smaller and less powerful than proprietary models, but open source models are more cost-effective than proprietary models. Well-known open source models are:

• BigScience's BLOOM
• Meta AI的LLaMA
• Google's Flan-T5
• Eleuther AI 的GPT-J

For more open source models, please refer to "Summary of Open Source Large Language Models (LLM)" .

Many open source models are hosted on Hugging Face . To obtain a Hugging Face API key, you need a Hugging Face account and create a "New token" under Access Tokens (see image below).

Similarly, you can save the Hugging Face Access Token in an environment variable for use by other modules:

import os

os.environ["HUGGINGFACEHUB_API_TOKEN"] = ... # 插入Hugging Face Access Token

vector database (optional)

If you want to use a specific vector database like Pinecone , Weaviate or Milvus , you need to register separately to get an API key. In this tutorial, we use Faiss which requires no registration .

LangChain 6 core modules

langchainThe package provides a common interface to many underlying models, supports hint management, and acts as a central interface to other components such as hint templates, other large language models, external data, and other tools through proxies.

LangChain has 6 core modules:

• Models : choose from different LLM and embedding models
• Prompts : manage LLM input
• Chains : Combining LLM with other components
• Indexes : access to external data
• Memory : Remember previous conversations
• Agents : access to other tools

Models

Currently, many different large language models are emerging. LangChain provides an integrated interface with various models, and provides a streamlined interface for all models.

LangChain distinguishes three models with different inputs and outputs:

• LLMs accept a string as input (prompt) and output a string

# 专有模型，例如：OpenAI
# pip install openai
from langchain.llms import OpenAI
llm = OpenAI(model_name="text-davinci-003")

# 或者托管在Hugging Face 上的开源模型
# pip install huggingface_hub
from langchain import HuggingFaceHub
llm = HuggingFaceHub(repo_id = "google/flan-t5-xl")

# llm实例接受一个提示输入，返回字符串作为输出
prompt = u"中国的首都是？"
completion = llm(prompt)

• Chat Model is similar to LLM, input the list of chat messages into the model and return a chat message.
• The Text embedding model takes text input and returns a list of floats (embeddings), which are numeric representations of the input text. Text embedding helps to extract information from text for subsequent use. For example, for computing similarity between texts such as movie summaries.

# 专有文本嵌入模型，例如：OpenAI
# pip install tiktoken
from langchain.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()

# 或者托管在Hugging Face 上的开源文本嵌入模型
# pip install sentence_transformers
from langchain.embeddings import HuggingFaceEmbeddings
embeddings = HuggingFaceEmbeddings(model_name = "sentence-transformers/all-MiniLM-L6-v2")

# 文本嵌入模型接受文本输入输出浮点数列表
text = u"中国的首都是？"
text_embedding = embeddings.embed_query(text)

Prompts

While feeding hints to large language models in natural language is intuitive, getting the expected good output requires quite a bit of tweaking of the hints. This process is called hint engineering.

In large language model usage, good hints are precious. Therefore, once you have a good tip, it is recommended that you save it as a template for later reuse. For this, LangChain provides PromptTemplates, which helps you build hints from multiple components.

from langchain import PromptTemplate

template = u"请为宠物{animal}起一个好听的名字。"

prompt = PromptTemplate(
    input_variables=["animal"],
    template=template,
)

prompt.format(animal="猫")

The hint above can be seen as a zero-shot problem setting where you want the LLM to be trained on enough relevant data to provide satisfactory responses.

Another trick to improve the output of the LLM is to add some examples in the prompt, the so-called few-shot problem setting .

from langchain import PromptTemplate, FewShotPromptTemplate

examples = [
    {
    
    "word": "高兴", "antonym": "悲伤"},
    {
    
    "word": "高大", "antonym": "矮小"},
]

example_template = """
词语: {word}
反义词: {antonym}\n
"""

example_prompt = PromptTemplate(
    input_variables=["word", "antonym"],
    template=example_template,
)

few_shot_prompt = FewShotPromptTemplate(
    examples=examples,
    example_prompt=example_prompt,
    prefix="给出输入词语的反义词",
    suffix="词语: {input}\n反义词:",
    input_variables=["input"],
    example_separator="\n",
)

few_shot_prompt.format(input="美丽")

The above code will generate a prompt template and compose the following prompt based on the provided example and input:

给出输入词语的反义词

词语: 高兴
反义词: 悲伤


词语: 高大
反义词: 矮小

词语: 美丽
反义词:

Chains

Chains in LangChain describe the process of combining large language models with other components to create applications. Here are some examples:

• Combining LLM with prompt templates (see this section)
• Cascading multiple LLMs using the output of the first LLM as the input of the second LLM (see this section)
• Combining LLM with external data, e.g. for question answering (see Indexes )
• Combine LLM with Memory, such as chat history (see Memory )

In the previous section, we created a prompt template. When we want to use it with our LLM, we can use it as follows LLMChain:

from langchain.chains import LLMChain

chain = LLMChain(llm = llm, prompt = prompt)

chain.run("猫")

If we want to use the output of the first LLM as the input of the second LLM, we can use SimpleSequentialChain:

from langchain.chains import LLMChain, SimpleSequentialChain

# 前面的代码定义了第一个chain
# ...

# 创建第二个chain
second_prompt = PromptTemplate(
    input_variables=["petname"],
    template="写一篇关于{petname}的小诗。",
)

chain_two = LLMChain(llm=llm, prompt=second_prompt)

# 将两个chain串联在一起
overall_chain = SimpleSequentialChain(chains=[chain, chain_two], verbose=True)

# 只需给出源头输入结合顺序运行整个chain
catchphrase = overall_chain.run("猫")

Indexes

A limitation of LLM is the lack of contextual information (eg access to some specific documents or emails). You can work around this by granting LLM access to specific external data.

To do this, you first need to load external data using a document loader. LangChain provides multiple loaders for different types of documents, from PDFs and emails to websites and YouTube videos.

Let's start with the simplest text file loading.

from langchain.document_loaders import TextLoader

loader = TextLoader('../state_of_the_union.txt', encoding='utf8')

documents = loader.load()

If the text is large, you can use CharacterTextSplitterto split the document:

from langchain.text_splitter import CharacterTextSplitter

text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)

texts = text_splitter.split_documents(documents)

In the code above, chunk_sizerepresents the maximum size of the block, lengh()measured by . chunk_overlapIndicates the maximum overlap between blocks. Having some overlap is nice to keep some continuity between blocks (e.g. do a sliding window). The final segmented text is in textsthe tuple , which can be read through the subscript index.

Once you have external data, you can index it using a text embedding model (see Model) in a vector database. There are many popular vector databases, for this article, I use Faiss because it does not require an API key.

# pip install faiss-cpu
from langchain.vectorstores import FAISS

# 创建 vectorestore 用作索引
db = FAISS.from_documents(texts[0], embeddings)

The above code saves the text as word embeddings into the vector database. These external data can do many things, below we will use these external data to make a question answering robot with retrieval function:

from langchain.chains import RetrievalQA

retriever = db.as_retriever()

qa = RetrievalQA.from_chain_type(
    llm=llm, 
    chain_type="stuff", 
    retriever=retriever, 
    return_source_documents=True)

query = u"中国古典四大名著是什么?"
result = qa({
    
    "query": query})

print(result['result'])

The above code will retrieve the closest answer from the database for the input question.

Memory

For applications like chatbots, they must be able to remember previous conversations. But by default, LLM doesn't have any long-term memory unless you feed it chat logs.

LangChain provides several different options for handling chat history:

• keep all conversations
• Keep the latest $k$ conversations
• sum up the conversation

In the following example, we will use to provide session memory ConversationChainto the application.

from langchain import ConversationChain

conversation = ConversationChain(llm=llm, verbose=True)

conversation.predict(input="所有的北极熊都是白色的")

conversation.predict(input="bob是一只北极熊")

conversation.predict(input="bob是什么颜色的?")

Without ConversationChaina to hold the dialog memory, the big model will say that it doesn't have enough information to answer the question. ConversationChainIt's a good answer to use .

Agents

Although the large language model is very powerful, it also has some limitations: such as the lack of domain-specific knowledge in the training data, and the training data of ChatGPT is as of September 2021.

To augment large language models, we can make large models accessible to auxiliary tools, such as search engines, calculators, or Wikipedia. In addition, we need the agent to decide which tools to use to complete the task based on the output of the LLM.

In the example below, the Agent first uses Wikipedia to look up Barack Obama's date of birth, and then uses a calculator to calculate his age in 2023.

# pip install wikipedia
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType

tools = load_tools(["wikipedia", "llm-math"], llm=llm)
agent = initialize_agent(tools, 
                         llm, 
                         agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, 
                         verbose=True)


agent.run("奥巴马的生日是哪天? 到2023年他多少岁了?")

Summarize

LangChain is an open-source Python library that anyone who can write code can use to build LLM-powered applications. This package provides a common interface to many underlying models, supports prompt management, and at the time of writing acts as a central interface to other components such as prompt templates, other LLMs, external data, and other tools.

The library provides a lot more functionality than mentioned in this article, and it is iterating so fast that at the current rate of development, this article may be out of date in a month. It is recommended that everyone pay attention to the LangChain project, keep an eye on its new function releases, and refer to LangChain's official documents.