Theory + practice detailed explanation of the hottest LLM application framework LangChain

17120408:

From data to large model application, on November 25th, Hangzhou Yuanchuang Conference was held to share development tips.

This article is shared from Huawei Cloud Community " What is LangChain?" Detailed introduction and usage scenarios of LangChain ", author: Ma Shanghua_Lancer.

1. Concept introduction

1.1 What is Langchain?

The official definition is: LangChain is a powerful framework designed to help developers build end-to-end applications using language models. It provides a set of tools, components, and interfaces that simplify the creation of large language models (LLM) and chat models. The process of providing supported applications. LangChain is a language model integration framework whose use cases broadly overlap with those of language models, including document analysis and summarization, chatbots, and code analysis.

Simply put, LangChain provides flexible abstractions and AI-first tools to help developers take LLM applications from prototype to production environment. It also provides a set of tools to help developers build context-aware, reasoning applications. LangChain’s tools include chatbots, document analysis, summarization, code analysis, workflow automation, custom search, and more.

1.2 How to use LangChain?

To use LangChain, developers first import the necessary components and tools, such as LLMs, chat models, agents, chains, and memory functions. These components combine to create an application that understands, processes, and responds to user input.

LangChain provides multiple components for specific use cases, such as personal assistants, document Q&A, chatbots, querying tabular data, interacting with APIs, extraction, evaluation and aggregation.

For the detailed local construction process, please go to: https://bbs.huaweicloud.com/blogs/414657

2. Main components:

Model I/O: manages large language models (Models), their input (Prompts) and formatted output (Output Parsers)

Data connection: manages vector data stores (Vector Stores), content data acquisition (Document Loaders) and transformations (Transformers) mainly used to build private domain knowledge (library), as well as vector data queries (Retrievers)
Memory: Function module for storing and obtaining conversation history
Chains: used to connect Memory ↔️ Model I/O ↔️ Data Connection to achieve serialized continuous dialogue and speculation process
Agents: further connect tools (Tools) based on Chains, thereby combining the capabilities of large language models with local and cloud service capabilities
Callbacks: Provides a callback system that can be connected to each stage of LLM application to facilitate data diversion such as logging and tracking.

Langchain core module architecture diagram:

3. Core module

3.1 Model I/O

The model is connected to the interactive component of LLM, which is used to complete business interaction with different types of models. LangChain divides the model into two model modes: LLMS and Chat Model, and completes the business interaction of the three models through different template operations.

3.2 LLMs

It refers to a commercial large-scale language model with language understanding and generation capabilities. It takes text strings as input and returns text strings as output. The LLM class is designed in LangChain for interface interaction with large language models. This class aims to provide standard interfaces for LLM providers, such as OpenAI, Cohere, and Hugging Face.

Taking the OpenAI LLM wrapper as an example, its usage is as follows:

from LangChain.llms import OpenAI llm = OpenAI(temperature=0, model_name='gpt-3.5-turbo', openai_api_key=OPENAI_API_KEY) llm("Please introduce yourself") llm.get_num_tokens(question)

3.3 Chat

The chat model is a variant of the language model. The chat model is based on the language model. It uses the language model internally. It no longer uses text strings as input and output, but lists chat information as input and output. They provide a more structured ized API. One or more messages can be delivered through the chat model. LangChain currently supports four message types: AIMessage, HumanMessage, SystemMessage and ChatMessage.

SystemMessage: System message is a tool used to set the model. It can be used to specify the specific environment and background of the model, such as role playing, etc.;
HumanMessage: Human message is user information, information given by people, such as asking questions; to use the Chat Model model, you have to put the system messages and human messages in a list, and then use it as the input of the Chat Model model
AIMessage: It is the message output by AI, which can be an answer to a question.
ChatMessage: Chat message can accept parameters of any role

In most cases, we only need to deal with the HumanMessage, AIMessage and SystemMessage message types. In addition, the chat model supports multiple messages as input, as shown below are examples of system messages and user messages.

messages = [ SystemMessage(cnotallow="You are a helpful assistant that translates English to French."), HumanMessage(cnotallow="I love programming.") ] chat(messages) AIMessage(cnotallow="J'aime programmer.", additional_kwargs={})

Use generate to generate multiple sets of messages

batch_messages = [ [ SystemMessage(cnotallow="You are a helpful assistant that translates English to French."), HumanMessage(cnotallow="I love programming.") ], [ SystemMessage(cnotallow="You are a helpful assistant that translates English to French."), HumanMessage(cnotallow="I love artificial intelligence.") ], ] result = chat.generate(batch_messages)

3.4 Prompts

Prompt refers to the input of the model. This input is rarely hard-coded, but is constructed using a specific template component. This template component is the PromptTemplate prompt template, which can provide a prompt template as input. Template Refers to the specific format and blueprint of the answers we hope to obtain. LangChain provides pre-designed prompt templates that can be used to generate prompts for different types of tasks. When the preset templates cannot meet the requirements, you can also use customized prompt templates.

In LangChain, we can set the prompt template as needed and connect it with the main chain for output prediction. In addition, LangChain also provides the function of output parser for further refining the results. The role of the output parser is to guide how the model output should be formatted and parse the output into the required format.

Several classes and functions are provided to make building and handling prompts easy:

3.4.1 PromptTemplate prompt template

Text templates can be generated and spliced into complete statements in the form of variable parameters:

from langchain.llms import OpenAI from langchain import PromptTemplate import os openai_api_key = os.environ["OPENAI_API_KEY"] # Use openAi model llm = OpenAI(model_name="gpt-3.5-turbo", openai_api_key=openai_api_key) # Template format template = " I feel like eating {value}. How should I make it?" # Build template prompt = PromptTemplate( input_variables=["value"], template=template, ) # Template generated content final_prompt = prompt.format(value='Braised Pork' ) print("Input content::", final_prompt) print("LLM output:", llm(final_prompt))

Input content: I want to eat braised pork. How should I make it? LLM output: The steps for making braised pork are as follows: 1. Prepare materials: 500 grams of pork belly, 2 garlic cloves, 1 piece of ginger, 2 spoons of sugar, 3 spoons of light soy sauce, 1 spoon of dark soy sauce, 2 spoons Cooking wine, 500 ml water. 2. Cut the pork belly into 2-3cm square pieces. 3. Boil the cut pork belly in cold water, blanch it to remove the fishy smell, remove it and set aside. 4. Heat the pan with cold oil, add ginger slices and garlic cloves and stir-fry until fragrant. 5. Add the blanched pork belly to the pot and fry until both sides are slightly brown. 6. Add sugar and stir-fry over low heat until sugar dissolves and changes color. 7. Add light soy sauce and dark soy sauce, stir-fry evenly and color evenly. 8. Pour in the cooking wine and stir-fry evenly. 9. Add the water used to cook the meat, enough to slightly cover the pork belly, then add a small tea bag or spice bag to enhance the flavor, cover the pot, and bring to a boil over high heat. 10. Turn to low heat and simmer for 40-50 minutes. During this period, the meat should be cooked frequently to keep the meat evenly colored. 11. Finally collect the juice. Once the juice is thickened, turn off the heat. 12. Take out the braised pork, slice it and serve on a plate.

3.4.2 FewShotPromptTemplate selector

The sample content of the prompt is also spliced into the sentence, allowing the model to understand the semantic meaning and then give the result.

from langchain.prompts.example_selector import SemanticSimilarityExampleSelector from langchain.vectorstores import FAISS from langchain.embeddings import OpenAIEmbeddings from langchain.prompts import FewShotPromptTemplate, PromptTemplate from langchain.llms import OpenAI import os ## prompt Selector example openai_api_key = os.environ["OPEN AI_API_KEY "] llm = OpenAI(model_name="gpt-3.5-turbo", openai_api_key=openai_api_key) example_prompt = PromptTemplate( input_variables=["input", "output"], template="Example input:{input}, example output:{ output}", ) # This is a list of examples to choose from examples = [ {"input": "Pilot", "output": "Aircraft"}, {"input": "Pilot", "output": " car"}, {"input": "Chef", "output": "Kitchen"}, {"input": "Stewardess", "output": "Airplane"}, ] # Select input similar to yours based on semantics Examples example_selector = SemanticSimilarityExampleSelector.from_examples( examples, # Generate embedding classes for measuring semantic similarity. OpenAIEmbeddings(openai_api_key=openai_api_key), # Store word vectors FAISS, # Number of generated examples k=4 ) # Selector examples prompt similar_prompt = FewShotPromptTemplate( example_selector=example_selector, example_prompt=example_prompt, # Prompt items added to the top and bottom of the prompt prefix="Write the output according to the following example", suffix="Input: {value}, output:", # Input Variable input_variables=["value"], ) value = "Student" # Template generated content final_prompt = similar_prompt.format(value=value) print("Input content::", final_prompt) print("LLM output:", llm( final_prompt))

Input content:: Based on the following example, write the output Example input: Chef, Example output: Kitchen Example input: Driver, Example output: Car Example input: Pilot, Example output: Airplane Example input: Stewardess, Example output: Airplane Input: Student, Output: LLM Output: Classroom

3.4.3 ChatPromptTemplate chat prompt template

Generate a complete prompt template using chat messages as input.

from langchain.schema import HumanMessage from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate # We will use the chat model, the default is gpt-3.5-turbo from langchain.chat_models import ChatOpenAI # Parse the output and get back the structured data from langchain.output_parsers import StructuredOutputParser, ResponseSchema import os openai_api_key = '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' chat_model = ChatOpenAI(temperature=0, model_name='gpt-3.5-turbo', openai_api_key=openai_api_key) prompt = ChatPromptTemplate( messages =[HumanMessagePromptTemplate.from_template("According to user content, Extract the company name and region name, use user content: {user_prompt}") ], input_variables=["user_prompt"] ) user_prompt = "Is Shuidi company headquartered in Beijing?" fruit_query = prompt.format_prompt(user_prompt=user_prompt) print( 'Input content:', fruit_query.messages[0].content) fruit_output = chat_model(fruit_query.to_messages()) print('LLM output:', fruit_output)

Input content: Based on the user content, extract the company name and region name, use the user content: Is Shuidi company headquartered in Beijing? LLM output: cnotallow='Company name: Shuidi Company\nRegion name: Beijing' additional_kwargs={} example=False

3.4.4 StructuredOutputParser output parser

The output parser refers to the component that parses and processes the results generated by the model. Its main function is to parse the text generated by the model, extract useful information and perform subsequent processing. Such as parsing the text generated by the model, extracting useful information, identifying entities, classifying and filtering the results, and post-processing the generated text to make the generated results easier to understand and use. It plays a role in parsing and processing results when interacting with large language models, enhancing the application and usability of the model.

The language model outputs text. But many times, you may want to obtain more structured information than text. This is what the output parser does. That is, the output parser is a class that helps the structured language model respond. The main class provided in LangChain is PydanticOutputParser.

3.5 Data connection

Open up the pipeline of external data, including document loading, document conversion, text embedding, and vector storage. This module contains utility functions for processing documents, different types of indexes, and the use of these indexes in the chain.

External data and LLM can be combined to understand and generate natural language. The external data can be local documents, databases and other resources. These data are sliced and vectorized and stored in the vector storage database, and then the vector database is retrieved through the user's Prompt. The similar information in the language is passed to the large language model to generate and execute actions.

Document loader

The focus includes txt (TextLoader), csv (CSVLoader), html (UnstructuredHTMLLoader), json (JSONLoader), markdown (UnstructuredMarkdownLoader) and pdf (because the format of pdf is more complex, PyPDFLoader, MathpixPDFLoader, UnstructuredPDFLoader, PyMuPDF and other forms are provided Loading engine) Content parsing of several commonly used formats.

3.5.2 Document transformers Document converters

There are a number of built-in document converters that make it easy to split, combine, filter, and otherwise manipulate documents, with a focus on recursively splitting by character with the RecursiveCharacterTextSplitter .

3.5.3 Text embedding models Text embedding

The Embeddings base class in LangChain exposes two methods: one for embedding documents and another for embedding queries. The former takes multiple texts as input, while the latter takes a single text. The reason for having these as two separate methods is that some embedding providers have different embedding methods for the document (the one being searched) versus the query (the search query itself).

The text embedding model text-embedding-model represents the text as a vector, so that operations such as semantic search can be performed on the text in the vector space, that is, searching for the most similar text fragments in the vector space. These are implemented through the Embedding class in LangChain.

The Embedding class is a class for interacting with text embeddings. This class aims to provide a standard interface to providers (there are many embedding providers such as OpenAI, Cohere, Hugging Face, etc.)

from langchain.schema import HumanMessage from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate # We will use the chat model, the default is gpt-3.5-turbo from langchain.chat_models import ChatOpenAI # Parse the output and get back the structured data from langchain.output_parsers import StructuredOutputParser, ResponseSchema import os openai_api_key = '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' chat_model = ChatOpenAI(temperature=0, model_name='gpt-3.5-turbo', openai_api_key=openai_api_key) prompt = ChatPromptTemplate( messages =[HumanMessagePromptTemplate.from_template("According to user content, Extract the company name and region name, use user content: {user_prompt}") ], input_variables=["user_prompt"] ) user_prompt = "Is Shuidi company headquartered in Beijing?" fruit_query = prompt.format_prompt(user_prompt=user_prompt) print( 'Input content:', fruit_query.messages[0].content) fruit_output = chat_model(fruit_query.to_messages()) print('LLM output:', fruit_output)

3.5.4 VectorStores vector storage

One of the most common ways to store and search unstructured data is to embed it and store the resulting embedding vector, and then at query time embed the unstructured query and retrieve the embedding vector that is "most similar" to the embedded query.

Vector storage is responsible for storing embedded data and performing vector searches for you. . The key part of dealing with vector storage is creating the vectors to put into it, usually through embedding. This is an explanation of the encapsulation interface of commonly used vector databases (Chroma, FAISS, Milvus, Pinecone, PGVector, etc.). The general process is: initialize database connection information -> create index -> store vectors -> similarity query. The following is Chroma For example:

# Load files loader = TextLoader(filePath) documents = loader.load() # Slice data text_splitter = CharacterTextSplitter( chunk_size=chunkSize, chunk_overlap=0, length_functinotallow=len, separator=separator) split_docs = text_splitter.split_documents(documents) # Initial Request vectorized data embeddings = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY) # Persist file address persist_directory = '/data/' + collectName # Execute vectorized vectorstore = Chroma.from_documents( split_docs, embeddings, persist_directory=persist_directory) # Persist to local vectorstore .persist()

3.5.5 Retrievers query

A retriever is an interface that returns documents based on an unstructured query. It is more general than vector storage. The retriever does not need to be able to store the document, just return (or retrieve) it. Vector stores can be used as the backbone of retrievers, but there are other types of retrievers as well.

The retriever interface is a general-purpose interface that makes document and language models easy to compose. There is a get_relevant_documents method exposed in LangChain, which accepts a query (string) and returns a list of documents.

Focusing on data compression, the purpose is to obtain the most relevant text and bring it into the prompt context, which can not only reduce token consumption, but also ensure the output quality of LLM.

from langchain.llms import OpenAI from langchain.retrievers import ContextualCompressionRetriever from langchain.retrievers.document_compressors import LLMChainExtractor from langchain.document_loaders import TextLoader from langchain.vectorstores import FAISS documents = TextLoader('../../../state_of_the_union.txt' ).load() text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0) texts = text_splitter.split_documents(documents) retriever = FAISS.from_documents(texts, OpenAIEmbeddings()).as_retriever() docs = retriever.get_relevant_documents("What did the president say about Ketanji Brown Jackson") # Basic search will return one or two relevant documents and some irrelevant documents. Even the relevant documents have a lot of irrelevant information pretty_print_docs(docs) llm = OpenAI(temperature=0 ) compressor = LLMChainExtractor.from_llm(llm) # Iterate through the initially returned documents and extract only the content relevant to the query from each document compression_retriever = ContextualCompressionRetriever(base_compressor=compressor, base_retriever=retriever) compressed_docs = compression_retriever.get_relevant_documents("What did the president say about Ketanji Jackson Brown") pretty_print_docs(compressed_docs)

3.5.6 Caching Embeddings Caching embeddings

Embeddings can be stored or temporarily cached to avoid the need to recompute them. Cache embeddings can be done using CacheBackedEmbeddings.

3.5.7 Memory

Memory is the concept of maintaining state throughout the user's interaction with the language model. It is reflected in the interactive chat message process between the user and the language model, which involves ingesting, capturing, transforming and extracting knowledge from a series of chat messages. Memory maintains state between Chains/Agents calls. By default, Chains and Agents are stateless, which means they handle each incoming query independently, but in some applications, such as chatbots, It's important to remember previous interactions, both in the short and long term. The concept of "Memory" is to achieve this.

There are two ways to use the memory storage component. One is to provide auxiliary tools for managing and operating previous chat messages to extract information from the message sequence; the other is to use it for correlation in Chains. Memory can return multiple pieces of information, such as the last N messages or summaries of all previous messages. The returned information can be a string or a message list.

LangChain provides method classes and interfaces for extracting memory information from chat records, buffer memories, and Chains, such as the ChatMessageHistory class, an ultra-lightweight wrapper that provides some convenient methods to save human messages and AI messages, and then extract them from Get them; another example is the ConversationBufferMemory class, which is a wrapper for ChatMessageHistory and is used to extract messages from variables and so on.

from langchain.memory import ChatMessageHistory from langchain.chat_models import ChatOpenAI import os openai_api_key=os.environ["OPENAI_API_KEY"] chat = ChatOpenAI(temperature=0, openai_api_key=openai_api_key) # Statement history history = ChatMessageHistory() history.add_user_message("you is a good AI robot that can help users find out where to travel in a short sentence") history.add_user_message("I like the beach, where should I go?") # Add AI language history.add_ai_message("You Should go to Shenzhen, Guangdong") # Add human language history.add_user_message("What else should I do while I'm there?") print('history message:', history.messages) # Call model ai_response = chat(history.messages ) print('result', ai_response) # Continue to add AI language history.add_ai_message(ai_response.content) print('history message:', history.messages) # Continue to add human language history.add_user_message("Recommend food and shopping places ") ai_response = chat(history.messages) print('result', ai_response)

3.5.8 Chains

Usually we can solve the problem using a separate LLM, but for more complex applications, we need to link between LLMs or with other systems to complete the task. This is usually called linked LLM.

Chains allow multiple components between models or systems to be combined to create a single, consistent application. For example, we create a chain that accepts input from the user, formats the input through the PromptTemplate template, and passes it to the LLM language model. Multiple chains can also be combined, or chains can be combined with other system components to build more complex chains and achieve more powerful functions. LangChain provides standard interfaces as well as common implementations for chains, has extensive integrations with other tools, and provides end-to-end chains for common applications.

Basic sequence chain:

from langchain.llms import OpenAI from langchain.chains import LLMChain from langchain.prompts import PromptTemplate from langchain.chains import SimpleSequentialChain import os openai_api_key = os.environ["OPENAI_API_KEY"] llm = OpenAI(temperature=1, openai_api_key=openai_api_key) # No. A chain template content template1 = "Recommend a suitable region based on the description entered by the user, the user input: {value}" prompt_template1 = PromptTemplate(input_variables=["value"], template=template1) # Build the first chain chain1 = LLMChain(llm=llm, prompt=prompt_template1) # The second chain template content template2 = "Recommend food in the region based on the region entered by the user, the user input: {value}" prompt_template2 = PromptTemplate(input_variables=["value" ], template=template2) # Build the second chain chain2 = LLMChain(llm=llm, prompt=prompt_template2) # Assemble the chain overall_chain = SimpleSequentialChain(chains=[chain1, chain2], verbose=True) # Run chain review = overall_chain.run("I want to see the sea in China") print('Result:', review)

from LangChain.prompts import PromptTemplate from LangChain.llms import OpenAI from LangChain.chains import LLMChain llm = OpenAI(temperature=0.9) prompt = PromptTemplate( input_variables=["product"], template=" Please help me to find a new name for my company that makes {product}?", ) chain = LLMChain(llm=llm, prompt=prompt) print(chain.run("super phone"))

In the above example, accepting user input through template formatting and passing it to the language model can be implemented through a simple chain class, such as the LLMChain class in LangChain. It is a simple chain that accepts a prompt template, formats it with user input, and returns a response from LLM. Of course, LLMChain can also be used in chat models.

3.5.9 Agents

Usually a user's problem may require multiple logical processing of the application to complete the related tasks, and it may often be dynamic, requiring different actions according to different user inputs, or different actions may be executed based on different LLM outputs. Therefore, the application not only needs to predetermine LLM and other tool call chains, but may also need to generate different chains based on different user inputs. Using a proxy can make LLM access tools more direct and efficient. The tools provide unlimited possibilities. LLM can search the network, perform mathematical calculations, run code and other related functions.

The agent uses LLM to determine which actions to take and in what order, review the observations, and repeat until the task is completed. The LangChain library provides a large number of pre-built tools and also allows modification of existing tools or creation of new tools. When proxies are used correctly, they can be very powerful. In LangChain, the concept of "agent" is used to access a series of tools in these types of chains to complete tasks. Based on user input, the agent can decide whether to invoke any of these tools. Here is a simple example:

#Load the language model to be used to control the agent llm = OpenAI(temperature=0); #Load some tools that need to be used, such as serpapi and llm-math. The llm-math tool requires LLM tools = load_tools(["serpapi" , "llm-math"], llm=llm) #Initialize an agent using tools, language models and agent types. agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True) #Control agent execution agent.run("Who is Vin Diesel's girlfriend? What is her current age raised to the 0.5 power?")

3.5.10 Callbacks

LangChain provides a callback system that allows you to connect to various stages of your LLM application. This is useful for logging, monitoring, streaming, and other tasks.

You can subscribe to these events using parameters available throughout the API. The argument is a list of handler objects that are expected to implement one or more methods described in more detail below.

4. How to quickly use LangChain to build an end-to-end language model application on Huawei Cloud Platform ModelArts

Preliminary preparation:

1. Log in to Huawei Cloud official account :

Click "Console" in the upper right corner and enter "ModelArts" in the search bar

Click "Development Environment" - "Notebook", "Create":

Enter to create a notebook, name "notebook-LangChain", select the GPU specification, "GPU: 1*T4 (16GB) | CPU: 8 core 32GB", click "Create Now", select the disk specification "50G", click "Create"

Click to return to "Task Center" and click notebook to enter

1. Install LangChain

First, install LangChain. Just run the following command:

!pip install langchain

Looking in indexes: http://repo.myhuaweicloud.com/repository/pypi/simple
Collecting langchain
  Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/1f/fd/f2aa39f8e63a6fbacf2e7be820b846c27b1e5830af9c2e2e208801b6c07f/langchain-0.0.27-py3-none-any.whl (124 kB)
     |██████████████████████████████████| 124 kB 49.0 MB/s eta 0:00:01
Collecting sqlalchemy
  Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/ac/d8/51e617a1eb143a48ab2dceb194afe40b3c42b785723a031cc29a8c04103d/SQLAlchemy-2.0.20-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB)
     |██████████████████████████████████| 2.9 MB 30.8 MB/s eta 0:00:01
Requirement already satisfied: numpy in /home/ma-user/anaconda3/envs/PyTorch-1.8/lib/python3.7/site-packages (from langchain) (1.19.5)
Requirement already satisfied: pyyaml in /home/ma-user/anaconda3/envs/PyTorch-1.8/lib/python3.7/site-packages (from langchain) (5.1)
Requirement already satisfied: requests in /home/ma-user/anaconda3/envs/PyTorch-1.8/lib/python3.7/site-packages (from langchain) (2.27.1)
Collecting pydantic
  Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/fd/35/86b1e7571e695587df0ddf2937100436dce0caa277d2f016d4e4f7d3791a/pydantic-2.2.1-py3-none-any.whl (373 kB)
     |██████████████████████████████████| 373 kB 55.0 MB/s eta 0:00:01
Collecting typing-extensions>=4.6.1
  Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/38/60/300ad6f93adca578bf05d5f6cd1d854b7d140bebe2f9829561aa9977d9f3/typing_extensions-4.6.2-py3-none-any.whl (31 kB)
Collecting pydantic-core==2.6.1
  Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/c0/ca/4cf24afe80f5839a5cad5e35e2a0a11fe41b0f4f6a544109f73337567579/pydantic_core-2.6.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.9 MB)
     |██████████████████████████████████| 1.9 MB 38.3 MB/s eta 0:00:01
Collecting annotated-types>=0.4.0
  Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/d8/f0/a2ee543a96cc624c35a9086f39b1ed2aa403c6d355dfe47a11ee5c64a164/annotated_types-0.5.0-py3-none-any.whl (11 kB)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ma-user/anaconda3/envs/PyTorch-1.8/lib/python3.7/site-packages (from requests->langchain) (1.26.12)
Requirement already satisfied: certifi>=2017.4.17 in /home/ma-user/anaconda3/envs/PyTorch-1.8/lib/python3.7/site-packages (from requests->langchain) (2022.9.24)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ma-user/anaconda3/envs/PyTorch-1.8/lib/python3.7/site-packages (from requests->langchain) (2.0.12)
Requirement already satisfied: idna<4,>=2.5 in /home/ma-user/anaconda3/envs/PyTorch-1.8/lib/python3.7/site-packages (from requests->langchain) (3.4)
Collecting greenlet!=0.4.17
  Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/1a/ed/72998fb3609f6c4b0817df32e2b98a88bb8510613d12d495bbab8534ebd0/greenlet-2.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (514 kB)
     |██████████████████████████████████| 514 kB 12.0 MB/s eta 0:00:01

2.Environment settings

Working in a Jupyter notebook or Python script, you can set environment variables like this:

import us
os .environ[ "OPENAI_API_KEY" ] = "..."

Building language model applications: LLM

After installing LangChain and setting up the environment, we can start building our language model application. LangChain provides a bunch of modules that you can use to create language model applications. You can combine these modules for more complex applications, or use them individually for simpler applications.

Building a language model application: Chat Model

In addition to LLM, you can also use chat models. These are variations of a language model that use a language model under the hood but have a different interface. The chat model uses chat messages as input and output rather than a "text in, text out" API. The use of chat model APIs is still relatively new, so everyone is still looking for the best way to use the abstraction.

To complete a chat, you need to pass one or more messages to the chat model. LangChain currently supports AIMessage, HumanMessage, SystemMessage and ChatMessage types. You will primarily use HumanMessage, AIMessage, and SystemMessage. Here is an example using the chat model:

from langchain.chat_models import ChatOpenAI
from langchain.schema import (
    LIKEMessage,
    HumanMessage,
    SystemMessage
)

chat = ChatOpenAI(temperature=0)

You can do this by passing a message:

chat([HumanMessage(content="Translate this sentence from English to French. I love programming.")])
# -> AIMessage(content="J'aime programmer.", additional_kwargs={})

Or pass multiple messages to OpenAI's gpt-3.5-turbo and gpt-4 models:

messages = [
    SystemMessage(content="You are a helpful assistant that translates English to Chinese."),
    HumanMessage(content="Translate this sentence from English to Chinese. I love programming.")
]
chat(messages)
# -> AIMessage(content="I like programming. (Wǒ xǐhuān biānchéng.)", additional_kwargs={})

You can also use generate to generate completions for multiple sets of messages. This returns an LLMResult with additional message parameters:

batch_messages = [
    [
        SystemMessage(content="You are a helpful assistant that translates English to Chinese."),
        HumanMessage(content="Translate this sentence from English to Chinese. I love programming.")
    ],
    [
        SystemMessage(content="You are a helpful assistant that translates English to Chinese."),
        HumanMessage(content="Translate this sentence from English to Chinese. I love artificial intelligence.")
    ],
]
result = chat.generate(batch_messages)
result
# -> LLMResult(generations=[[ChatGeneration(text="I like programming. (Wǒ xǐhuān biānchéng.)", generation_info=None, message=AIMessage(content="I like programming. (Wǒ xǐhuān biānchéng.)", additional_kwargs ={}))], [ChatGeneration(text="I like artificial intelligence. (Wǒ xǐ'ài rén gōng zhì néng.)", generation_info=None, message=AIMessage(content="I like artificial intelligence. (Wǒ xǐ'ài rén gōng zhì néng.)", generation_info=None, message=AIMessage(content="I like artificial intelligence. (Wǒ 'ài rén gōng zhì néng.)", additional_kwargs={}))]], llm_output={'token_usage': {'prompt_tokens': 71, 'completion_tokens': 18, 'total_tokens': 89}})

You can also extract token usage and other information from LLMResult:

result.llm_output['token_usage']
# -> {'prompt_tokens': 71, 'completion_tokens': 18, 'total_tokens': 89}

For chat models, you can also use templates by using MessagePromptTemplate. You can create a ChatPromptTemplate from one or more MessagePromptTemplates. The method format_prompt of ChatPromptTemplate returns a PromptValue, which you can convert to a string or a Message object, depending on whether you want to use the formatted value as input to an LLM or a chat model.

Here is an example:

from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (
    ChatPromptTemplate,
    SystemMessagePromptTemplate,
    HumanMessagePromptTemplate,
)
chat = ChatOpenAI(temperature=0)
template="You are a helpful assistant that translates {input_language} to {output_language}."
system_message_prompt = SystemMessagePromptTemplate.from_template(template)
human_template="{text}"
human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)
chat_prompt = ChatPromptTemplate.from_messages([system_message_prompt, human_message_prompt])
# get a chat completion from the formatted messages
chat(chat_prompt.format_prompt(input_language="English", output_language="Chinese", text="I love programming.").to_messages())
# -> AIMessage(content="I like programming. (Wǒ xǐhuān biānchéng.)", additional_kwargs={})

You can also use LLMChain with Chat Model:

from langchain.chat_models import ChatOpenAI
from langchain import LLMChain
from langchain.prompts.chat import (
    ChatPromptTemplate,
    SystemMessagePromptTemplate,
    HumanMessagePromptTemplate,
)
chat = ChatOpenAI(temperature=0)
template="You are a helpful assistant that translates {input_language} to {output_language}."
system_message_prompt = SystemMessagePromptTemplate.from_template(template)
human_template="{text}"
human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)
chat_prompt = ChatPromptTemplate.from_messages([system_message_prompt, human_message_prompt])
chain = LLMChain(llm=chat, prompt=chat_prompt)
chain.run(input_language="English", output_language="Chinese", text="I love programming.")
# -> "I like programming. (Wǒ xǐhuān biānchéng.)"

You can also use agents with chat models. Initialize Agent using AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION as agent type

from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI
# First, let's load the language model we're going to use to control the agent.
chat = ChatOpenAI(temperature=0)
# Next, let's load some tools to use. Note that the `llm-math` tool uses an LLM, so we need to pass that in.
llm = OpenAI(temperature=0)
tools = load_tools(["serpapi", "llm-math"], llm=llm)
# Finally, let's initialize an agent with the tools, the language model, and the type of agent we want to use.
agent = initialize_agent(tools, chat, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
# Now let's test it out!
agent.run("Who is Olivia Wilde's boyfriend? What is his current age raised to the 0.23 power?")

In this example, the agent will interactively perform searches and calculations to provide the final answer.

Finally, let's explore using memory with chains and proxies initialized with chat models. The main difference between this and Memory for LLMs is that we can keep previous messages as their own unique memory objects, rather than compressing them into a string.

Here is an example using a ConversationChain:

from langchain.prompts import (
    ChatPromptTemplate, 
    MessagesPlaceholder, 
    SystemMessagePromptTemplate, 
    HumanMessagePromptTemplate
)
from langchain.chains import ConversationChain
from langchain.chat_models import ChatOpenAI
from langchain.memory import ConversationBufferMemory

prompt = ChatPromptTemplate.from_messages([
    SystemMessagePromptTemplate.from_template("The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know."),
    MessagesPlaceholder(variable_name="history"),
    HumanMessagePromptTemplate.from_template("{input}")
])
llm = ChatOpenAI(temperature=0)
memory = ConversationBufferMemory(return_messages=True)
conversation = ConversationChain(memory=memory, prompt=prompt, llm=llm)
conversation.predict(input="Hi there!")
# -> 'Hello! How can I assist you today?'
conversation.predict(input="I'm doing well! Just having a conversation with an AI.")
# -> "That sounds like fun! I'm happy to chat with you. Is there anything specific you'd like to talk about?"
conversation.predict(input="Tell me about yourself.")
# -> "Sure! I am an AI language model created by OpenAI. I was trained on a large dataset of text from the internet, which allows me to understand and generate human-like language. I can answer questions, provide information, and even have conversations like this one. Is there anything else you'd like to know about me?"

In this example, we use aConversationChain to maintain conversation context across multiple interactions with the AI.

That's it! Now you have an in-depth understanding of how to use LangChain to build end-to-end language model applications. By following these examples, you can develop powerful language model applications using LLM, chat models, agents, chains, and memory features.

in conclusion

In summary, LangChain is a powerful framework that simplifies the process of building high-level language model applications by providing a modular and flexible approach. By understanding core concepts such as components, chains, prompt templates, output parsers, indexes, retrievers, chat message history, and proxies, you can create a custom solution that suits your specific needs. LangChain's adaptability and ease of use make it a valuable tool for developers, allowing them to unleash the full potential of language models and create intelligent, context-aware applications across a wide range of use cases.

references

https://python.langchain.com/docs/get_started/introduction.html
https://blog.csdn.net/crystal_csdn8/article/details/131753160?ops_request_misc=%257B%2522request%255Fid%2522%253A%2522169163537516800184147830%2522%252C%2522scm%2522%253A%252220140713.130102334.pc%255Fall.%2522%257D&request_id=169163537516800184147830&biz_id=0&utm_medium=distribute.pc_search_result.none-task-blog-2~all~first_rank_ecpm_v1~rank_v31_ecpm-22-131753160-null-null.142^v92^insert_down1&utm_term=langchain&spm=1018.2226.3001.4187
https://blog.csdn.net/qq_43692950/article/details/131359743?ops_request_misc=&request_id=&biz_id=102&utm_term=langchain&utm_medium=distribute.pc_search_result.none-task-blog-2~all~sobaiduweb~default-0-131359743.nonecase&spm=1018.2226.3001.4187
https://www.bilibili.com/video/BV1vX4y127v4/?spm_id_from=333.788&vd_source=288508e12cf774ee9ad65190776756be
https://www.bilibili.com/video/BV1GL411e7K4/?spm_id_from=333.337.search-card.all.click&vd_source=288508e12cf774ee9ad65190776756be

Vector database related:

https://www.bilibili.com/video/BV11a4y1c7SW/?spm_id_from=333.337.search-card.all.click&vd_source=288508e12cf774ee9ad65190776756be
https://www.bilibili.com/video/BV1BM4y177Dk/?spm_id_from=333.788.recommend_more_video.-1&vd_source=288508e12cf774ee9ad65190776756be

Click to follow and learn about Huawei Cloud’s new technologies as soon as possible ~