Main module of langchain (1): model input and output

langchain

1. Concept

What is LangChain?

Origin: LangChain came about when Harrison was talking to some people in the field who were building complex LLM applications, and he was developing methods

I saw some parts that can be abstracted. An application may need to prompt LLM multiple times and parse its output, thus requiring a lot of copy-pasting to be written.

LangChain makes this development process easier. Once launched, it was widely adopted in the community, with not only many users but also many contributors participating.

Work with open source.

There is also the problem of the large model itself, which cannot perceive real-time data and cannot interact with the current world.

LangChain is a framework for developing large language models.

Main features:

\1. Data awareness: Ability to connect language models with other data sources.

\2. Agency: allows the language model to interact with its environment. You can do various things by writing tools, and write and update data.

Main values:

1. It components the functions needed to develop LLM and provides many tools for easy use.

2. There are some ready-made chains that can complete specific functions, which can also be understood as improving the ease of use of the tool.

2. Main modules

LangChain provides standard, extensible interfaces and external integrations for the following modules, listed from least to most complex:

Model I/O

Interface with language models

Data connection

Interface with application-specific data

Chain assembly (Chains)

Construct call sequence

Agents

Let chain assembly choose which tools to use based on high-level instructions

Memory

Persist application state across multiple runs of chained assembly

Callbacks

Record and stream the intermediate steps of any chain assembly

3. Model input/output (Model I/O)

Model I/O = (Model + Prompt + Output Parsers)

• Model : various LLM, ChatGPT, ChatGLM, Claude, Tongyi Qianwen, vicuna

• Prompts : model input template, which can dynamically replace content, as well as some other tool templates

• Output parsers : extract information from model outputs

hint

prompt template

prompt template

A prompt template refers to a reproducible way of generating prompts. It contains a text string ("template") that receives a set of parameters from the end user and generates a prompt.

Prompt templates can contain:

• Provide instructions to the language model,
• a set of few-shot examples to help language models generate better responses,
• Ask questions to the language model.

from langchain import PromptTemplate


template = """\
你是一个新公司的命名咨询顾问.
为制作 {product} 的公司起一个好的名字?
"""

prompt = PromptTemplate.from_template(template)
prompt.format(product="五颜六色的袜子")

'你是一个新公司的命名咨询顾问.
为制作 五颜六色的袜子 的公司起一个好的名字?'

Example selector

Example selectors

If you have some examples, you may need to use some examples and add them to the prompt.

from langchain.prompts import PromptTemplate
from langchain.prompts import FewShotPromptTemplate
from langchain.prompts.example_selector import LengthBasedExampleSelector

# 这里有很多关于创建反义词的示例。
examples = [
    {
    
    "input": "happy", "output": "sad"},
    {
    
    "input": "tall", "output": "short"},
    {
    
    "input": "energetic", "output": "lethargic"},
    {
    
    "input": "sunny", "output": "gloomy"},
    {
    
    "input": "windy", "output": "calm"},
]

example_prompt = PromptTemplate(
    input_variables=["input", "output"],
    template="输入: {input}\n输出: {output}",
)
example_selector = LengthBasedExampleSelector(
    # 这些是它可以选择的示例。
    examples=examples,
    # 这是用来格式化示例的PromptTemplate。
    example_prompt=example_prompt,
    # 这是格式化的示例应该的最大长度。
    # 长度是通过下面的get_text_length函数来测量的。
    max_length=25,
    # 这是用来获取字符串长度的函数，用于确定要包含哪些示例。
    # 它被注释掉了，因为如果没有指定，它会作为默认值提供。
    # get_text_length: Callable[[str], int] = lambda x: len(re.split("\n| ", x))
)
dynamic_prompt = FewShotPromptTemplate(
    # 我们提供一个ExampleSelector，而不是示例。
    example_selector=example_selector,
    example_prompt=example_prompt,
    prefix="给出每个输入的反义词",
    suffix="输入: {adjective}\n输出:",
    input_variables=["adjective"],
)
print(dynamic_prompt.format(adjective="big"))
# 由于max_length设置了25.所以只有一个示例.
long_string = "big and huge and massive and large and gigantic and tall and much much much much much bigger than everything else"
print(dynamic_prompt.format(adjective=long_string))

给出每个输入的反义词

输入: happy
输出: sad

输入: tall
输出: short

输入: energetic
输出: lethargic

输入: sunny
输出: gloomy

输入: windy
输出: calm

输入: big
输出:
给出每个输入的反义词

输入: happy
输出: sad

输入: big and huge and massive and large and gigantic and tall and much much much much much bigger than everything else
输出:

Model

LLMs

A model whose input is a text string and whose output is a text string

from langchain.llms import OpenAI

llm = OpenAI()
# 输入为字符串，输出也为字符串
output = llm("给我讲个笑话")
print(output)

两个病人在医院病房里，一个说：“你知道为什么医院的灯都是绿色的吗？”另一个病人答道：“不知道，为什么？”第一个病人说：“因为绿色是医生的最爱！”

ChatModels

Supported by a language model, the input is a list of chat messages and the output is a model of chat messages

from langchain.chat_models import ChatOpenAI
from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage
)

chat = ChatOpenAI()
chat([HumanMessage(content="把下面的英文翻译为中文: I love programming.")])

AIMessage(content='我喜欢编程。', additional_kwargs={}, example=False)

output interpreter

The language model outputs text. But many times you may want more structured information than just text. This is what the output parser does.

Output parsers are classes that help structure language model responses. An output parser must implement two main methods:

• "get formatting directive": A method that returns a string containing how the language model output should be formatted.
• "parse": A method that takes a string (assumed to be a response from a language model) and parses it into some structure.

Then add an optional method:

• "Parse with hints": A method that takes a string (assumed to be the response from the language model) and a hint (assumed to be the prompt that generated this response) and parses them into some structure. Hints are usually provided in situations where information from the hint is needed to retry or fix the output.

Example: list parser comma_separated

This output parser can be used when you want to return a comma-separated list of items.

1. Load local model

from gpt_server import ChatGLMService

llm_model_path = "/mnt/code/LLM_Service/model/chatglm2-6b-32k/"
chatLLM = ChatGLMService()
chatLLM.load_model(model_name_or_path=llm_model_path)

2. Building a list interpreter

from langchain.output_parsers import CommaSeparatedListOutputParser
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate

output_parser = CommaSeparatedListOutputParser()
format_instructions = output_parser.get_format_instructions()
prompt = PromptTemplate(
    template="List five {subject}.\n{format_instructions}",
    input_variables=["subject"],
    partial_variables={
    
    "format_instructions": format_instructions}
)
_input = prompt.format(subject="ice cream flavors")
output = chatLLM(_input)
print(output)

There are many flavors of ice cream. Here are five common flavors: 1. Chocolate 2. Strawberry 3. Vanilla 4. Mango 5. Mixed chocolate, strawberry and vanilla

3. parse

output_parser.parse(output)

['There are many flavors of ice cream, here are five common flavors:\n\n1. Chocolate flavor\n2. Strawberry flavor\n3. Vanilla flavor\n4. Mango flavor\n5. Chocolate, strawberry and vanilla mixed flavor']