Chapter 7 ChatGPT Prompting Development Practice
Application of prompting in LangChain framework
This section tells you about Prompt Engineering, which is mainly based on industrial-level source code and specific projects. Whether you are an engineer or a person without a technical background, everyone has heard of prompt words or used them frequently. In short, when we use OpenAI, we use its API or ChatGPT to communicate with it using prompt words. As shown in Figure 7-1, this is the ChatGPT interface. The author generally uses the GPT-4 large model.
Figure 7- 1 ChatGPT interface
Any information you enter here is called a prompt from the perspective of the model, but industrial-grade prompts are not the same as those commonly understood by everyone. Let’s first take a look at what industrial-grade prompts look like. We have prepared many specific examples for you, as shown in Figure 7-2. These examples basically come from DeepLearning.AI. From the perspective of prompts, these examples are very systematic and very valuable. For example, the chains-of-thought mentioned here is obviously a very important one. Important example. Gavin's WeChat account: NLP_Matrix_Space
Figure 7- 2 Prompt word case
Now the most recognized large-scale application development framework in open source, industry and academia is LangChain, which involves the use of a large number of prompt words. The prompt words have been reviewed and improved by many people, as well as the practice of the industry or the practice of academic research. If you download the source code of LangChain, you will find that LangChain basically has prompt words (Prompt) under different functions, as shown in Figure 7-3.
Figure 7- 3 LangChain function directory
To give a simple example, look at any directory, such as retrieval_qa, base.py can be considered as a basic API, and the core part is its prompt word (prompt), as shown in Figure 7-4.
Figure 7- 4 Prompt words for retrieval_qa
The prompt in prompt.py will have a template (template), described as "Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer." ("Use the following pieces of context to answer the question. If you don't know the answer, just say you don't know, Don't try to make up the answer.")
The code implementation of prompt.py:
# flake8: me
from langchain.prompts import PromptTemplate
prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.
{context}
Question: {question}
Helpful Answer:"""
PROMPT = PromptTemplate(
template=prompt_template, input_variables=["context", "question"]
You can look at the examples again, for example: question_answering, which provides information on prompt templates such as map_reduce_prompt, map_rerank_prompt, refine_prompts, stuff_prompt, etc.
Figure 7- 5 Prompt words for question_answering
The map_reduce_prompt.py code sets a question prompt template in its own specific format, described as: "Use the following portion of a long document to see if any of the text is relevant to answer the question. Return any relevant text verbatim" ("Use the following part of the long document to see if there is any text related to answering the question, and return any relevant text verbatim"), here There is context, there are questions and relevant content.
Code implementation of map_reduce_prompt.py:
…
question_prompt_template = """Use the following portion of a long document to see if any of the text is relevant to answer the question.
Return any relevant text verbatim.
{context}
Question: {question}
Relevant text, if any:"""
QUESTION_PROMPT = PromptTemplate(
template=question_prompt_template, input_variables=["context", "question"]
)
system_template = """Use the following portion of a long document to see if any of the text is relevant to answer the question.
Return any relevant text verbatim.
______________________
{context}"""
messages = [
SystemMessagePromptTemplate.from_template(system_template),
HumanMessagePromptTemplate.from_template("{question}"),
]
CHAT_QUESTION_PROMPT = ChatPromptTemplate.from_messages(messages)
…….
Also have a look at query_constructor, as shown in Figure 7-6.
Figure 7-6 prompt words of query_constructor
In the prompt.py code file, a lot of information about the prompt word is also given, including some descriptions of it.
The code implementation of prompt.py:
….
EXAMPLES_WITH_LIMIT = [
{
"i": 1,
"data_source": SONG_DATA_SOURCE,
"user_query": "What are songs by Taylor Swift or Katy Perry about teenage romance under 3 minutes long in the dance pop genre",
"structured_request": FULL_ANSWER,
},
{
"i": 2,
"data_source": SONG_DATA_SOURCE,
"user_query": "What are songs that were not published on Spotify",
"structured_request": NO_FILTER_ANSWER,
},
{
"i": 3,
"data_source": SONG_DATA_SOURCE,
"user_query": "What are three songs about love",
"structured_request": WITH_LIMIT_ANSWER,
},
]
EXAMPLE_PROMPT_TEMPLATE = """\
<< Example {i}. >>
Data Source:
{data_source}
User Query:
{user_query}
Structured Request:
{structured_request}
"""
EXAMPLE_PROMPT = PromptTemplate(
input_variables=["i", "data_source", "user_query", "structured_request"],
template=EXAMPLE_PROMPT_TEMPLATE,
)
….
However, what we focus on in this section is Agents. As shown in Figure 7-7, there are many different types of agents, and two are particularly important. You must pay attention to them. One is react, and the other is called mrkl. Gavin's WeChat account: NLP_Matrix_Space
Figure 7- 7 Agent directory of LangChain
If you do industrial-strength, large-scale model-based application development, you generally need a manager, which can be collectively called an agent. When we talk about the ChatGPT plug-in (Plugin), we actually think about things from the perspective of an agent, which is the idea of an agent. gpt-4-0613 and gpt-3.5-turbo-0613 are new models released by OpenAI in June 2023. They can well support function calling (Function calling). The documentation on OpenAI’s official website says a very important sentence: the large model extracts the name and parameters of the function from your information, and passes them to you to mobilize the function. server). Going back to LangChain, LangChain actually includes all the functions above, including the functions of the ChatGPT plug-in, including the function of OpenAI function calling (Function calling). The functions provided by OpenAI can be immediately mapped to LangChain. Therefore, the core problem here is to make it work effectively, rather than you simply input information, and then you get the result, of course, the result may include some steps, or some interactions and so on. However, if you do not have an agent, you can only generate text and cannot perform actions (Action), and if you develop an application, you must implement Action, which is the main role of the agent (Agent). From the perspective of open source software naming, at the beginning of LangChain, the core focus was on Chain, and later on Agent.
As shown in Figure 7-8, it is the directory structure of the mrkl agent. The author of the "MRKL Systems" paper pronounces it as ˈmɪrək(ə)l, which is the same as the pronunciation of miracle.
Figure 7- 8 The directory structure of mrkl
As shown in Figure 7-9, MRKL (Modular Reasoning, Knowledge and Language) is a new type of language understanding system that combines neural network models, external knowledge bases and expert systems, taking into account neural models and symbolic reasoning capabilities, and improves the efficiency of natural language processing.
Figure 7-9 Schematic diagram of MRKL architecture
