The full text has a total of more than 10,000 words, and the expected reading time is about 40~60 minutes | Full of dry goods (with code), it is recommended to collect!
The goal of this article: to automatically build a function that meets the requirements through the LtM prompt process, and gradually and completely test the function of the code_generate function through experiments.
1. Introduction
This article is the second part of building a highly automated AI project development process from 0 to 1, which has been implemented in Large Model Development (15): Building a highly automated AI project development process from 0 to 1 (Part 1) The optimization of the first part: the complete execution guides the Chat model to create external function codes, code management and the entire process of testing. This article will propose a "automatic function writing strategy" solution to further deepen the role of large models in the actual development process, that is, try to sort out the core links in the current development process, and try to use the Chat model to complete the core links. Work.
2. Use the Chat model to automatically realize the writing of demand functions
2.1 Complete the automatic writing of the demand function with the help of Few-shot
In the previous process, the function of allowing the Chat model to write its own code to complete the requirement has been realized around a specific requirement . Therefore, the last key is still left before the intelligent mail sending and receiving system realizes functional self-growth. The first step is to translate and organize the real-time needs of users, and form a set of automatic code flow and prompt methods to guide the model to create corresponding external functions .
This step is very important. For example, during the current conversation, the user puts forward a new requirement: check whether there is an unread email of an important person in the mailbox, and interpret the content of the email. None of the previously defined external functions can meet the current needs. Let’s review the four previously defined functions:
functions_list = [get_recent_emails, send_email, get_latest_email, get_email_count]
functions = auto_functions(functions_list)
The description of each external function JSON Schema format is as follows:
If you use these external functions at this time, to answer new requirements: check whether there is an unread email of an important person in the mailbox, and interpret the content of the email
messages = [{
"role": "user", "content": '请帮我查询一下我的Gmail邮箱中是否有来自算法小陈的未读邮件，如果有的话请帮我解读下邮件内容。'}]
response = openai.ChatCompletion.create(
model="gpt-4-0613",
messages=messages,
functions=functions,
function_call="auto",
)
The output at this point is this:
For this question and answer, the model found the get_recent_emails function, which seems to be correct, but when using response.choices[0].message['content'] to view the returned content, you will find that the returned content data is empty, which is very It is easy to find that from the perspective of function implementation, this function is similar to querying recent email information , but there are still differences in requirements. In order to avoid such external functions that confuse the large model, it is necessary to further base on new requirements. Create a new external function.
So a better solution is to let the large model (LLMs) automatically complete new functions by prompting the project . The above process looks like this:
The prompt word process is actually a command translation process, which is similar to the large model development (9): Based on the Few-Shot-LtM prompt project to reproduce the command translation task under the SCAN dataset. The only difference is that for the SCAN command Behind translation is a set of grammatical rules that are completely different from natural language. The current command translation task is relatively simple, and the input and output can be understood based on the semantics of natural language. The grammatical rules behind it are relatively clear.
So from the perspective of instruction translation, can we empower large models through the Few-shot hint method? So the following series of tests were done:
- Step 1: Build prompt words
Directly use this large model development (fifteen): from 0 to 1 to build a highly automated AI project development process (above) in the two handwritten examples of creating external functions, as follows:
get_email_input = "请帮我查下邮箱里最后一封邮件内容。"
get_email_out = "请帮我编写一个python函数，用于查看我的Gmail邮箱中最后一封邮件信息，函数要求如下：\
1.函数参数userId，userId是字符串参数，默认情况下取值为'me'，表示查看我的邮件；\
2.函数返回结果是一个包含最后一封邮件信息的对象，返回结果本身必须是一个json格式对象；\
3.请将全部功能封装在一个函数内；\
4.请在函数编写过程中，在函数内部加入中文编写的详细的函数说明文档，用于说明函数功能、函数参数情况以及函数返回结果等信息；"
email_counts_input = "请帮我查下邮箱里现在总共有多少封邮件。"
email_counts_out = "请帮我编写一个python函数，用于查看我的Gmail邮箱中总共有多少封邮件，函数要求如下：\
1.函数参数userId，userId是字符串参数，默认情况下取值为'me'，表示查看我的邮件；\
2.函数返回结果是当前邮件总数，返回结果本身必须是一个json格式对象；\
3.请将全部功能封装在一个函数内；\
4.请在函数编写过程中，在函数内部加入中文编写的详细的函数说明文档，用于说明函数功能、函数参数情况以及函数返回结果等信息；"
- Step 2: Splicing into Few-shot input form
user_content = "请查下我的邮箱里是否有来自无敌小怪兽的未读邮件，并解读最近一封未读邮件的内容"
messages_fewShot_stage1 = [{
"role": "system", "content": "请按照格式，编写一段函数功能说明。"},
{
"role": "user", "name":"example1_user", "content": get_email_input},
{
"role": "assistant", "name":"example1_assistant", "content": get_email_out},
{
"role": "user", "name":"example2_user", "content": email_counts_input},
{
"role": "assistant", "name":"example2_assistant", "content": email_counts_out},
{
"role": "user", "name":"example_user", "content": user_content}]
messages_fewShot_stage1
Look at the output:
- Step 3: Call the GPT 4.0 API
response = openai.ChatCompletion.create(
model="gpt-4-0613",
messages=messages_fewShot_stage1
)
Take a look at the output:
From the results returned in this paragraph, it can be seen that the model can learn the instruction translation specification of the Few-shot prompt example better, let's test it further.
- Step 4: Functional testing
Read the previously stored get_latest_email() external function:
with open('./functions/tested functions/%s_module.py' % 'get_latest_email', encoding='utf-8') as f:
assistant_example_content = f.read()
The description is this:
In Step 3, the function description generated by GPT 4.0 API is as follows:
- Step 5: Prompt words for building tests
messages_fewShot_stage2 = [{
"role": "system", "content": system_content},
{
"role": "user", "name":"example_user", "content": user_example_content},
{
"role": "assistant", "name":"example_assistant", "content": assistant_example_content},
{
"role": "user", "name":"example_user", "content": user_content}]
messages_fewShot_stage2
The output is this:
- Step 6: Call the GPT 4.0 API again
response = openai.ChatCompletion.create(
model="gpt-4-0613",
messages=messages_fewShot_stage2
)
The resulting output is this:
- Step 7: Extract to native code file
def extract_function_code(s, detail=0, tested=False):
"""
函数提取函数，同时执行函数内容，可以选择打印函数信息，并选择代码保存的地址
"""
def extract_code(s):
"""
如果输入的字符串s是一个包含Python代码的Markdown格式字符串，提取出代码部分。
否则，返回原字符串。
参数:
s: 输入的字符串。
返回:
提取出的代码部分，或原字符串。
"""
# 判断字符串是否是Markdown格式
if '```python' in s or 'Python' in s or'PYTHON' in s:
# 找到代码块的开始和结束位置
code_start = s.find('def')
code_end = s.find('```\n', code_start)
# 提取代码部分
code = s[code_start:code_end]
else:
# 如果字符串不是Markdown格式，返回原字符串
code = s
return code
# 提取代码字符串
code = extract_code(s)
# 提取函数名称
match = re.search(r'def (\w+)', code)
function_name = match.group(1)
# 将函数写入本地
if tested == False:
with open('./functions/untested functions/%s_module.py' % function_name, 'w', encoding='utf-8') as f:
f.write(code)
else:
with open('./functions/tested functions/%s_module.py' % function_name, 'w', encoding='utf-8') as f:
f.write(code)
# 执行该函数
try:
exec(code, globals())
except Exception as e:
print("An error occurred while executing the code:")
print(e)
# 打印函数名称
if detail == 0:
print("The function name is:%s" % function_name)
if detail == 1:
with open('./functions/untested functions/%s_module.py' % function_name, encoding='utf-8') as f:
content = f.read()
print(content)
Take a look at the code description:
From the test results, there is no problem with this function. Basically, Few-shot can be used to complete the automatic creation of the function.
2.2 Use the LtM prompt process to complete more stable function writing
The content generated by large language models (LLMs) is not fixed. Although Few-shot can solve the need for automatic function writing to a certain extent, Few-shot is not particularly stable in the actual testing process. An effective and The easy-to-think solution is to use a stronger prompt method: such as the LtM prompt method .
In the article Large Model Development (VIII): Advanced Prompt Project Based on Chain of Thought (CoT), a large number of experiments have proved that LtM is the most effective prompt method so far , through multi-stage prompt strategy and combined with current instruction translation tasks. , a more promising prompting method that can guide the model to successfully complete the instruction translation process is: first guide the model to disassemble the "variables" in the current demand as parameters of subsequent functions, and then guide the model to complete the translation based on the determined variables Work.
As far as the entire instruction translation task is concerned, the most difficult part is the understanding process of function parameters . Once the model can accurately translate which parameters are required by the external function, other parts of the translation task will naturally be solved. Therefore, after optimizing the prompt process in the first stage, a better overall prompt process is as follows:
It is a measure to improve the function reuse rate to make the model reasonably analyze the parameter settings that meet the current needs. For example, if you want to query the unread emails from the invincible little monster, if the model can regard the unread emails of "who" as a variable, and set the object to read the emails as a parameter, then the subsequent function can be better improved reuse rate.
If you still don't understand the LtM prompt process, it is strongly recommended to read this article before continuing this article: Large Model Development (8): Advanced Prompt Project Based on Chain of Thought (CoT)
Take a look at the flowchart of LtM:
There are two stages for the LtM prompt. The prompt of the first stage is named Command decomposition, abbreviated as CD, and the prompt of the second stage is named Command mapping, abbreviated as CM.
- Step 1: Example of using Few-shot to construct split sub-problems
Taking the two functions of "view the content of the last email" and "send email" as an example of the two stages of the LtM prompt, it uses the Few-shot prompt to first disassemble the sub-problem of the example, as follows:
system_content1 = "为了更好编写满足用户需求的python函数，我们需要先识别用户需求中的变量，以作为python函数的参数。需要注意的是，当前编写的函数中涉及到的邮件收发查阅等功能，都是通过调用Gmail API来完成。"
input1 = "请帮我查下Gmail邮箱里最后一封邮件内容。"
pi1 = "当前需求中可以作为函数参数的是：1.查看谁的邮箱。"
input2 = "请帮我给小陈发送一封Gmail邮件，请他明天早上9点半来我办公室开会，商量我的100亿应该怎么花。"
pi2 = "当前需求中可以作为函数参数的是：1.发送邮件的对象；2.发送邮件的主题；3.邮件具体内容"
input3 = "请查下我的邮箱里是否有来自无敌小怪兽的未读邮件，并解读最近一封未读邮件的内容。"
- Step 2: Create Prompt
messages_CD = [{
"role": "system", "content": system_content1},
{
"role": "user", "name":"example1_user", "content": input1},
{
"role": "assistant", "name":"example1_assistant", "content": pi1},
{
"role": "user", "name":"example2_user", "content": input2},
{
"role": "assistant", "name":"example2_assistant", "content": pi2},
{
"role": "user", "name":"example_user", "content": input3}]
Look at the results:
- Step 3: Call the GPT 4.0 API
response = openai.ChatCompletion.create(
model="gpt-4-0613",
messages=messages_CD
)
Take a look at the disassembly results under Few-shot:
From the results, the model can clearly judge the variables that can be used as functions in the current demand by dismantling the problem in few-shot and giving prompts in the first stage.
- Step 4: Create LtM Command mapping stage prompt
system_content2 = "我现在已完成Gmail API授权，授权文件为本地文件token.json。所编写的函数要求参数类型都必须是字符串类型。"
get_email_input = "请帮我查下邮箱里最后一封邮件内容。" + pi1
get_email_out = "请帮我编写一个python函数，用于查看我的Gmail邮箱中最后一封邮件信息，函数要求如下：\
1.函数参数userId，userId是字符串参数，默认情况下取值为'me'，表示查看我的邮件；\
2.函数返回结果是一个包含最后一封邮件信息的对象，返回结果本身必须是一个json格式对象；\
3.请将全部功能封装在一个函数内；\
4.请在函数编写过程中，在函数内部加入中文编写的详细的函数说明文档，用于说明函数功能、函数参数情况以及函数返回结果等信息；"
send_email_input = "请帮我给小陈发送一封Gmail邮件，请他明天早上9点半来我办公室开会，商量我的100亿应该怎么花。" + pi2
send_email_out = "请帮我编写一个python函数，用于给小陈发送邮件，请他明天早上9点半来我办公室开会，商量我的100亿应该怎么花。，函数要求如下：\
1.函数参数to、subject和message_text，三个参数都是字符串类型，其中to表示发送邮件对象，subject表示邮件主题，message_text表示邮件具体内容；\
2.函数返回结果是当前邮件发送状态，返回结果本身必须是一个json格式对象；\
3.请将全部功能封装在一个函数内；\
4.请在函数编写过程中，在函数内部加入中文编写的详细的函数说明文档，用于说明函数功能、函数参数情况以及函数返回结果等信息；"
user_content = input3 + pi3
- Step 4: Create the prompt for the second stage
messages_CM = [{
"role": "system", "content": system_content2},
{
"role": "user", "name":"example1_user", "content": get_email_input},
{
"role": "assistant", "name":"example1_assistant", "content": get_email_out},
{
"role": "user", "name":"example2_user", "content": send_email_input},
{
"role": "assistant", "name":"example2_assistant", "content": send_email_out},
{
"role": "user", "name":"example_user", "content": user_content}]
Look at the output:
- **Step 5: Call GPT 4.0 API again**
response = openai.ChatCompletion.create(
model="gpt-4-0613",
messages=messages_CM
)
Take a look at the results:
It can be found that under reasonable function parameter settings, this function description looks more reasonable. So far, the prompt process of LtM-based function description creation is completed, which is the following figure:
For the second stage of the overall prompt, it is to guide the large model to create the corresponding function based on function_description.
- Step 6: Create functions based on Prompt
In the function creation phase, the prompt is also used in the Few-shot method. The prompt examples are still the two functions get_email_out and send_email_out. The prompt process is as follows:
For get_email_out, its function description and function definition are as follows:
For send_email_out, its function description and function definition are as follows:
- Step 7: Create Prompt
system_content3 = "我现在已完成Gmail API授权，授权文件为本地文件token.json。函数参数必须是字符串类型对象，函数返回结果必须是json表示的字符串对象。"
messages_stage2 = [{
"role": "system", "content": system_content3},
{
"role": "user", "name":"example1_user", "content": func1_description},
{
"role": "assistant", "name":"example1_assistant", "content": func1_str},
{
"role": "user", "name":"example2_user", "content": func2_description},
{
"role": "assistant", "name":"example2_assistant", "content": func2_str},
{
"role": "user","content": function_description}]
messages_stage2
The generated Prompt looks like this:
- Step 8: Call the GPT 4.0 API
response = openai.ChatCompletion.create(
model="gpt-4",
messages=messages_stage2
)
Take a look at the output:
- Step 9: Parse the string
The function defined by the model is get_unread_email, which can meet the current functional requirements after manual review.
- Step 10: Bring this function into the Chat model for functional verification
First look at the analysis of the function:
functions_list = [get_unread_email_from]
functions = auto_functions(functions_list)
functions
Send a test email first, as shown in the figure:
Then test the AI application situation through LLMs + Gmail API:
messages = [{
"role": "user", "content": '请查询我的邮箱，看下里面是否有来自算法小陈的未读邮件。'}]
response = openai.ChatCompletion.create(
model="gpt-4",
messages=messages,
functions=functions,
function_call="auto",
)
Look at the output:
So far, it can be verified that the model can normally meet user needs and create corresponding parameters that meet the requirements .
2.3 Multi-round dialogue test
After completing the above process, you can directly bring this function into the multi-round dialogue function to test the effect, and bring in other external functions at the same time for more complex functional tests.
- Step 1: Prepare test email
First send an email to your Gmail mailbox in advance, the content of the email is as follows:
- Step 2: Import the multi-round dialogue function:
def chat_with_model(functions_list=None,
prompt="你好呀",
model="gpt-4-0613",
system_message=[{
"role": "system", "content": "你是以为乐于助人的助手。"}]):
messages = system_message
messages.append({
"role": "user", "content": prompt})
while True:
answer = run_conversation(messages=messages,
functions_list=functions_list,
model=model)
print(f"模型回答: {
answer}")
# 询问用户是否还有其他问题
user_input = input("您还有其他问题吗？(输入退出以结束对话): ")
if user_input == "退出":
break
# 记录用户回答
messages.append({
"role": "user", "content": user_input})
- Step 3: Update the list of external functions
functions_list = [get_recent_emails, send_email, get_latest_email, get_email_count, get_unread_email_from]
- Step 4: Functional testing
chat_with_model(functions_list=functions_list,
system_message=[{
"role": "system", "content": "算法小陈的邮箱地址是:[email protected]"}])
Take a look at the conversation process:
Look at the mailbox situation:
So far, a complete process of automatically writing function implementation for new requirements has been completely executed.
3. Realize the fully automatic function writing process
The fundamental purpose of the above-mentioned process of stably writing external functions based on the LtM prompt process is to explore a fully automatic development process applicable to more general situations. It is far from enough to achieve the current function encapsulation through a higher level of abstraction. There are several key intermediate links.
3.1 Prompt word management
In fact, regardless of the development tasks in any field, only by finding an effective prompt word management method for the model and intermediate links of the process can we have the ability to control the output of large languages (LLMs). **Different development projects need manual work in the early stage Exploration of experience to obtain a prompt process suitable for the current development process (that can obtain stable code results). There are a total of three prompting stages behind each function. An effective solution is to create a JSON object around each tested function to save the prompting process of each stage . The process is as follows:
- Step 1: get_latest_email complete prompt sample creation process
The prompt words and results of the three stages of the function manually written in the previous process are as follows:
# 第一阶段LtM_CD阶段提示词及输出结果
get_latest_email_CD_input = "请帮我查下Gmail邮箱里最后一封邮件内容。"
get_latest_email_pi = "当前需求中可以作为函数参数的是：1.查看谁的邮箱。"
get_latest_email_messages_CD = [
{
"role": "user", "content": get_latest_email_CD_input},
{
"role": "assistant", "content": get_latest_email_pi}
]
# 第一阶段LtM_CM阶段提示词及输出结果
get_latest_email_CM_input = get_latest_email_CD_input + get_latest_email_pi
get_latest_email_description = "请帮我编写一个python函数，用于查看我的Gmail邮箱中最后一封邮件信息，函数要求如下：\
1.函数参数userId，userId是字符串参数，默认情况下取值为'me'，表示查看我的邮件；\
2.函数返回结果是一个包含最后一封邮件信息的对象，返回结果本身必须是一个json格式对象；\
3.请将全部功能封装在一个函数内；\
4.请在函数编写过程中，在函数内部加入中文编写的详细的函数说明文档，用于说明函数功能、函数参数情况以及函数返回结果等信息；"
get_latest_email_messages_CM = [
{
"role": "user", "content": get_latest_email_CM_input},
{
"role": "assistant", "content":get_latest_email_description}
]
# 第二阶段提示词及输出结果
with open('./functions/tested functions/%s_module.py' % 'get_latest_email', encoding='utf-8') as f:
get_latest_email_function = f.read()
get_latest_email_messages = [
{
"role": "user", "content": get_latest_email_description},
{
"role": "assistant", "content":get_latest_email_function}
]
There are two points to note in this process:
- In the process of saving prompt words, not only the prompt words of prompts need to be saved, but also the prompt results need to be saved, so as to serve as prompt examples for other prompt processes
- The actual development process mostly uses the Chat model, so it is recommended to save it in the form of messages-like parameters
After saving the prompt content of each stage in the form of message, save the three prompt examples in the following format, that is, save all the prompt examples of the three stages of a function object as a dictionary, and each key value in the dictionary A pair of prompts representing a stage. At the same time, the complete prompt example of each function is named after function_name_prompt. The specific get_latest_email_prompt object creation process is as follows:
# 第一阶段LtM_CD阶段提示词及输出结果
get_latest_email_CD_input = "请帮我查下Gmail邮箱里最后一封邮件内容。"
get_latest_email_pi = "当前需求中可以作为函数参数的是：1.查看谁的邮箱。"
get_latest_email_messages_CD = [
{
"role": "user", "content": get_latest_email_CD_input},
{
"role": "assistant", "content": get_latest_email_pi}
]
# 第一阶段LtM_CM阶段提示词及输出结果
get_latest_email_CM_input = get_latest_email_CD_input + get_latest_email_pi
get_latest_email_description = "请帮我编写一个python函数，用于查看我的Gmail邮箱中最后一封邮件信息，函数要求如下：\
1.函数参数userId，userId是字符串参数，默认情况下取值为'me'，表示查看我的邮件；\
2.函数返回结果是一个包含最后一封邮件信息的对象，返回结果本身必须是一个json格式对象；\
3.请将全部功能封装在一个函数内；\
4.请在函数编写过程中，在函数内部加入中文编写的详细的函数说明文档，用于说明函数功能、函数参数情况以及函数返回结果等信息；"
get_latest_email_messages_CM = [
{
"role": "user", "content": get_latest_email_CM_input},
{
"role": "assistant", "content":get_latest_email_description}
]
# 第二阶段提示词及输出结果
with open('./functions/tested functions/%s_module.py' % 'get_latest_email', encoding='utf-8') as f:
get_latest_email_function = f.read()
get_latest_email_messages = [
{
"role": "user", "content": get_latest_email_description},
{
"role": "assistant", "content":get_latest_email_function}
]
get_latest_email_prompt = {
"stage1_CD": get_latest_email_messages_CD,
"stage1_CM": get_latest_email_messages_CM,
"stage2": get_latest_email_messages
}
Take a look at the full prompt:
- Step 2: Write it locally, and save it and the py file of the corresponding function in the same folder
with open('./functions/tested functions/%s_prompt.json' % 'get_latest_email', 'w') as f:
json.dump(get_latest_email_prompt, f)
The same operation can be done to save the complete prompt example of get_recent_emails, send_email, get_email_count, get_unread_email_from.
- Step 3: Save the system prompt message
system_messages = {
"system_message_CD": [{
"role": "system", "content": "为了更好编写满足用户需求的python函数，我们需要先识别用户需求中的变量，以作为python函数的参数。需要注意的是，当前编写的函数中涉及到的邮件收发查阅等功能，都是通过调用Gmail API来完成。"}],
"system_message_CM": [{
"role": "system", "content": "我现在已完成Gmail API授权，授权文件为本地文件token.json。函数参数必须是字符串类型对象，函数返回结果必须是json表示的字符串对象。"}],
"system_message": [{
"role": "system", "content":"我现在已完成Gmail API授权，授权文件为本地文件token.json。函数参数必须是字符串类型对象，函数返回结果必须是json表示的字符串对象。"}]}
with open('./functions/tested functions/%s.json' % 'system_messages', 'w') as f:
json.dump(system_messages, f)
Look at the final save result:
3.2 Project code file management
A complete development project must have a lot of function codes and related prompt word description documents. In order to better manage the project, it is necessary to modify the code structure and file structure of the current project.
Take the project development of Gmail as an example, you can try this operation:
- Rename the original general functions folder to a specific autoGmail_project, indicating that this folder is specially used to save autoGmail related files.
- Save the function codes and prompt files with the same name in tested and untested to a folder with the same name, and the location of the original system_messages.json file remains unchanged.
It was like this before modification:
Put the module file and prompt file of the same function in a folder with the same name as the function, and the modified tested file looks like this:
3.3 Auxiliary functions for prompt word management
- Step 1: Modify the remove_to_tested function
After modifying the file path, you need to redefine a remove_to_tested function to realize the transfer process from untested to tested in the function folder. The code is as follows:
def remove_to_tested(function_name):
"""
将函数同名文件夹由untested文件夹转移至tested文件夹内。\
完成转移则说明函数通过测试，可以使用。此时需要将该函数的源码写入gptLearning.py中方便下次调用。
"""
# 将函数代码写入gptLearning.py文件中
with open('./functions/untested functions/%s/%s_module.py' % (function_name, function_name), encoding='utf-8') as f:
function_code = f.read()
with open('gptLearning.py', 'a', encoding='utf-8') as f:
f.write(function_code)
# 源文件夹路径
src_dir = './functions/untested functions/%s' % function_name
# 目标文件夹路径
dst_dir = './functions/tested functions/%s' % function_name
# 移动文件夹
shutil.move(src_dir, dst_dir)
- Step 2: Modify the extract_function_code function
extract_function_code needs to add the function of creating a function folder with the same name in the untested folder when the function extraction is completed, and needs to modify the file path for reading and writing code. The code is as follows:
def extract_function_code(s, detail=0, tested=False, g=globals()):
"""
函数提取函数，同时执行函数内容，可以选择打印函数信息，并选择代码保存的地址
"""
def extract_code(s):
"""
如果输入的字符串s是一个包含Python代码的Markdown格式字符串，提取出代码部分。
否则，返回原字符串。
参数:
s: 输入的字符串。
返回:
提取出的代码部分，或原字符串。
"""
# 判断字符串是否是Markdown格式
if '```python' in s or 'Python' in s or'PYTHON' in s:
# 找到代码块的开始和结束位置
code_start = s.find('def')
code_end = s.find('```\n', code_start)
# 提取代码部分
code = s[code_start:code_end]
else:
# 如果字符串不是Markdown格式，返回原字符串
code = s
return code
# 提取代码字符串
code = extract_code(s)
# 提取函数名称
match = re.search(r'def (\w+)', code)
function_name = match.group(1)
# 在untested文件夹内创建函数同名文件夹
directory = './functions/untested functions/%s' % function_name
if not os.path.exists(directory):
os.makedirs(directory)
# 将函数写入本地
if tested == False:
with open('./functions/untested functions/%s/%s_module.py' % (function_name, function_name), 'w', encoding='utf-8') as f:
f.write(code)
else:
# 调用remove_to_test函数将函数文件夹转移至tested文件夹内
remove_to_tested(function_name)
with open('./functions/tested functions/%s/%s_module.py' % (function_name, function_name), 'w', encoding='utf-8') as f:
f.write(code)
# 执行该函数
try:
exec(code, g)
except Exception as e:
print("An error occurred while executing the code:")
print(e)
# 打印函数名称
if detail == 0:
print("The function name is:%s" % function_name)
if detail == 1:
if tested == False:
with open('./functions/untested functions/%s/%s_module.py' % (function_name, function_name), 'r', encoding='utf-8') as f:
content = f.read()
else:
with open('./functions/tested functions/%s/%s_module.py' % (function_name, function_name), 'r', encoding='utf-8') as f:
content = f.read()
print(content)
return function_name
- Step 3: Modify the show_functions function
The show_functions function needs to be modified to display the file name in the folder, the code is as follows:
def show_functions(tested=False, if_print=False):
"""
打印tested或untested文件夹内全部函数
"""
current_directory = os.getcwd()
if tested == False:
directory = current_directory + '\\functions\\untested functions'
else:
directory = current_directory + '\\functions\\tested functions'
files_and_directories = os.listdir(directory)
# 过滤结果，只保留.py文件和非__pycache__文件夹
files_and_directories = files_and_directories = [name for name in files_and_directories if (os.path.splitext(name)[1] == '.py' or os.path.isdir(os.path.join(directory, name))) and name != "__pycache__"]
if if_print != False:
for name in files_and_directories:
print(name)
return files_and_directories
So far, the prompt word management of the tested functions has been completed, a new prompt word management method has been built, and related functions to assist in file management have been created.
4. Realize the creation method of fully automatic programming function
After the automatic writing of functional functions is realized, it is also necessary to encapsulate the complete process from user requirements to creation of external functions by LtM prompt method into a function named code_generate . The basic function of this function, the interaction relationship with other functions, and the calling relationship with the function library are as follows:
Encapsulate the above process into a high-level function, the code is as follows:
def code_generate(req, few_shot='all', model='gpt-4-0613', g=globals(), detail=0):
"""
Function calling外部函数自动创建函数，可以根据用户的需求，直接将其翻译为Chat模型可以直接调用的外部函数代码。
:param req: 必要参数，字符串类型，表示输入的用户需求；
:param few_shot: 可选参数，默认取值为字符串all，用于描述Few-shot提示示例的选取方案，当输入字符串all时，则代表提取当前外部函数库中全部测试过的函数作为Few-shot；\
而如果输入的是一个包含了多个函数名称的list，则表示使用这些函数作为Few-shot。
:param model: 可选参数，表示调用的Chat模型，默认选取gpt-4-0613；
:param g: 可选参数，表示extract_function_code函数作用域，默认为globals()，即在当前操作空间全域内生效；
:param detail: 可选参数，默认取值为0，还可以取值为1，表示extract_function_code函数打印新创建的外部函数细节；
:return：新创建的函数名称。需要注意的是，在函数创建时，该函数也会在当前操作空间被定义，后续可以直接调用；
"""
# 提取提示示例的函数名称
if few_shot == 'all':
few_shot_functions_name = show_functions(tested=True)
elif type(few_shot) == list:
few_shot_functions_name = few_shot
# few_shot_functions = [globals()[name] for name in few_shot_functions_name]
# 读取各阶段系统提示
with open('./functions/tested functions/system_messages.json', 'r') as f:
system_messages = json.load(f)
# 各阶段提示message对象
few_shot_messages_CM = []
few_shot_messages_CD = []
few_shot_messages = []
# 先保存第一条消息，也就是system message
few_shot_messages_CD += system_messages["system_message_CD"]
few_shot_messages_CM += system_messages["system_message_CM"]
few_shot_messages += system_messages["system_message"]
# 创建不同阶段提示message
for function_name in few_shot_functions_name:
with open('./functions/tested functions/%s/%s_prompt.json' % (function_name, function_name), 'r') as f:
msg = json.load(f)
few_shot_messages_CD += msg["stage1_CD"]
few_shot_messages_CM += msg["stage1_CM"]
few_shot_messages += msg['stage2']
# 读取用户需求，作为第一阶段CD环节User content
new_req_CD_input = req
few_shot_messages_CD.append({
"role": "user", "content": new_req_CD_input})
print('第一阶段CD环节提示创建完毕，正在进行CD提示...')
# 第一阶段CD环节Chat模型调用过程
response = openai.ChatCompletion.create(
model=model,
messages=few_shot_messages_CD
)
new_req_pi = response.choices[0].message['content']
print('第一阶段CD环节提示完毕')
# 第一阶段CM环节Messages创建
new_req_CM_input = new_req_CD_input + new_req_pi
few_shot_messages_CM.append({
"role": "user", "content": new_req_CM_input})
print('第一阶段CM环节提示创建完毕，正在进行第一阶段CM提示...')
# 第一阶段CM环节Chat模型调用过程
response = openai.ChatCompletion.create(
model=model,
messages=few_shot_messages_CM
)
new_req_description = response.choices[0].message['content']
print('第一阶段CM环节提示完毕')
# 第二阶段Messages创建过程
few_shot_messages.append({
"role": "user", "content": new_req_description})
print('第二阶段提示创建完毕，正在进行第二阶段提示...')
# 第二阶段Chat模型调用过程
response = openai.ChatCompletion.create(
model=model,
messages=few_shot_messages
)
new_req_function = response.choices[0].message['content']
print('第二阶段提示完毕，准备运行函数并编写提示示例')
# 提取函数并运行，创建函数名称对象，统一都写入untested文件夹内
function_name = extract_function_code(s=new_req_function, detail=detail, g=g)
print('新函数保存在./functions/untested functions/%s/%s_module.py文件中' % (function_name, function_name))
# 创建该函数提示示例
new_req_messages_CD = [
{
"role": "user", "content": new_req_CD_input},
{
"role": "assistant", "content": new_req_pi}
]
new_req_messages_CM = [
{
"role": "user", "content": new_req_CM_input},
{
"role": "assistant", "content":new_req_description}
]
with open('./functions/untested functions/%s/%s_module.py' % (function_name, function_name), encoding='utf-8') as f:
new_req_function = f.read()
new_req_messages = [
{
"role": "user", "content": new_req_description},
{
"role": "assistant", "content":new_req_function}
]
new_req_prompt = {
"stage1_CD": new_req_messages_CD,
"stage1_CM": new_req_messages_CM,
"stage2": new_req_messages
}
with open('./functions/untested functions/%s/%s_prompt.json' % (function_name, function_name), 'w') as f:
json.dump(new_req_prompt, f)
print('新函数提示示例保存在./functions/untested functions/%s/%s_prompt.json文件中' % (function_name, function_name))
print('done')
return function_name
For the above process, if you don’t understand it, you can try to verify it manually. Take get_latest_email as an example:
- Step 1: Read the function description of get_latest_email
function_name = 'get_latest_email'
with open('./functions/tested functions/%s/%s_prompt.json' % (function_name, function_name), 'r') as f:
msg = json.load(f)
msg['stage2']
Take a look at the output:
- Step 2: View the saved function file
few_shot_functions_name = show_functions(tested=True)
Look at the returned results:
- Step 3: View the system prompts at each stage
with open('./functions/tested functions/system_messages.json', 'r') as f:
system_messages = json.load(f)
# 各阶段提示message对象
few_shot_messages_CM = []
few_shot_messages_CD = []
few_shot_messages = []
# 先保存第一条消息，也就是system message
few_shot_messages_CD += system_messages["system_message_CD"]
few_shot_messages_CM += system_messages["system_message_CM"]
few_shot_messages += system_messages["system_message"]
Look at the returned results:
- Step 4: View the complete Prompt of each function
for function_name in few_shot_functions_name:
with open('./functions/tested functions/%s/%s_prompt.json' % (function_name, function_name), 'r') as f:
msg = json.load(f)
few_shot_messages_CD += msg["stage1_CD"]
few_shot_messages_CM += msg["stage1_CM"]
few_shot_messages += msg['stage2']
Look at the results:
- Step 4: Enter new requirements
new_req_CD_input = "请查下我的邮箱里是否有来自算法小陈的未读邮件，有的话请解读下这封未读邮件的内容。"
- Step 5: Prompt for splicing new requirements
few_shot_messages_CD.append({
"role": "user", "content": new_req_CD_input})
The output prompt is as follows:
- Step 6: Proceed to the CD prompt of the first stage
response = openai.ChatCompletion.create(
model="gpt-4-0613",
messages=few_shot_messages_CD
)
Look at the model output:
- Step 7: Carry out the CM prompt of the first stage
new_req_CM_input = new_req_CD_input + new_req_pi
few_shot_messages_CM.append({
"role": "user", "content": new_req_CM_input})
few_shot_messages_CM
Look at the spliced Prompt:
Input model:
response = openai.ChatCompletion.create(
model="gpt-4-0613",
messages=few_shot_messages_CM
)
Take a look at the results returned by the model:
- Step 8: Prompts for the second stage
The prompt at this stage is system input + function description and Few-shot of the specific function, and guides the large language model (LLMs) to generate specific functions according to the prompt. The code is as follows:
few_shot_messages.append({
"role": "user", "content": new_req_description})
response = openai.ChatCompletion.create(
model="gpt-4-0613",
messages=few_shot_messages
)
Look at the model output:
- Step 9: Extract the string and convert it into a function
extract_function_code(new_req_function, detail=1, g=globals())
Take a look at the generated function:
- Step 10: Test the code_generate one-click generation function
few_shot_functions = ['get_latest_email', 'send_email']
req = "请查下我的邮箱里是否有来自算法小陈的未读邮件，有的话请解读下这封未读邮件的内容。"
function_name = code_generate(req=req, few_shot=few_shot_functions)
Its creation process is as follows:
- Step 11: Manually test the generated functions
functions_list = [check_unread_emails]
messages = [{
"role": "system", "content": "算法小陈的邮箱地址是:[email protected] "},
{
"role": "user", "content": req}]
final_response = run_conversation(messages=messages, functions_list=functions_list, model="gpt-3.5-turbo-16k-0613")
Based on this set of processes, efficient and convenient creation of external functions is basically realized. You can try to add new functions based on code_generate. Examples are as follows:
few_shot_functions = ['get_latest_email', 'send_email']
req = "请查下我的邮箱里第一封邮件，并告诉我第一封邮件的收件时间和邮件内容"
function_name = code_generate(req=req, few_shot=few_shot_functions)
functions_list = [get_first_email]
messages = [{
"role": "user", "content": req}]
final_response = run_conversation(messages=messages, functions_list=functions_list, model="gpt-3.5-turbo-16k-0613")
final_response
V. Summary
This article is the second step of optimizing the highly automated AI project development process. Through the LtM prompt process, it solves the problem of automatically building functions that meet the requirements according to user needs, and fully tests the code_generate function step by step through examples.
In general, the entire process has been basically fixed and conforms to a high degree of automation, and has a certain degree of versatility. However, it is found in the experimental test that the output capability of the gpt4.0 interface is still better than that of the gpt3.5 interface. In order to be stable, however, even if it is currently the strongest gpt4.0, there will still be unstable codes when creating batch codes. This is also a problem that needs to be further thought about how to solve it.
Finally, thank you for reading this article! If you feel that you have gained something, don't forget to like, bookmark and follow me, this is the motivation for my continuous creation. If you have any questions or suggestions, you can leave a message in the comment area, I will try my best to answer and accept your feedback. If there's a particular topic you'd like to know about, please let me know and I'd be happy to write an article about it. Thank you for your support and look forward to growing up with you!