Building Systems with the ChatGPT API course link: https://learn.deeplearning.ai/chatgpt-building-system/
Section 1 Introduction
Two cases of LLM are introduced: Base LLM uses supervised learning for training, and its development cycle is quite long, while using Instruction tuned LLM to develop prompt-based AI can greatly shorten the development process.
Section 2 Language Models, the Chat Format and Tokens
Speaking of the tokenizor mechanism of LLM, the English seen by AI is at the sub-word level instead of a single letter, which makes it impossible for AI to complete basic tasks such as outputting a word in reverse alphabetical order. I think similar problems will be encountered in Chinese, I have tested Chinese with tiktokenizor, practice shows that some Chinese are cut in one piece, and some may be cut into several parts.
Then we talked about the Chat Format, which divides the conversation into three roles system, user, and assistant. Among them, it is best to put the setting of the assistant's role style and behavior in the system.
Section 3Classification
It talked about the practice of using GPT to classify user problems. The example is to classify user problems in a second level with a customer service role and ask GPT to return them in JSON format.
It emphasizes the role of the delimiter (delimiter), and it will be better to wrap the user questions that need to be classified with the delimiter.
import os
import openai
from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv()) # read local .env file
openai.api_key = os.environ['OPENAI_API_KEY']
def get_completion_from_messages(messages,
model="gpt-3.5-turbo",
temperature=0,
max_tokens=500):
response = openai.ChatCompletion.create(
model=model,
messages=messages,
temperature=temperature,
max_tokens=max_tokens,
)
return response.choices[0].message["content"]
delimiter = "####"
system_message = f"""
You will be provided with customer service queries. \
The customer service query will be delimited with \
{
delimiter} characters.
Classify each query into a primary category \
and a secondary category.
Provide your output in json format with the \
keys: primary and secondary.
Primary categories: Billing, Technical Support, \
Account Management, or General Inquiry.
Billing secondary categories:
Unsubscribe or upgrade
Add a payment method
Explanation for charge
Dispute a charge
Technical Support secondary categories:
General troubleshooting
Device compatibility
Software updates
Account Management secondary categories:
Password reset
Update personal information
Close account
Account security
General Inquiry secondary categories:
Product information
Pricing
Feedback
Speak to a human
"""
user_message = f"""\
I want you to delete my profile and all of my user data"""
messages = [
{
'role':'system',
'content': system_message},
{
'role':'user',
'content': f"{
delimiter}{
user_message}{
delimiter}"},
]
response = get_completion_from_messages(messages)
print(response)
user_message = f"""\
Tell me more about your flat screen tvs"""
messages = [
{
'role':'system',
'content': system_message},
{
'role':'user',
'content': f"{
delimiter}{
user_message}{
delimiter}"},
]
response = get_completion_from_messages(messages)
print(response)
Section IV Moderation
When it comes to using openai's Moderation API to review the content generated by GPT, a series of scores will be returned, mainly in four categories: hatred (threat), self-harm, pornography, and violence.
Then it talks about building a prompt to let GPT help judge whether the user's input is prompt injection, and some preprocessing, such as removing the delimiter that may exist in the user's input in advance.
Section 5 Chain of Thought Reasoning
Mentioned the use case of a thinking chain, GPT as a shopping guide divided into 5 steps to think:
- Step 1: Determine whether the user's question is asking about a specific product or a product category;
- Step 2: If the user is asking about a specific product, identify whether the product is in the given list, and then give a list;
- Step 3: List any assumptions the user has made in their message if the message contains items listed above;
- Step 4: If the user has made any assumptions, please determine whether the assumptions are true based on the product information;
- Step 5: First politely correct the customer's false assumptions (if applicable) and only politely answer the customer when mentioning or referencing the 5 offered products in the list (because these are the only 5 products sold in the store).
Then it is mentioned that LLM is required to use delimiter to divide each step in the Prompt, so when the actual output is shown to the user, the output can be divided and the last paragraph can be output.
Section 6Chaining Prompts
It is mentioned that Prompts can be chained to replace CoT for development.
Its benefits include:
- More Focused is more focused (Break down a complex task splits a complex task)
- Context Limitations context restrictions (Max tokens for input and output maximum input and output token length)
- Reduced Costs Less fees (Pay per token pay per token)
And more convenient development and debugging advantages.
Explains an example of using multiple prompts to handle complex tasks:
- Step 1: Extract related product and category names (using GPT)
- Step 2: Retrieve the detailed product information of the extracted products and categories (query from the structured data according to the return of the previous step)
- Step 3: Generate answers to user queries based on detailed product information (the detailed product information is the JSON given in the previous step)
The code for the last step is as follows:
system_message = f"""
You are a customer service assistant for a \
large electronic store. \
Respond in a friendly and helpful tone, \
with very concise answers. \
Make sure to ask the user relevant follow up questions.
"""
user_message_1 = f"""
tell me about the smartx pro phone and \
the fotosnap camera, the dslr one. \
Also tell me about your tvs"""
messages = [
{
'role':'system',
'content': system_message},
{
'role':'user',
'content': user_message_1},
{
'role':'assistant',
'content': f"""Relevant product information:\n\
{
product_information_for_user_message_1}"""},
]
final_response = get_completion_from_messages(messages)
print(final_response)
Section VII Check outputs
This section describes several ways to check the model output, such as using openai's Moderation API to review whether the content is harmful, or write a Promp to let GPT check whether the output is true based on the provided product information. The code in the case is as follows:
system_message = f"""
You are an assistant that evaluates whether \
customer service agent responses sufficiently \
answer customer questions, and also validates that \
all the facts the assistant cites from the product \
information are correct.
The product information and user and customer \
service agent messages will be delimited by \
3 backticks, i.e. ```.
Respond with a Y or N character, with no punctuation:
Y - if the output sufficiently answers the question \
AND the response correctly uses product information
N - otherwise
Output a single letter only.
"""
customer_message = f"""
tell me about the smartx pro phone and \
the fotosnap camera, the dslr one. \
Also tell me about your tvs"""
product_information = """{ "name": "SmartX ProPhone", "category": "Smartphones and Accessories", "brand": "SmartX", "model_number": "SX-PP10", "warranty": "1 year", "rating": 4.6, "features": [ "6.1-inch display", "128GB storage", "12MP dual camera", "5G" ], "description": "A powerful smartphone with advanced camera features.", "price": 899.99 } { "name": "FotoSnap DSLR Camera", "category": "Cameras and Camcorders", "brand": "FotoSnap", "model_number": "FS-DSLR200", "warranty": "1 year", "rating": 4.7, "features": [ "24.2MP sensor", "1080p video", "3-inch LCD", "Interchangeable lenses" ], "description": "Capture stunning photos and videos with this versatile DSLR camera.", "price": 599.99 } { "name": "CineView 4K TV", "category": "Televisions and Home Theater Systems", "brand": "CineView", "model_number": "CV-4K55", "warranty": "2 years", "rating": 4.8, "features": [ "55-inch display", "4K resolution", "HDR", "Smart TV" ], "description": "A stunning 4K TV with vibrant colors and smart features.", "price": 599.99 } { "name": "SoundMax Home Theater", "category": "Televisions and Home Theater Systems", "brand": "SoundMax", "model_number": "SM-HT100", "warranty": "1 year", "rating": 4.4, "features": [ "5.1 channel", "1000W output", "Wireless subwoofer", "Bluetooth" ], "description": "A powerful home theater system for an immersive audio experience.", "price": 399.99 } { "name": "CineView 8K TV", "category": "Televisions and Home Theater Systems", "brand": "CineView", "model_number": "CV-8K65", "warranty": "2 years", "rating": 4.9, "features": [ "65-inch display", "8K resolution", "HDR", "Smart TV" ], "description": "Experience the future of television with this stunning 8K TV.", "price": 2999.99 } { "name": "SoundMax Soundbar", "category": "Televisions and Home Theater Systems", "brand": "SoundMax", "model_number": "SM-SB50", "warranty": "1 year", "rating": 4.3, "features": [ "2.1 channel", "300W output", "Wireless subwoofer", "Bluetooth" ], "description": "Upgrade your TV's audio with this sleek and powerful soundbar.", "price": 199.99 } { "name": "CineView OLED TV", "category": "Televisions and Home Theater Systems", "brand": "CineView", "model_number": "CV-OLED55", "warranty": "2 years", "rating": 4.7, "features": [ "55-inch display", "4K resolution", "HDR", "Smart TV" ], "description": "Experience true blacks and vibrant colors with this OLED TV.", "price": 1499.99 }"""
q_a_pair = f"""
Customer message: ```{
customer_message}```
Product information: ```{
product_information}```
Agent response: ```{
final_response_to_customer}```
Does the response use the retrieved information correctly?
Does the response sufficiently answer the question
Output Y or N
"""
messages = [
{
'role': 'system', 'content': system_message},
{
'role': 'user', 'content': q_a_pair}
]
response = get_completion_from_messages(messages, max_tokens=1)
print(response)
another_response = "life is like a box of chocolates"
q_a_pair = f"""
Customer message: ```{
customer_message}```
Product information: ```{
product_information}```
Agent response: ```{
another_response}```
Does the response use the retrieved information correctly?
Does the response sufficiently answer the question?
Output Y or N
"""
messages = [
{
'role': 'system', 'content': system_message},
{
'role': 'user', 'content': q_a_pair}
]
response = get_completion_from_messages(messages)
print(response)
Section 8 Build an End-to-End System
Shows a complete example of a system that chains prompts to process user requests
import os
import openai
import sys
sys.path.append('../..')
import utils
import panel as pn # GUI
pn.extension()
from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv()) # read local .env file
openai.api_key = os.environ['OPENAI_API_KEY']
def get_completion_from_messages(messages, model="gpt-3.5-turbo", temperature=0, max_tokens=500):
response = openai.ChatCompletion.create(
model=model,
messages=messages,
temperature=temperature,
max_tokens=max_tokens,
)
return response.choices[0].message["content"]
def process_user_message(user_input, all_messages, debug=True):
delimiter = "```"
# Step 1: Check input to see if it flags the Moderation API or is a prompt injection
response = openai.Moderation.create(input=user_input)
moderation_output = response["results"][0]
if moderation_output["flagged"]:
print("Step 1: Input flagged by Moderation API.")
return "Sorry, we cannot process this request."
if debug: print("Step 1: Input passed moderation check.")
category_and_product_response = utils.find_category_and_product_only(user_input, utils.get_products_and_category())
#print(print(category_and_product_response)
# Step 2: Extract the list of products
category_and_product_list = utils.read_string_to_list(category_and_product_response)
#print(category_and_product_list)
if debug: print("Step 2: Extracted list of products.")
# Step 3: If products are found, look them up
product_information = utils.generate_output_string(category_and_product_list)
if debug: print("Step 3: Looked up product information.")
# Step 4: Answer the user question
system_message = f"""
You are a customer service assistant for a large electronic store. \
Respond in a friendly and helpful tone, with concise answers. \
Make sure to ask the user relevant follow-up questions.
"""
messages = [
{
'role': 'system', 'content': system_message},
{
'role': 'user', 'content': f"{
delimiter}{
user_input}{
delimiter}"},
{
'role': 'assistant', 'content': f"Relevant product information:\n{
product_information}"}
]
final_response = get_completion_from_messages(all_messages + messages)
if debug:print("Step 4: Generated response to user question.")
all_messages = all_messages + messages[1:]
# Step 5: Put the answer through the Moderation API
response = openai.Moderation.create(input=final_response)
moderation_output = response["results"][0]
if moderation_output["flagged"]:
if debug: print("Step 5: Response flagged by Moderation API.")
return "Sorry, we cannot provide this information."
if debug: print("Step 5: Response passed moderation check.")
# Step 6: Ask the model if the response answers the initial user query well
user_message = f"""
Customer message: {
delimiter}{
user_input}{
delimiter}
Agent response: {
delimiter}{
final_response}{
delimiter}
Does the response sufficiently answer the question?
"""
messages = [
{
'role': 'system', 'content': system_message},
{
'role': 'user', 'content': user_message}
]
evaluation_response = get_completion_from_messages(messages)
if debug: print("Step 6: Model evaluated the response.")
# Step 7: If yes, use this answer; if not, say that you will connect the user to a human
if "Y" in evaluation_response: # Using "in" instead of "==" to be safer for model output variation (e.g., "Y." or "Yes")
if debug: print("Step 7: Model approved the response.")
return final_response, all_messages
else:
if debug: print("Step 7: Model disapproved the response.")
neg_str = "I'm unable to provide the information you're looking for. I'll connect you with a human representative for further assistance."
return neg_str, all_messages
user_input = "tell me about the smartx pro phone and the fotosnap camera, the dslr one. Also what tell me about your tvs"
response,_ = process_user_message(user_input,[])
print(response)
def collect_messages(debug=False):
user_input = inp.value_input
if debug: print(f"User Input = {
user_input}")
if user_input == "":
return
inp.value = ''
global context
#response, context = process_user_message(user_input, context, utils.get_products_and_category(),debug=True)
response, context = process_user_message(user_input, context, debug=False)
context.append({
'role':'assistant', 'content':f"{
response}"})
panels.append(
pn.Row('User:', pn.pane.Markdown(user_input, width=600)))
panels.append(
pn.Row('Assistant:', pn.pane.Markdown(response, width=600, style={
'background-color': '#F6F6F6'})))
return pn.Column(*panels)
remaining content
The remaining part is the method of evaluating the output of LLM, and the summary.