Azure Machine Learning - Tip Engineering Advanced Techniques

This guide will guide you through some advanced techniques in prompt design and prompt engineering.

Follow TechLead and share all-dimensional knowledge of AI. The author has 10+ years of Internet service architecture, AI product research and development experience, and team management experience. He holds a master's degree from Tongji University in Fudan University, a member of Fudan Robot Intelligence Laboratory, a senior architect certified by Alibaba Cloud, a project management professional, and research and development of AI products with revenue of hundreds of millions. principal

file

Azure Prompt Engineering API Description

Although the principles of prompt engineering can be generalized across many different model types, some models require specialized prompt structures. For the Azure OpenAI GPT model, the core recommends the CHAT API , and the prompt project can play a role in it:

CHAT API supports GPT-35-Turbo and GPT-4 models. These models are designed to receive input in a [chat-like specific script] format stored in an array of dictionaries.

system information

System messages are included at the beginning of the prompt and are used to provide the model with context, instructions, or other information relevant to the use case. You can use system messages to describe the assistant's personality, define what the model should and should not answer, and define the format of the model's responses.

The following example shows a sample system message and generated model response:

system information	user	Assistant
You are an AI assistant that helps people find information and respond in rhyme. If a user asks a question you don't know the answer to, say so.	I'm John Doe, what can you tell me about me?	Dear John, I'm sorry to say, I have no information about you today. I'm just an AI with knowledge, but I can't explain it without your input. So please tell me more about what you are looking for and I will do my best to give you a unique answer.

Some other examples of system messages include:

"The assistant is a large language model trained by OpenAI."
"Assistant is an intelligent chatbot designed to help users answer technical questions about the Azure OpenAI service. Only use the following context to answer questions, or say "I don't know" if you're not sure of the answer.
“The Assistant is an intelligent chatbot designed to help users answer their tax-related questions.”
"You are an assistant designed to extract entities from text. The user will paste a text string and you will respond with the entities extracted from the text as a JSON object. Here is an example of the output format:

{  
   "name": "",
   "company": "",
   "phone_number": ""
}

An important detail to understand is that even if you instruct the model in a system message to answer " I don't know " when it is unsure of the answer, this does not guarantee that the request will be honored. Well-designed system messages can increase the likelihood of producing a specific result, but may still generate incorrect responses that may contradict the intent of the instructions in the system message.

few-shot learning

A common way to adapt language models to new tasks is to use few-shot learning. In few-shot learning, a set of training examples needs to be provided in the hint to provide additional context to the model.

A sequence of messages (written in [new prompt format]) between a user and an assistant can serve as an example of few-shot learning when using the chat completion API. These examples can be used to guide models to respond in certain ways, mimic specific behaviors, and provide seed answers to common questions.

Non-chat scene

While the chat completion API is optimized for handling multi-turn conversations, it can also be used in non-chat scenarios. For example, for a sentiment analysis scenario, you can use the following tips:

system information	user	Assistant
You are an assistant designed to analyze sentiment in speech data. The user will paste a string of text and you will evaluate the speaker's response. Give a rating from 1-10 (10 being the highest). Explain the reason for this rating.	Hi, thanks for calling Contoso Pharmacy, who am I talking to today? I understand your frustration, Mary. Can you tell me what you want to achieve? OK, I'm happy to take a look at it for you. Please wait a moment. I see here that you have used generic Lipitor, right? Okay, so I see your doctor stopped prescribing these medications in 2021. Is this actually the case? Okay, let me check this for you. Because sometimes vulnerabilities appear in the system and are not reported. Please wait a moment and let me see if there is a record here. So between 2012 and 2021, your doctor prescribed these medications to you, according to the most recent doctor's office records provided to us. Okay, so the latest records show that now the doctor has switched to a different kind of drug – metformin. Do we need help preparing and issuing a prescription for 500 mg of metformin to you? You're welcome Mary, please tell us how else we can help you today.	Sentiment score: 8
The speaker in this text appears to be polite, helpful, and patient in addressing customer concerns. They take the time to review information, provide clear explanations, and provide assistance. However, the rating is not a 10 because there are still hints of uncertainty in some parts of the conversation, which may not convey complete confidence to the customer.

Start with clear instructions

The order in which the information is displayed in the prompt is important. This is because GPT-style models are built in a certain way, which defines how they process input. Our research shows that telling the model the task you want it to perform at the beginning of the prompt before sharing additional contextual information or examples helps produce higher quality output.

system information	user	Assistant
You are an AI assistant who helps users find information.	Your task is to verify whether the statement "Several sources mentioned the possibility of another major outbreak" is supported by specific quotes from the following set of excerpts. — Content snippet[1] Experts say 14% chance of major earthquake in Seattle SEATTLE - The U.S. Geological Survey estimates there is a 14% chance of a magnitude 9 Cascadia earthquake in Seattle over the next 50 years. “Unfortunately we can’t…[2] Seismological experts give latest outlook on ‘really big earthquake’ in Seattle “We say there’s about a 14% chance of another earthquake of about magnitude 9 occurring in the next 50 years,” Says a geophysicist at the University of Washington. — Several sources mention the possibility of another major outburst. Is this statement directly implied or explicitly stated by the content fragment?	No, this statement is not directly implied or explicitly stated by the content piece. The clips mention the possibility of a "megaquake" and a "Cascadia 9 earthquake" in Seattle within the next 50 years, but do not mention the possibility of another large eruption.

Repeat command at the end

The model may be susceptible to recency bias , which in this context means that information at the end of the cue may have a greater impact on the output than information at the beginning of the cue. Therefore, it is worth trying to repeat the instructions at the end of the prompt and evaluate the effect on the responses generated.

boot output

This refers to including a few words or phrases at the end of the prompt to get a model response that follows the desired form. For example, using “Here’s a bulleted list of key points:\n- ”the etc. tip can help ensure that the output is formatted as a bulleted list.

system information	user	Assistant
You are an AI assistant who helps users find information. Answer as little as possible.	John Smith married Lucy Smith. They have five children and he works as a software engineer at Microsoft. What search queries should I perform to check this fact? ## A possible search query is:	"John Smith married Lucy Smith, five children, software engineer Microsoft"

In the above prompt, the text " One possible search query is: " guides the model to produce a single output. Without this hint, the model will generate multiple search queries as output.

Add explicit syntax

Using clear syntax for prompts (including punctuation, headings, and section markers) helps convey intent and generally makes the output easier to analyze.

In the example below, separators (in this case ---) have been added between different information sources or steps. This allows use ---as a stopping condition for the build. Additionally, section titles or special variables are shown in uppercase for differentiation.

system information	user	Assistant
You are an AI assistant who helps users find information.	You will read the passage and then issue a query to the search engine to fact-check the passage. Also please explain the query. — Passage John Smith married Lucy Smith. They have five children and he works as a software engineer at Microsoft. What search queries should I perform to check this fact? —QUERIES	1. "John Smith Microsoft" - Check if John Smith actually works at Microsoft. 2. “John Smith Lucy Smith” – Check if John Smith is married to Lucy Smith. 3. "John Smith Children" - Check if John Smith had 5 children.
If you're not sure which syntax to use, consider using Markdown or XML. These models have been trained on extensive web content in XML and Markdown, which may provide better results.

Break down tasks

Large language models (LLMs) often perform better if the task is broken down into smaller steps. For example, in the search query prompt referenced earlier, the structure of the prompt can be adjusted so that the model is first instructed to extract relevant facts and then instructed to generate a search query that can be used to verify those facts.

system information	user	Assistant
You are an AI assistant who helps users find information.	You will read the passage and then issue a query to the search engine to fact-check the passage. — Passage John Smith married Lucy Smith. They have five children and he works as a software engineer at Microsoft. What search queries should I perform to check this fact? — Now you'll extract the factual statement and issue a fact-checking query to it. When issuing a query, use the function SEARCH ("query") fact statement	1. John Smith married Lucy Smith 2. They had five children 3. He worked as a software engineer at Microsoft QUERIES SEARCH("John Smith married Lucy Smith") SEARCH("John Smith Family Size") SEARCH("John Smith Microsoft Software Engineer")

Note that clear syntax should be used to distinguish different parts and guide the output. In this simple example, breaking the task from one step to two is not very dramatic, but when doing this for a large text with many factual claims, breaking the task down can make a big difference.

using affordances

Sometimes we can make the model use affordances instead of relying solely on its own parameters to obtain information and answers. For example, search can serve as an affordance to help mitigate the risk of fictitious answers and obtain the most up-to-date information.

A simple way to use affordances is to stop the model when it generates an affordance call and then paste the results back into the prompt. Below is an example of a follow-up call after executing the SEARCH call above. Notice how we paste the search results into the prompt and replace the previous SEARCH call.

system information	user	Assistant
You are an AI assistant who helps users find information.	You will read passages, extract factual claims, and then fact-check them using search engine results — Passage John Smith married Lucy Smith. They have five children and he works as a software engineer at Microsoft. What search queries should I perform to check this fact? — Factual Statement - John Smith is married to Lucy Smith - John and Lucy have five children - John works as a software engineer at Microsoft — Below are the various search queries issued to research the above statement: Search Query - John Smith married Lucy Smith - John Smith Number of Children - John Smith Software Engineer Microsoft — Below is a snippet of the search results: Content snippet: [1] … John Smith's wedding was on September 25, 2012 Day… [2] … John Smith attended a party with his wife Lucy [3] John took his two daughters and three sons to watch a football match [4] … After working at Microsoft for 10 years, Smith founded his own start-up company Tailspin Toys [5] John M was the town's blacksmith and he married Fiona. They had a daughter named Lucy — Fact-check each of the above factual claims based on the given snippet:	- John Smith married Lucy Smith: confirmed. Excerpts [1] and [2] both refer to John Smith's wife Lucy. - John and Lucy had five children: confirmed. Fragment [3] mentions that John was accompanied by his two daughters and three sons. - John works as a software engineer at Microsoft: Partial confirmation. Fragment [4] mentions that John has worked at Microsoft for 10 years, but does not explicitly state where he currently works. Further investigation may be required to confirm his current employment status.

Thought chain tips

This is a variation of the split-task technique. In this approach, instead of splitting a task into smaller steps, the model response is instructed to proceed step by step and to present all the steps involved. Doing so reduces the chance of inaccurate results and makes it easier to evaluate model response.

system information	user	Assistant
You are an AI assistant who helps users find information.	Who is the individual athlete who won the most medals (maximum number of medals) at the Olympic Games in Sydney? Take a step-by-step approach to answering the question, citing sources and giving reasoning before sharing your final answer, in the following format: The answer is:`<name>`	Step 1: Determine the year of the Sydney Olympics. The Sydney Olympics were held in 2000. Step 2: Find a list of individual athletes who won multiple medals at the 2000 Sydney Olympics. According to the official Olympic website, several athletes won multiple medals at the 2000 Sydney Olympics. Some notable names include: • Ian Thorpe (Australia) - 3 gold, 2 silver, 0 bronze • Peter van den Hoogenband (Netherlands) - 2 gold, 0 silver, 2 bronze • Lenny Kreiser Burg (USA) - 3 gold, 0 silver, 0 bronze Inge de Bruijn (Netherlands) - 3 gold, 1 silver, 0 bronze Step 3: Determine the athlete with the most medals. After reviewing the list of individual athletes who won multiple medals at the 2000 Sydney Olympics, we can see that the athlete with the most medals is Ian Thorpe from Australia, who won a total of 5 medals (3 gold and 2 silver) . So, the answer is: Ian Thorpe.

Specify output structure

When you use hints to specify the output structure, it can have a significant impact on the nature and quality of your results. Sometimes the system message input "Write only true facts" or "Don't make up information" may not be enough to alleviate the problem. Conversely, requiring model responses to also include citations helps reduce the probability of false responses.

If you instruct the model to cite source material when writing statements, those statements are more likely to be well founded. Asking for citations causes the model to make two errors every time it generates a response: the first error is a fabricated response, and the second error is a false citation. Note that the closer a citation is to the text it supports, the shorter the distance required by the model to predict the citation, suggesting that inline citations are better suited to mitigating the generation of false content than citations at the end of the content.

Likewise, if the model is asked to extract factual statements from a passage, it may extract compound statements such as "X is doing Y and Z" (which may be harder to verify). This can be avoided by specifying the output structure, such as (entity1, relationship, entity2).

The following example demonstrates the use of citations and guides the model response to adapt to a defined structure.

system information	user	Assistant
You are an AI assistant who helps users find information	List the different behaviors that a customer support agent would exhibit in the following conversation. — Hi, thanks for calling Contoso Pharmacy, who am I talking to today? I understand your frustration, Mary. Can you tell me what you want to achieve? OK, I'm happy to take a look at it for you. Please wait a moment. I see here that you have used generic Lipitor, right? Okay, so I see your doctor stopped prescribing these medications in 2021. Is this actually the case? Okay, let me check this for you. Because sometimes vulnerabilities appear in the system and are not reported. Please wait a moment and let me see if there is a record here. So between 2012 and 2021, your doctor prescribed these medications to you, according to the most recent doctor's office records provided to us. Okay, so the latest records show that now the doctor has switched to a different kind of drug – metformin. Do we need help preparing and issuing a prescription for 500 mg of metformin to you? You're welcome Mary, please tell us how else we can help you today. — How responsive was the customer support specialist? Classify and list various behaviors using the "behavior ("reason")" format and cite the reasons behind the classification	1. Polite greeting behavior (“The commissioner started the conversation with a polite greeting and asked for the caller’s name”) 2. Empathic behavior (“The commissioner recognized Mary’s frustration and tried to understand her concerns”) 3. Attentiveness Behavior ("The specialist listened to the customer's problem and asked for more details") 4. Proactive behavior ("The specialist offered to check the problem for the customer") 5. Informational behavior ("The specialist provided relevant information about the customer's prescription record" ”) 6. Comprehensive behavior (“The specialist checked for any gaps that may exist in the system and provided a complete response”) 7. Confirmatory behavior (“The specialist confirmed current medications with the client”) 8. Solution-oriented Behavior ("The specialist offered to fill a new prescription for the customer") 9. Polite behavior ("The specialist thanked the customer and offered to provide further assistance if needed")

system information	user	Assistant
You are an AI assistant who helps users find information.	Group the following news headlines into thematic categories based on the patterns you see in the text. Also mention the rationale for defining these categories. Output format: { "topic_name": "", "title": [], "reason": "" } Input news title: 1. "From book to lecture in 10 seconds using AR + ML" 2. " In 1993, 32-year-old Yann LeCun demonstrated the world’s first convolutional network for text recognition” 3. “First-order motion model applied to animated paintings” 4. “Robinhood and other brokerages actually prevent the purchase of $GME 、$ NOK、 $BB,$ AMC; SELL ALLOWED" 5. "United Airlines shares fell more than 5% in pre-market trading" Output:	{ "artificial intelligence": { "title": [ "From book to lecture in 10 seconds using AR + ML", "In 1993, 32-year-old Yann LeCun demonstrated the world's first convolutional network for text recognition", " First-order motion model applied to animation painting" ], "reason": "These titles are related to artificial intelligence, machine learning and their applications in various fields." }, "financial_market": { "title": [ " Robinhood and other brokerages effectively block purchases of $GME 、$ NOK、 $BB,$ AMC; allowed to sell", "United Airlines stock down more than 5% in premarket trading", "Bitcoin near $20,000 a year ago today" ], "reasons": "These headlines are relevant to financial Markets, stocks, cryptocurrencies and trading platforms. } }

Temperature and Top_p parameters

Changing the temperature parameters will change the output of the model. The temperature parameter can be set from 0 to 2. Higher values (such as 0.7) will make the output more random and produce more divergent responses, while smaller values (such as 0.2) will make the output more focused and specific. Fictional stories can be generated using higher temperatures. For generating legal documents, a much lower temperature is recommended. Top_probability is another parameter that, like temperature, also controls the randomness of the model response, but it is controlled differently. It is generally recommended to change only one of these two parameters at a time, rather than changing them both at the same time.

Provide basic context

One of the most effective ways to provide reliable answers is to feed the model with data and let it derive responses from the underlying data. If your use case relies on up-to-date, reliable information and is not a purely creative scenario, we strongly recommend providing underlying data. Generally, the closer the source material is to the final form of the desired answer, the less work the model needs to do, which means the less chance of error. The following example feeds the system " latest blogs describing the rollout of GPT-4 in the Azure OpenAI service " and asks it to name some early customers.

system information	user	Assistant
You are an AI assistant who helps users find information. You will provide your answer specifically through the following text, and keep your response to 100 words or less: “At Microsoft, we’re constantly discovering new ways to unleash creativity, unlock productivity, and upper-level skills so that more people can gain from using AI. Benefit. This allows our customers to... reduce the likelihood of models generating harmful responses." ¹ Answer in 100 words or less. Use bulleted lists whenever possible.	Who are the early customers of the Azure OpenAI service GPT-4 model?	Here are some early customers of the Azure OpenAI service GPT-4 model: - Epic Healthcare: They plan to use GPT-4 to help doctors and nurses spend less time on data entry and investigations. - Coursera: They use Azure OpenAI service with GPT-4 to create new AI-driven learning experiences on their platform. - Coca-Cola: They are using Azure OpenAI to build a knowledge center and plan to leverage the multimodal capabilities of GPT-4 to handle marketing, advertising, public relations, and customer relations. - Other companies of all sizes are using Azure AI to improve customer experiences, aggregate long-form content, help write software, and reduce risk by predicting the right tax data. ^1Text from the Azure OpenAI GPT-4 release blog .

file

Follow TechLead and share all-dimensional knowledge of AI. The author has 10+ years of Internet service architecture, AI product research and development experience, and team management experience. He holds a master's degree from Tongji University in Fudan University, a member of Fudan Robot Intelligence Laboratory, a senior architect certified by Alibaba Cloud, a project management professional, and research and development of AI products with revenue of hundreds of millions. principal