By Andrew Macri , Garrett Spong

Large language models (LLMs) can give inconsistent answers when the same question is asked multiple times. For example, if you ask for help writing an Elasticsearch query, sometimes the resulting query might be wrapped with an API call, even though we didn't request it. This sometimes subtle, sometimes dramatic change adds complexity when integrating generative AI into analyst workflows that require a specific format for responses, such as queries. In the short term, LLM providers may offer a patchwork of vendor-specific techniques to improve the repeatability of responses.

Fortunately, beyond these emerging vendor-specific techniques, there is a general pattern for building prompts at the beginning of a conversation to steer it in the right direction. We'll create an instant sandwich to improve the quality and consistency of chatting with your preferred LL.M. We'll explore the various parts of an instant sandwich and illustrate its practical application in the Elastic AI Assistant .

DALL E 2 generated cue image: cyberpunk handing steampunk AI assistant a sandwich

Elastic AI Assistant integrates with Elastic Security to detect alerts , events, rules , and data quality checks as context for generating AI chats. The helper can help make sandwiches quickly and reduce privacy risks through field-level pseudonymization of values.

Making Instant Sandwiches for Generative AI

Starting a new conversation with a prompt sandwich applies to writing prompts for the most popular LLMs and sets the tone for high-quality and consistent responses because it:

Guide the Assistant's dialogue and roles
provide background information
ask a question or make a request

Consider the following example of a quick sandwich:

You are an expert in Elastic Security. I have an alert for a user named alice. How should I investigate it?

The prompt can be deconstructed into three vertically stacked parts, similar to a sandwich:

_________________________________________________
/ \
| You are an expert in Elastic Security. | (1)
\ _______________________________________________/
+------------------------------------------------+
| I have an alert for a user named alice. | (2)
+------------------------------------------------+
/ \
| How should I investigate it? | (3)
\_______________________________________________/

Each layer of the tip sandwich has a name and a specific purpose:

System prompt
Context
User prompt

The unique purpose of each layer is outlined below:

Purpose	example
System prompt	Steers the conversation and role of the assistant	You are an expert in Elastic Security.
Context	Data or metadata, which may be structured or freeform	I have an alert for a user named alice.
User prompt	Asks a question or makes a request	How should I investigate it?

This three-layer format provides guidance on building the prompt and which parts to modify first when building and tuning the prompt for reproducibility.

System promt - system prompt

System hints are the top layer of the sandwich, using common techniques like character hints to build dialogue and the Assistant's persona.

A system prompt can be combined with many different context and user prompts.

When building new prompts or tweaking existing prompts for more consistent responses, it's often best to start with existing system prompts and add specific instructions to user prompts. System hints are more reusable if details are preserved where needed.

Reasons to consider modifying or creating new system prompts include:

Repeat the same command across numerous user prompts
Request to include or exclude answer categories
Results need to be explained to different audiences such as security analysts or executives

Context - the context

The context is the middle layer of the sandwich, usually data or metadata. It is critical to keep data privacy in mind when providing background information in prompts to third-party LLMs. Most popular LLMs accept contextual data or metadata in structured format (such as comma-separated values) or unstructured text.

Context Data may contain potentially personally identifiable information. To reduce privacy risks, certain values may be overridden before being sent as chat context. For example, when user.name is sent to LLM as context, jem (the value of the user.name field ) might be replaced with a newly generated unique ID like 3a4a99d0-60a5-4ab9-99dd-b6786da5b8b2. When a response is received from LLM, the Unique ID will be replaced with the original value. This technique is an example of pseudonymization, since re-identification of the data subject is impractical without additional information.

We'll discuss later how Elastic AI Assistant's data anonymization feature provides customizable field-level options for including, excluding, and generating pseudo-anonymized values for certain types of structured contexts, such as alerts.

User promt - user prompt

User hints are the bottom layer of the sandwich. We're here to ask questions like "How should I investigate it?" Or make a request, such as summarizing this alert.

When optimizing for repetitive prompts, consider requesting output in a specific format. For example, you can add format your output in markdown syntax to the user prompt.

Typically, you will create many more user prompts than system prompts. User hints should be your starting point when refining hints to ensure consistency.

Build repeatable prompts in Elastic AI Assistant

Now that you understand how to identify the layers in a hint sandwich, let's examine the technique in action to build repeatable hints in the Elastic AI Assistant.

Clicking the "Chat" button throughout Elastic Security puts data (such as the example alert in the screenshot below) as context for the chat:

Clicking the "Chat" button will bring the alert into the chat as context

The three layers of the prompt sandwich - (1) System prompt, (2) Context, and (3) User promt - are labeled in the screenshot below.

Elastic AI Assistant annotated screenshot

Before sending any data to LLM, you can use Elastic AI Assistant's inline editing promt preview (shown above) to:

Add new, edit or delete system prompts
Add additional context, delete context, and configure field-level anonymization for certain types of data
Preview or edit user prompts selected via the Quick prompts feature

You can choose to create custom system prompts and user prompts through the settings shown in the screenshot below:

The "Settings" view above also configures the default values for data anonymization.

Data anonymization (pseudonymization)

Elastic AI assistants can optionally provide anonymization for certain types of contextual data, such as alerts, so that you can:

Allow or deny specific fields to be sent to LLM as context
Turn anonymization on or off for specific fields
Set the above defaults by field or in bulk

Only the Allowed field in the context of the Alert (from summary) example below will be sent to LLM:

In the example above, the values of all three fields are anonymized when sent to LLM. You can accept the configured defaults or interact with the preview to toggle specific fields.

See what was actually sent

Click the "Showed anonymized" switch to display the anonymized data sent to and received from the LLM in the chat, as shown in the animated gif below:

Review what was actually sent

Above: The "Show anonymized" switch shows anonymized data sent and received from LLM.

in conclusion

Making instant sandwiches can improve the consistency of responses when generative AI is incorporated into the workflow. When tweaking prompts, use the three layers of the prompt sandwich: System prompt, Context, and User prompt as a guide for what to change and when.

Keep in mind the importance of privacy when sending data to LLM as context. Elastic AI Assistant's anonymity feature reduces privacy risks by allowing only specific fields to be sent to LLM and applying field-level pseudonymization to specific values.

Start your free Elastic Cloud trial with a free trial of the Elastic AI Assistant. For more information, including how to integrate it with your model of choice, read our documentation .

If you're a developer, check out Elasticsearch Relevance Engine™ (ESRE) , vector search, and Elastic's first contribution to LangChain .

The release and timing of any features or functionality described in this article is at the sole discretion of Elastic. Any features or functionality not currently available may not be delivered on time or at all.

In this blog post, we may have used third-party generative artificial intelligence tools that are owned and operated by their respective owners. Elastic has no control over third-party tools, and we are not responsible for their content, operation, or use, nor shall we be liable for any loss or damage that may arise from your use of such tools. Exercise caution when using artificial intelligence tools with personal, sensitive or confidential information. Any data you submit may be used for artificial intelligence training or other purposes. There can be no guarantee that information you provide will be secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative artificial intelligence tool before using it.

Elastic, Elasticsearch and related marks are trademarks, logos or registered trademarks of Elasticsearch NV. in the United States and other countries. All other company and product names are trademarks, logos or registered trademarks of their respective owners.

原文：Crafting Prompt Sandwiches for Generative AI | Elastic Blog