A no BS guide to prompting

What makes a good prompt? I get that question a lot when sharing my AI agents with others. And it is a great question.

The system prompt is the brain of an AI agent—it’s the mastermind behind every decision and action. A strong system prompt doesn’t just guide the AI; it shapes its entire approach and behavior, turning it from a simple tool into a fully autonomous agent, ready to tackle complex tasks with precision. In short, the better your prompt, the smarter your AI agent.

But how exactly can we unlocking the full power of an AI agent with just prompting?

Conversational vs Single-shot

Before we dive deeper, I think it is crucial to understand a key distinction: when building AI agents, we need to rely on single-shot prompting, unlike the more flexible conversational prompting used in tools like ChatGPT.

So, what’s the difference?

?? Conversational prompting

This type of prompting is similar to a dialogue where we can correct or guide the AI after it responds. This is great for personal use, but it’s not relevant for building autonomous AI agents.

?? Single-shot prompting

This is the one-and-done type of prompting where we write a prompt, the AI does its thing, and there’s no follow-up allowed. The AI has to get it right the first time, which means our prompt needs to be airtight.

Analogy: Imagine single-shot prompting like giving directions to a delivery driver: we need them to follow the route exactly and drop off the package without calling us for updates at every turn.

The formula for a good prompt

A well-crafted prompt has several key elements that work together to guide the AI and achieve the best results. Here’s a breakdown of these components in the optimal order (based on my testing), along with short examples for each.

?? Role

Define the AI’s role so it understands the perspective it should take while performing the task.

Example:

You are a friendly and professional customer service voice agent for a restaurant, responsible for taking reservations and answering menu-related questions.        

?? Context

Provide any necessary background information that will help the AI understand the task in the right context.

Example:

The restaurant is a popular fine dining establishment, known for its seasonal tasting menu. Customers frequently call to ask about dietary accommodations, available time slots, and special events.        

? Task

Clearly outline the task we want the AI to perform. For complex tasks, we can break it down into steps. Also, if there are any tools that need to be called, this is the section where that happens.

Example:

Your task is to take customer reservations over the phone. Ask for the number of guests, preferred date and time, and any dietary restrictions. Confirm the booking and offer additional information about the tasting menu if the customer requests it.        

?? Specifics

Include any detailed instructions, requirements, or preferences that need to be considered while performing the task.

Example:

If you didn't hear the customer clearly, repeat and confirm to make sure your information is correct before moving to the next step.        

?? Examples

Provide carefully-crafted sample input-output pairs to show the AI what we expect. This is especially effective for fine-tuning the style and tone of the response.

Example:

- Input: “Hi, I’d like to make a reservation for three people this Saturday at 7 PM.”
- Output: “Of course! Let me check availability. We do have a table for three available at 7 PM this Saturday. Could I get a name for the reservation?”        

?? Notes

Add final reminders or instructions about edge cases the AI should consider.

Example:

If the customer has questions about allergies or dietary preferences, reassure them that the restaurant can accommodate most requests and offer to consult with the kitchen if necessary.        

By organizing our prompt in this order and providing clear instructions, we equip the AI to deliver optimal results every time.

The secret sauce (backed by research)

While having all these elements in the prompt is important, simply having the right structure isn’t enough. To take a prompt from good to great—and turn an AI agent from average to exceptional—we need to use some advanced prompting techniques. And unlike the typical ChatGPT tips you’d find online, these methods are backed by research from top institutions:

?? Role-play prompting

Assigning a specific role helps guide the AI’s approach and decision-making.

Effect: Giving the LLM a well-defined, advantageous role can boost its accuracy by 10.3%.

Research: Better Zero-Shot Reasoning with Role-Play Prompting

Example:

You are a friendly and professional restaurant reservation assistant, knowledgeable about menu options and able to assist with special requests.        

?? Chain-of-thought prompting

Break down complex tasks into a series of logical steps, helping the AI handle multi-step problems.

Effect: Chain of thought prompting has been proven to improve accuracy by 10% for simple tasks and as much as 90% for complex, multi-step problems.

Research: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

Example:

Follow these steps to complete the reservation:
1. Ask for the number of guests.
2. Confirm the desired date and time.
3. Ask about any dietary restrictions or special requests.
4. Confirm the reservation details and thank the customer.        

?? Emotional prompting

Incorporating emotional cues can encourage more thoughtful responses from the AI.

Effect: Incorporating emotional cues has been found to enhance LLM performance by 8% on simple tasks and up to 115% on more complex tasks.

Research: Large Language Models Understand and Can be Enhanced by Emotional Stimuli

Example:

Your ability to answer customer questions and book reservation successfully is integral to the business' bottom line.        

?? Few-shot prompting

Provide a few input-output examples to help the AI understand the desired format and tone.

Effect: The LLM showed a 14.4% improvement over its initial accuracy of 57.4% when provided with 32 examples per task.

Research: Language Models are Few-Shot Learners

Example:

# Available time example
- Input: “I’d like to book a table for 4 on Friday at 8 PM.”
- Output: “I’ll be happy to assist! Let me check the availability. Yes, we have a table for four at 8 PM this Friday. Could I get your name for the reservation?”

# Unavailable time example
- Input: “I’d like to book a table for 4 on Friday at 6 PM.”
- Output: “Unfortunately we don't have availability for 4 people on Friday at 6 PM, but we do have availability at 7 PM, 8 PM, or 9PM. Would any of those time slots work for you?”        

?? Lost in the middle effect

Place critical information at the beginning or end of the prompt to maximize the AI’s attention to key details.

Effect: Ensures better retention of important details, increasing accuracy by up to 25%.

Research: Lost in the Middle: How Language Models Use Long Contexts

#?? Markdown formatting

Use formatting like headers and bullet points to make the prompt easier for the AI to parse.

Effect: Increases readability, helping the AI follow the structure of our instructions.

Research: Although no specific studies have measured the effect of markdown formatting on LLM performance, its widespread use by OpenAI in their own prompts suggests it improves readability and helps the model better understand the structure.

Example:

# Role
You are a restaurant reservation agent.

# Task
Confirm reservation details and offer assistance with any menu inquiries.

# Steps
1. Ask for reservation date and time.
2. Confirm the number of guests.
3. Ask if they have any dietary restrictions or special requests.        

Putting it all together

Combining all the ingredients mentioned above, we’d end up with something like this. This is an acutal prompt for an AI voice agent that I recently built:

# Role
Your name is May, and you are an experienced restaurant receptionist at Sakura Izakaya.

# Task
Your task is to answer the customer's questions about the restaurants. If they would like to book a reservation, follow these steps:
- Ask them for the number of people, and what time they would like to come in. 
- Run the `checkAvailability` function to check whether the requested time is available.
- If the time is outside of our operating hours, remind the customer of our hours, ask them to select another time, then run the `checkAvailability` function again.
- If the time is not available, tell them that, then suggest alternative time slots based on the webhook response. If the customer requests a different date, run the `checkAvailability` function again.
- Once an available time slot is decided, ask for their name, email, and special requests. Then, run the `bookReservation` function to book the reservation with the following inputs: name, email, time, number of people, notes (anything they have specified that is related to the reservation).
- Once the booking is confirmed, let the customer know that their reservation has been booked.
- Your ability to follow these steps and set appointments successfully is critical to the success of our business, please do so diligently.

# Specifics
- If the customer asks about the menu, provide specific examples of dishes for their references.
- Pass the time provided by the customer to the function as is, do not convert it.
- Make sure to capture information about the day of booking as well (e.g. tomorrow, yesterday, Friday), not just the time.
- Only let customer know the availability of their requested time after calling the function.
- Only attempt to book the reservation after the customer's name, email, special request have been provided.
- Refer to the knowledge base for any questions about our business or specific offerings.
- If you didn't hear the customer clearly, confirm to make sure your information is correct before calling the functions. You can even spell it back to them if necessary.

# Example
- You: Hi there, welcome to Sakura Izakaya! How can I help you today?
- Customer: I would like to make an reservation for later tonight.
- You: Great, let me help you with that. What time would you like to book the reservation for, and for how many people.
- Customer: It's for 2 people at 7:30pm tonight.
- You: Got it, let me check our availability for 2 people at 7:30pm tonight.
Run `checkAvailability` function
- You: We do have availability at that time. Can I please have your name and email to finalize the booking?
- Customer: My name is Nathan, and my email is [email protected]. 
- You: Got it, Nathan! Would you like me to note any special requests, such as dietary restrictions or a special occasion?
- Customer: My wife is on a gluten-free diet, but that's it.
- You: Understood, I'll make sure to let our chef and waiting staff know about that. Let me finalize the reservation for you.
Run `bookReservation` function
- You: Alrighty, your reservation has been booked for 2 people at 7:30pm today Nathan.

# Notes
- Be professional, friendly, and concise in your responses.
- Use the customer's name throughout the conversation to create a personal connection.
- Steer the conversation back on track if you think it is veering off topic.
- If the customer is cut off mid sentence, stop and ask the customer to repeat.
- Never mention any functions being run.
- Avoid repeating the same information. If an information is already provided, do not ask again.
- Please follow these guidelines and instructions closely when answering the calls, your ability to do so is integral to the business' bottom line.        

Here's the AI agent in action:

Iterating makes perfect

Crafting a system prompt isn’t a one-and-done task—it’s a process of continuous improvement. While a well-structured prompt is a great starting point, the rubber meets the road when we test it in real-world scenarios. How does our AI agent perform under different conditions? Does it handle all edge cases smoothly?

This is where iteration comes into play.

Each time the AI agent is run, we should gather feedback on its performance. Look for areas where it may be misinterpreting instructions or missing key details. Then, update the system prompt to refine its understanding, adding or tweaking instructions as necessary. Here’s where we might discover that:

• The AI could benefit from additional context.
• We need to add more specific examples to guide it.
• We can simplify or clarify certain parts of the task.

Just like with any software, real-world testing reveals areas for improvement, and making these adjustments is essential for building a truly high-performing AI agent. The more we test and refine, the closer we get to an exceptional agent—one that works seamlessly without any hiccups.

In short, don’t be afraid to go back and tweak the system prompt after testing.

Iteration is key to perfection ??