Engineering Prompt Engineering: Ensuring You and LLMs Are on the Same Page

Almost everyone seems to be on the hunt for the ultimate prompt (or two) that will help them make money, save the world, and figure out dinner by 6PM. If you haven’t joined the race and are unfamiliar with prompts or prompt engineering: prompts are simply the messages we submit to AIs to ask them a question or perform a task. Prompt Engineering is the “art” of finding optimal ways of telling the AI what you’re after.?Here’s an example from an education-focused site:?

Create a magical system that emphasizes education and is based on [topic of your choice].

As you can see some can be quite simple but do provide some inspiration for educators to improve engagement. You may be more like me and prefer to blaze your own trail, which should improve one’s ability to understand how to ask LLMs for what you really want.

With that said, I’d like to share a conversational method I’m starting to use to help get my prompts just right.

Lost in Translation?

The point of prompt engineering is having an unambiguous request that can provide predictably good results in general. One problem at present is everyone is having a go at “prompt engineering” and sharing their tricks with the world. Sometimes they are great, sometimes the really miss the mark.

For our own attempts at finding these gems we need to make sure the AI understands precisely what we mean. To do that I’m asking LLMs (Large Language Models) like ChatGPT to tell me what it thinks I asked of it.

Checking Assumptions?

Please tell me in your own words how you would interpret the following text prompt. The goal is to help me better understand how to relay my questions to LLMs in general and ChatGPT 4 specifically.

A little on the nose, but it did provide an excellent description of what I was after. If my prompt didn’t successfully communicate what I wanted, this is where those misunderstandings will show up.

In this scenario, I’d go through an iterative process with the LLM providing it with more explicit descriptions of what I want it to do. There’s a prompt made infamous by a Ted Talk because that caused ChatGPT to miss some basic assumptions about the conditions for drying clothes. These misunderstandings would have likely become apparent under this method of checking how it interpreted the prompt. In cases where assumptions are critical, adding text to the “in your own words” prompt requesting a list of the assumptions it’s using would definitely be beneficial.?

Once the AI can restate to you the full understanding of what it’s being asked to do, you can move on to the final stage. Now we want to shrink it down to a reasonably concise prompt that will leave no ambiguities or missing context in order to effectively reuse the prompt we’re working on.

Are there possible changes that would improve how it's structured? I'm hoping to have a version that preserves every point of critique without any awkward formatting. If you could rewrite it as the perfect prompt for asking you to perform the critique you were just able to describe, what would that be?

You should end up with a fairly easy-to-read version of your initial prompt.?

Here’s a real-world example of this process, which I used while working on improving the results when ChatGPT critiques my writing.

Original Prompt:

I would like you to review the following text and provide a critique in the following areas: clarity of language, clarity of ideas, avoidance of awkward phrasing, potential points of simplification if it can be done without losing both detail and my personal writing 'voice,' persuasiveness, and feasibility. For each of these, I'd like you to provide a point-by-point list of problem areas you discover and your suggestions to fix each one.?I would also like your view on its reading level. Please perform this review on the following task but don't rewrite it for me.

ChatGPT’s understanding of the task:

The prompt is asking me to provide a detailed critique of a text that will be provided. The criteria for the critique have been explicitly stated. They include:

1. Clarity of Language: This refers to how easily the text can be understood. Are the words used correctly? Is the syntax correct? Is the text free of jargon, or if jargon is used, is it properly defined?
2. Clarity of Ideas: This is about how effectively the text conveys its main points. Are the ideas presented in a logical sequence? Are the main points clearly stated? Is there enough evidence or explanation to support each point?
3. Avoidance of Awkward Phrasing: Awkward phrasing can disrupt the flow of the text and make it harder to understand. Are there sentences that are difficult to read because of the way they're structured? Are there phrases that are too long or too complicated?
4. Potential Points of Simplification: This is about identifying areas where the text could be simplified without losing detail or the unique voice of the writer. Are there complex sentences that could be broken down into shorter ones? Are there words or phrases that could be replaced with simpler ones?
5. Persuasiveness: This refers to how compelling the text is. Does it make a convincing argument? Does it use evidence effectively? Does it address potential counterarguments?
6. Feasibility: This is about whether the ideas presented in the text are practical and achievable. Are there any logical inconsistencies? Are the proposals realistic given the constraints that might exist?

For each of these criteria, I am being asked to identify specific problem areas in the text and suggest ways to improve them. The prompt also asks me not to rewrite the text, but simply to offer suggestions for improvements.

Finally, I am asked to determine the reading level of the text. This is a measure of how difficult the text is to understand. It can be assessed based on factors such as sentence length, word complexity, and the use of jargon or complex ideas.”

It’s final version:

“I have a piece of text for which I'd like a thorough review. Please analyze it based on these specific criteria:

1. Language clarity: Are the words and phrases clear and easy to understand?
2. Idea clarity: Are the ideas or arguments presented logically and coherently?
3. Phrasing: Are there any awkwardly phrased sentences or phrases?
4. Simplification: Can any part of the text be simplified without losing detail or the unique style of the writer?
5. Persuasiveness: Is the text compelling and convincing?
6. Feasibility: Are the ideas or proposals realistic and achievable?

For each of these criteria, please identify any issues and provide specific recommendations for improvement. I don't need you to rewrite the text, just to give guidance on how it could be improved. Lastly, could you assess and provide the reading level of this text?”