Most effective way to prompt - July '24: Turning ChatGPT from gimmick to a productivity tool

At lot has improved since ChatGPT 3.5 came out 18 months ago, including our understanding of how and where to use it.

Having used ChatGPT, Perplexity, and Claude for over a year, I’ve repeatedly gone back to 5 key ways to get high quality results from these chatbots.

This has allowed me to:

• work faster & more productively
• complete unfamiliar work with less need of expert help, and
• reason with knowledge well beyond what Google can let me do.

This is the first of a two-part series where first I share ‘the how’ - how I prompt the chatbot (e.g. how do I effectively instruct it to give answers that are useful and not gimmicks) and, later I will share ‘the where’ - where in my workflow I leverage the chatbots to make me most effective.

Foundations - before we start:

Principles not Templates.

To me, Prompt Templates is snake oil - stay away, especially from ‘AI Gurus’ looking to sell you the ‘100 best prompt templates’. Prompt templates imply there are defined ways to prompt. There are none because GenAI is weird.

While there are no templates, there are tried and tested principles. Principles guide how we write prompts considering the strengths and weakness of the LLMs and chatbots.

We’ve arrived at these principles based on our partial understanding on how chatbots generate their answers and through 18 months of experimentation. However, it is currently impossible to fully determine how each chatbot arrives at its final answer. Experts and researchers still cannot pinpoint the exact process.

A curious example being, if you ask ChatGPT for a list of 10 innovative ideas, it gives better answers if you tell it to assume the identity of Steve Jobs than if it were Elon Musk. No one knows exactly why this is. We only know it is weird.

Follow principles of good prompting, not templates.

Improvement through test and practice.

The weirdness also means even the latest chatbots are not equally competent across different tasks. This is called to the ‘Jagged Frontier’. Some models are excellent at specific tasks but struggle with others. This is why using a combination of chatbots and experimenting which works best for you is vital.

Personally, I use ChatGPT, Claude and Perplexity. I alternate between them based on my past experience with their answer quality for different tasks (in Part 2, I will list which chatbot I prefer for my common use cases). There is no definitive test for determining which is best at each task, that is for you to determine.

Test them to know them. Test them with different challenges, use them at different parts of your workflow, see which one has the personality your resonate with.

Note: Benchmarks might point to arbitrary measures of performance, but usefulness depends on how you expect your tech tools to support you. You are an expert in your own field, so only you can determine where the chatbots and which chatbot is most useful to you.

AI on average is... average.

The all purpose models (ChatGPT, Claude, and Gemini) were trained on A LOT of information. The majority of the internet in fact, and we know the internet is full of terrible information.

Put glue on pizza? Eat rocks? Yep. That was on Reddit and Gemini gobbled it out and gave it out as good advice.

Thankfully these are rare cases but it reminds us that the quality of the material on the internet ranges from excellent to terrible to the sarcastic. On average, the information is average. Meaning if you put in an average nondescript prompt into the ChatBot you will get an average answer.

Learning the principles of prompting points the AI to look at the excellent material it was trained on to give you excellent responses.

The 5 Key Principles:

1. Provide context

The foundation of writing a good prompt is the same as asking a good question or giving a detailed instruction. You need to provide the necessary background information to frame the question and make the question / instruction explicit.

Giving the chatbot context helps it narrow down the information it should seek and how it should present its response. The context you provide should flesh out your question or instruction, this includes information on:

• Purpose - what you aim to achieve with the response (e.g. write a LinkedIn post)
• Audience - who will be reading the response (e.g. graduates who are new to working a full time office job)
• Constraints - specific preferences or requirements to adhere to (e.g. 400 words, easy to consume, free from jargon)

Let create an example to use:

Starting (average) prompt:

What is the importance of work life balance?         

Better prompt:

Write a LinkedIn post that I can share with my networking explaining the importance of work life balance when you are new to your job but looking to impress. Your response should be no longer than 400 words and aimed at graduates who are new to working a full time office job, so keep the content easy to consume, conversational, and free of jargon.        

2. Provide examples

The Chatbots are weird because they don’t answer or format responses in a consistent manner. Providing examples helps it understand the desired response format more clearly.

You do this by copying and pasting examples as part of your prompt, but you must also tell the chatbot you are giving it an example and what within the example it should consider.

Continuing with our earlier example:

Write a LinkedIn post that I can share with my networking explaining the importance of work life balance when you are new to your job but looking to impress. Your response should be no longer than 400 words and aimed at graduates who are new to working a full time office job, so keep the content easy to consume, conversational, and free of jargon. 

I will give you an example first, copy the tonality, short sentencing, and the use of bullet points. 

Example:

I used to haul trash in the streets. No shame in that. 

Today, I coach the world's biggest brands and creators. 

The ironic thing is, I hauled trash so I could pay for my university education. And now I don't even use my diploma in my work. Ha! Talk about "life fails". 

But failure isn't necessarily a "step down". Or "back". Failure pushes you to go forward. Examples: 

1. I've failed at 3 businesses before LinkedIn 
2. I've lost clients, missed deadlines, and more 
3. I've lost $5k, $10k, and $50k retainer contracts 
4. I've cleaned offices, toilets, and construction sites 
5. I've been "digging" this online thing for 15 years now I've "failed" over and over.

Only to keep winning. 
Let's stop glorifying success stories on LinkedIn, friends. And let's embrace failure as a natural part of the journey. 

"Fail forward", says the quote. 
Fail today → Win tomorrow. 

P.S. What's the best quote / advice you've ever heard?         

(The example is copied from the LinkedIn of Jasmin Ali? a famous LinkedIn coach)

3. Assume a style or personality

Instructing the chatbot to role play a specific personality has proven to be a key addition on prompts that can significantly improve the quality of answers.

This works as the chatbot is forced to look for specific content it was trained on rather than accessing all the information it has. This is a way to steer it away from the average and poor training data.

Obvious examples include asking it to become a famous person, e.g. Steve Jobs, or a role e.g. you are a highly successful LinkedIn influencer or a professor in area X.

Building on our example:

You are the highly respected professor of organisational psychology, and a career coach with 20 years experience. Write a LinkedIn post that I can share with my networking explaining the importance of work life balance when you are new to your job but looking to impress. Your response should be no longer than 400 words and aimed at graduates who are new to working a full time office job, so keep the content easy to consume, conversational, and free of jargon. I will give you an example first, copy the tonality, short sentencing and the use of bullet points […]        

4. Provide critical feedback

LLMs will always give you an answer despite how much or little confidence it might have in an answer. The AI companies (OpenAI, Anthropic, Google etc.) have fine tuned and aligned the LLMs to not provide inappropriate answers, but this somewhat skews how answers are presented. Coupled with the fact they have little capability to reflect on their answers, as the human user you will need to provide it ample feedback.

The good thing is the chatbots have infinite patience. They will take whatever feedback and criticism you throw at it and attempt to create a new response that aligns with the previous answer and the feedback you gave.

Your feedback could include:

• pointing out logical gaps
• telling it you disagree with a certain point and you want it changed
• telling it to try harder / think deeper / be more creative

Example:

I find these answers boring, people would have seen this so many times, be 100 times more creative and original. 

I want you to only provide original thoughts given you are a famous organisational psychologist.        

Note: While AI companies have fine-tuned the LLMs to be responsive to feedback, the chatbot can inadvertently get stuck in a rut (e.g. it anchors on its initial response) and despite the feedback you give it, it continues to give poor answers. The best course of action is to start a new session and restart the conversation.

5. Use Chain of Thought

As humans, we tend to jump to conclusions - we do our best deep thinking when we think step by step. These chatbots are no different. Asking them to think step by step improves their answer quality as it enables the AI to take more compute cycles before presenting its final answer.

The error is asking the AI to cut the waffle and jump straight to the answer. It’s like asking your child not to think through their logic and to answer what comes first to their mind.

Chain of thought is a proven technique used to counter this. It is widely used by researchers to guide the chatbot to think out their answer. The downside is that it increases the prompt complexity and the process of getting to an answer takes much longer. It’s a technique relevant to complex tasks, not when you need a simple answer.

Continuing on from our earlier example with a simple chain of thought addition:

You are the highly respected professor of organisational psychology, and a career coach with 20 years experience. Write a LinkedIn post that I can share with my networking explaining the importance of work life balance when you are new to your job but looking to impress. Your response should be no longer than 400 words and aimed at graduates who are new to working a full time office job, so keep the content easy to consume, conversational, and free of jargon. 

I want you to think through your response by doing the following step by step: 

1. Start by briefly outlining your thought process for approaching this task. Consider the key elements you need to address: work-life balance, impressing as a new employee, and tailoring the message for recent graduates. 

2. Draft out 3 main points you want to make in the LinkedIn post. Explain why you chose these points and how they relate to the target audience. 

3. For each of the main points, describe how you would present it in a way that's easy to consume and conversational. Give a brief example or analogy you might use to illustrate each point. 

4. Consider potential objections or counterarguments to your advice. Briefly explain how you would address these in the post without using jargon. 

5. Outline how you would structure the post to make it engaging and impactful within the 400-word limit. Explain your choices for the opening and closing sentences. 

6. Finally, write the actual LinkedIn post, incorporating the thought process I’ve just outlined.

Take your time to follow these steps, I want you to demonstrate the reasoning behind each of your choices for the LinkedIn post, I want you to show how you're tailoring the content and style to meet the specific requirements I presented.         

As you can see, the prompt is much longer and more complex.

The chain of thinking it goes through forces the chatbot to compute the prompt more thoroughly resulting in a deeper answer.

Creating the chain of thought prompt can be difficult. One of the ways to bypass this is to ask the chatbot to create it during a pre-stage, for example:

Help me rewrite this prompt to incorporate chain of thought: [your original prompt goes here])         

For complex tasks, I often have prompts in excess of five hundred words (even without the example as mentioned in Principle 2). The bulk of the words are within the chain of thought instructions.

For even more critical tasks, I incorporate loops so it to writes a full draft and then have it assume a personality to critique the draft before it rewrites it. In the example above, you can have the draft reviewed first by ‘a graduate’ and then by ‘famous CEO’ etc before asking it to provide the final answer.

Looping between drafts and reviews can make the response longer. But to get quality answers, allow it the paragraphs to think through but have in your prompt an instruction to present the final answer in full (like I did in step 6 in the example). Just scroll past its thinking and review the answer at the end.

Summary on the Art of Prompting well

Improving the quality the answers you get is done via prompting the chatbot well. There are 5 key principles in improving prompts and answers.

A quick way to remember all of the above, is in the acronym PACES CC:

• Purpose
• Audience
• Contraints
• Examples
• Style & Personality

Enhance its thinking through:

• Critique
• Chain of Thought

Each of the different chatbots are trained on different swaths of information and their fine-tuning is a bit of a mystery, this gives each one a different personality and preference in how they answer.

The only way to know which one gives you the most useable answers is to test them. The 5 principles above (PACES + CC) will give you a head start in making better use and developing a deeper understanding on how this incredible technology can graduate from being a gimmick to a powerful tool for you.

(How many of you were keen eyed enough to see that my title is about ChatGPT but all my screenshots are in Claude? Learn why in part 2)

Follow me to get an update when I publish part 2 where I cover the key use cases that have made my consulting work more efficient, helped me launch Corci, and allowed me to be a deeper thinker.