Experiments with ChatGPT: Short vs. Long Prompts Part 2

If you’re just tuning in, I’m running the latest of a series of ChatGPT experiments, this one comparing the results of using short vs. long prompts. You can read part 1 here just in case you missed it.

The whole run of tests took me about a day.

Each test took me about 15-30 minutes. The long prompt took the longest. It’s a multi-stage sequence that is long, wordy, and takes time to complete.

Here’s what happened.

Control Group (Long Prompt)

Time-to-Complete: 1 Hour

I feed ChatGPT my standard prompt sequence, broken down into several stages:

1.? Stage 1: Introduce the task

2. Stage 2: Analyze the company website

3. Stage 3: Analyze recent blog posts

4. Stage 4: Read the content brief

5. Stage 5: Competitor Analysis

The resulting outline is what I typically expect. The initial notes it generates are a little generic, and I have to ask Bradbury to expand the outline with more detail.

Short Prompt 1

Time to Complete: 20 minutes

Prompt: Create an outline for a blog post about conducting a performance review.

This was a much simpler prompt. The resulting outline was generic and neutral and had no particular alignment with my target audience of HR professionals.

I expected that though. I wanted to see how much ChatGPT can figure out on its own without additional guidance or details.

Short Prompt 2

Time to Complete: 15-20 minutes

Prompt: Outline a blog post on performance review methods, including key steps and best practices.

This prompt adds a little more detail by giving suggestions for sections and points to include.

The resulting outline was much better than the first prompt. It provided a much more comprehensive overview of the topic.

I notice a pattern. The initial outline that ChatGPT gives me is usually general and I have to ask for more detail. Is there a way I can get Bradbury to give me a detailed outline without me having to ask him?

Short Prompt 3

Time to Complete: 10-15 minutes

Prompt:

Stage 1: “What are the main topics to cover in a blog post about performance reviews?”

Stage 2: “Based on the topics, create an outline including subheadings and key points for each section.”

In this prompt, I introduce stages. I ask Bradbury to identify the best topics to cover first before making the outline.

The resulting outline covered more aspects of the topic like legal considerations and case studies. Overall it’s close to usable.

Short Prompt 4

Time to Complete: 22 minutes

Prompt: “Create an outline for a blog post about conducting a performance review, focusing on challenges and solutions.”

How does introducing a theme or a direction change the content of the outline?

This outline has the most detail, up front. I had to ask it to add more sections to give it more of an HR focus, but overall it’s not bad. I’m impressed.

Short Prompt 5

Time to Complete: 10-15 minutes

Prompt: Outline a blog post about performance reviews, targeting HR professionals. Include sections on preparation, execution, and follow-up.

Right off the bat, surprisingly detailed for a first-stage outline. It’s close to usable, but I have to do just a little bit of my own independent research before using it.

Decision-Making

Winning Prompt: Short Prompt 5

My tests confirmed what I suspected: my research and outline prompts are too bloated, too bulky, and overkill. I can achieve the same quality outline in half the time by keeping my prompts simple.

After a discussion with Bradbury, we agreed that Short Prompt 5 was the best one. It provided a clear structure, targeted HR professionals, and included specific sections that were relevant to the client. The result with a detailed and relevant outline.

A few takeaways we learned:

• Clarity and Specificity: Clear and specific prompts lead to more detailed and relevant outlines.
• Audience Targeting: Mentioning the target audience helps in tailoring the content.
• Balance: Striking a balance between providing enough guidance without overloading the prompt is key.

Prompts with detailed instructions resulted in more detailed and targeted outlines, guiding the language model in the right direction.

Vague prompts resulted in more generic outlines that needed more depth or relevance to the target audience.

Including a content brief - whether made by you or the client - is really helpful in creating outlines that align with the client’s business goals.

Based on the tests, the best approach seems to be a combination of clear instructions, audience targeting, and the inclusion of specific sections or client content.

Another thing I noticed is that in many cases, the client brief and competitor analysis stages can often be combined into one.

With those changes, I modified my prompt to make it more closely resemble Short Prompt 5, and reflect those changes in my guide to ChatGPT for content marketing. While I keep the Content Brief and Competitor Analysis stages, I also make notes to combine them to save time.

The result? I can now make outlines in 15 minutes when before it would take me as much as an hour.

Thank goodness. Now I can go back to being a writer.

SaaS founders and agency owners, how have you built ChatGPT and AI tools into your workflows to make your business more efficient? Share your insights in the comments.