Almost Timely News: Improving the Performance of Generative AI Prompts (2023-12-17)

Almost Timely News: Improving the Performance of Generative AI Prompts (2023-12-17) :: View in Browser

?? Register for my new Generative AI for Marketers course! Use ALMOSTTIMELY for $50 off the tuition

Content Authenticity Statement

90% of this newsletter's content was generated by me, the human. Some of the prompt responses in the opening are generated by ChatGPT's GPT-4 model and are marked as such. Learn why this kind of disclosure is important.

Watch This Newsletter On YouTube ??

Click here for the video ?? version of this newsletter on YouTube?

Click here for an MP3 audio ?? only version ?

What's On My Mind: Improving the Performance of Generative AI Prompts

Today, let’s talk about getting better performance out of large language model systems using prompt engineering. Over the past 3 months, I’ve had a change of heart and mind about prompt engineering. Originally, I was aligned with what a lot of industry folks were saying about prompting, that the need for prompt engineering was going to go away as models became smarter. But the more time I spent in the trenches with models, especially the open source ones, the more I realize there’s some nuance there.

In general, for the average user of a large language model, that is a true statement, that prompt engineering will probably get less important over time. As models get smarter, they generally get better at guessing user intent, thanks to human feedback being incorporated into language models. However, there are a couple of shades of grey here.

The first is that large public models are also being censored more and more heavily. Ask an image model for a Disney reference and you’ll likely be told no. Ask a language model for a point of view about politics and you’ll get some linguistic gymnastics worthy of a politician themselves.

Here’s the thing with censorship of models: it diminishes their performance. Imagine you had a cookbook and you decided to censor the use of wheat. Now imagine going through that cookbook and ripping out every page that referenced wheat. You would have a severely diminished cookbook when you were done, and you would be capable of cooking much less, including recipes where wheat wasn’t the main focus, like a Beef Wellington. Imagine pretending Beef Wellington didn’t exist because you eliminated references to wheat. That’s what model censorship does. With added censorship comes added skill needed to get the most out of models.

The second shade of grey is that more advanced prompt engineering takes advantage of the architecture and structures of the models to get better results faster. For example, imagine you have a library, and you want to put together some books to check out. You could absolutely just walk around the library and collect books, and you’d end up with what you were looking for. That’s general prompting. Now imagine the library had a specific classification system and internal architecture - say, ISBN numbers or the Dewey Decimal system. How much faster could you find the books you were looking for if you had that internal representation and architecture of the library?

That’s what prompt engineering at its peak does - it doesn’t just instruct the models about what to do, but takes advantage of the way models work to deliver better results in less work. Now, to be clear, that doesn’t mean you’re doing it wrong today. If you’re getting good results from models, then that’s really all that matters. But if you’re curious about how to get better results in less work, then you’ll want to adapt a few techniques to improve your use of language models.

We’ve talked before about the RACE structure for prompt engineering , and it’s really good at what it does. The reason is that the RACE structure, when you follow it, has enough of the terms needed for a model to form the statistical associations it needs to generate great output. Here's what I mean. Suppose you said to a chef, "I'm hungry," and that was the entire prompt. The chef has so little to go on that they'll cook you SOMETHING, but it stands to reason it's not going to be what you want.

Suppose you said, "I'm hungry for pizza". That's a lot more specific than I'm hungry, but there are limitless variations of pizza. The chef might be Japanese and make you a favorite in Japan, creamed corn and squid pizza. If you love Japanese pizza, then you get lucky and have a good pizza experience. If you don't love Japanese pizza, then there's a good chance you're still not going to have an enjoyable experience.

Suppose you said, "I'm hungry for pizza. I'd like a margarita-style pizza with fresh mozzarella, fresh basil that's been cut chiffonade, and a tomato sauce made from Roma tomatoes and tomato paste to a very thick consistency. I'd like the crust to be thin, less than a centimeter thick, and I'd like it to be cooked at very high heat for very fast, so that the bottom is partially blackened. I'd like it to be 510 centimeters in diameter with 10 grams of sea salt and 15 grams of black pepper sprinkled onto the sauce prior to the basil and cheese being put on."

Do you have a very clear mental image of this pizza? Would our chef? Absolutely. With this level of clarity and detail, we can be reasonably sure our chef is going to produce a pizza that we want to eat. That's what the RACE structure - and many other prompt engineering structures - do. They provide enough detail and keywords and concepts that the language model is able to execute the task well.

Now, here's where we'll go beyond the basics. If you're not getting good results from a model today, there's a good chance the task you're asking it is too vague, too imprecise, and too big. Let's dig into a very specific example. Suppose you want a language model to emulate your writing style. You say something along the lines of "You're a writing expert in the style of Ann Handley. You know writing, writing voice, writing styles, how to be a great writer. Your first task is to read this sample of my writing. {insert sample here} Provide a bullet point list of my writing style."

That's a good prompt; it follows the RACE framework and will certainly give you a better result than a vague prompt like "Analyze my writing style." Here's an example of the output, using GPT-4 and the ChatGPT interface:

• Conversational Tone: Your writing feels like a dialogue, engaging the reader as if in a conversation. This approach makes complex topics more accessible and relatable.
• Use of Analogies and Examples: You frequently use analogies (like the cookbook and wheat example) and detailed examples (such as the pizza description) to clarify and illustrate your points, making abstract concepts more tangible and understandable.
• Educational and Informative: Your writing aims to educate the reader about a specific topic, in this case, prompt engineering. You provide a thorough explanation, backed by examples, to ensure the reader understands the subject matter.
• Structural Clarity: You organize your content logically, leading the reader through your thought process in a step-by-step manner. This clarity in structure aids in comprehension and keeps the reader engaged.
• Incorporation of Personal Insights and Experiences: You include your personal journey and changes in perspective, which adds authenticity and depth to your writing.
• Balanced Argumentation: You present multiple sides of an issue (the nuances of prompt engineering) and explore different aspects (like censorship and advanced prompt engineering techniques), offering a balanced viewpoint.
• Technical yet Accessible Language: While discussing technical topics, your language remains accessible, avoiding jargon that might alienate readers not familiar with the subject.
• Engaging and Thought-Provoking: Your writing is designed to provoke thought and encourage readers to consider their own views and practices in light of the information provided.
• Emphasis on Practical Application: You focus on the practical applications of the concepts discussed, showing how they can be used effectively in real-world scenarios.
• Clear and Direct Communication: Your writing is straightforward, making your points clearly and directly without unnecessary embellishment.

This is a good start, to be sure, but it's still too vague. Watch what happens when I ask it to write a paragraph explaining the importance of the 10-year/3-month Treasury spread in macroeconomics:

Imagine diving into the world of macroeconomics like you're flipping through a cookbook, where the 10-year/3-month Treasury spread is a key ingredient, much like salt in a recipe. Picture the 10-year Treasury bond as a hearty stew, taking time to simmer and develop flavors, and the 3-month Treasury bill as a quick, refreshing salad. The spread between them, the difference in their yields, is crucial in predicting the economy's health, akin to a chef tasting and adjusting their dish. When the 10-year yield is higher, it's like having all the right ingredients for a robust economy, but when this spread narrows or inverts - imagine your quick salad taking longer than your stew - it often signals economic downturns, like a chef sensing a recipe might flop. This analogy helps to demystify a concept that, like a dash of salt, might seem small but is essential in bringing the whole economic picture to life.

That's not bad, but it's not quite my style. Why is that?

It's because writing style is too big a task for a language model to produce in one shot. What is a writing style? Take a few moments to mentally think about that. What is a writing style? What are the parts of a writing style?

There's word choice, rhythm, cultural perspective, grammar, voice, tone, punctuation, spacing, intertextuality, lexical diversity... the list of components that form a writing style itself is a full page of text. How many different pieces did you think of? Now, suppose you took the same writing sample and the same prompt, but broke it up into pieces. Here's an example:

"You're a writing expert in the style of Ann Handley. You know writing, writing voice, writing styles, how to be a great writer. Your first task is to read this sample of my writing. {insert sample here} Provide a bullet point list analysis of my writing style specifically about my use of grammar."

Here's what comes out:

• Complex Sentences: You frequently use complex sentences with multiple clauses, showcasing a strong grasp of advanced sentence structure. This helps in conveying detailed information and nuanced ideas effectively.
• Varied Sentence Length: Your writing includes a mix of short and long sentences. This variation helps maintain reader engagement and aids in emphasizing key points.
• Correct Use of Punctuation: You skillfully use punctuation, like commas, periods, and quotation marks, to clarify meaning and separate ideas. This contributes to the readability and professionalism of your writing.
• Consistent Verb Tenses: Your writing demonstrates a consistent use of verb tenses, which maintains clarity and coherence throughout the text.
• Appropriate Use of Conjunctions: You effectively use conjunctions to connect ideas and sentences, facilitating a smooth flow of thoughts and a logical progression of ideas.
• Active Voice Predominance: The majority of your sentences are in the active voice, making your writing more direct and engaging.
• Proper Noun-Verb Agreement: Your writing shows a consistent noun-verb agreement, which is fundamental for grammatical accuracy.
• Effective Use of Modifiers: You use adjectives and adverbs to modify nouns and verbs for more precise and descriptive communication.
• Well-structured Paragraphs: Each paragraph is well-structured, with clear beginning, development, and conclusion, enhancing the overall coherence of your writing.
• Use of Direct and Indirect Speech: You skillfully incorporate both direct and indirect speech, adding variety and depth to your narrative.

That's a lot more specific, isn't it? Now suppose I re-ran that prompt, but I asked about punctuation, or spelling, or whitespace, or pacing, or... you get the idea. Writing style is so large and vague a concept that the response of a language model is going to be equally vague.

By breaking writing style down into its component pieces and asking the model for its analysis of my writing in those component pieces, I end up with a much more detailed, focused, and practical definition of my writing style.

Let's see what the results are if I use a prompt with a gigantic specification of my writing style, based on this technique of breaking down a big concept into its component concepts.

In the realm of macroeconomics, the 10-year/3-month Treasury spread is a crucial indicator, often likened to a compass guiding the financial markets. Essentially, the spread measures the difference in yields between the 10-year Treasury note and the 3-month Treasury bill. When investors are nervous, they flock to long-term bonds, pushing those yields down, seeking safety but signaling danger. When the yield on the 10-year note falls below that of the 3-month bill, it's an inversion, a signal historically predictive of an impending recession. This inversion is like a warning light in a car's dashboard, alerting economists and business professionals alike to potential economic slowdowns. For instance, in digital marketing, being ahead of the curve in understanding economic trends can be as crucial as predicting the next big shift in social media algorithms. An inverted spread can indicate a shift in investor sentiment towards risk aversion, much like how consumer preferences can suddenly shift online, affecting marketing strategies.

That's much better and more clear; behind the scenes, the prompt that I used to fuel this was 17 paragraphs long, each paragraph an aspect of my writing style.

Why does this work? Writing style is an aggregated concept. It's a concept that bundles a whole bunch of other concepts together; when you do that kind of bundling, you get a good overview of the concept, but you miss a lot of the nuance and subtlety in the individual components. And that nuance, that subtlety, is where style emerges. Everyone knows the basic big picture concept behind spaghetti and meatballs - but the little details are what sets apart one dish from another, details that might not be captured in the big picture.

Any time you're getting results that aren't quite what you want using prompt engineering, ask yourself whether the task is broken down in enough detail that the model knows what to do. Today's big public models can handle prompts that are very large in size, so you can afford to be more detailed in what you provide for instructions. Think of language models like the world's smartest interns. The results you get are directly proportional to the clarity of instructions you provide.

If you'd like to learn more about the RACE framework and prompt engineering, good news: my new Generative AI for Marketers course just launched ! With over 5 hours of instruction, tons of hands-on exercises, a workbook, and a certificate of completion, it's a great way to level up your generative AI skills. Use discount code ALMOSTTIMELY for $50 off the tuition.

If you'd like a deep dive into what's in the course to see if it's right for you, check out this video tour of the course .

How Was This Issue?

Rate this week's newsletter issue with a single click. Your feedback over time helps me figure out what content to create for you.

• ?? Good
• ?? Neutral
• ?? Bad

Share With a Friend or Colleague

If you enjoy this newsletter and want to share it with a friend/colleague, please do. Send this URL to your friend/colleague:

https://www.christopherspenn.com/newsletter

For enrolled subscribers on Substack, there are referral rewards if you refer 100, 200, or 300 other readers. Visit the Leaderboard here .

ICYMI: In Case You Missed it

Besides the new Generative AI for Marketers course I'm relentlessly flogging , I recommend

• You Ask, I Answer: How Not To Use Generative AI In Healthcare?
• You Ask, I Answer: Experimenting with Generative AI?
• Mind Readings: Generative AI For Marketers Course
• You Ask, I Answer: Generative AI Time Savings?
• You Ask, I Answer: Favorite Uses of Generative AI Workflow?
• Almost Timely News, December 10, 2023: Where Generative AI and Language Models are Probably Going in 2024
• So What? Generative AI for Marketers
• In-Ear Insights: Can Generative AI Replace the CEO?

12 Days of Data

As is tradition every year, I start publishing the 12 Days of Data, looking at the data that made the year. Here's the first 5:

• 12 Days of Data 2023 Day 1: Private Social Media Communities
• 12 Days of Data 2023 Day 2: Instagram for Brands
• 12 Days of Data 2023 Day 3: Instagram for Influencers
• 12 Days of Data 2023 Day 4: Global ChatGPT Conversations
• 12 Days of Data 2023 Day 5: Content Republishing
• 12 Days of Data 2023 Day 6: Top News Stories of 2023
• 12 Days of Data 2023 Day 7: Press Release Statistics of 2023
• 12 Days of Data 2023 Day 8: The Marketing Jobs Outlook
• 12 Days of Data 2023 Day 9: The 2023 Economy Big Picture
• 12 Days of Data 2023 Day 10: Creator Platforms For Monetization

Skill Up With Classes

These are just a few of the classes I have available over at the Trust Insights website that you can take.

Premium

• ?? Generative AI for Marketers
• ?? Google Analytics 4 for Marketers
• ?? Google Search Console for Marketers (?? just updated with AI SEO stuff! ??)

Free

• ?? The Marketing Singularity: How Generative AI Means the End of Marketing As We Knew It
• Powering Up Your LinkedIn Profile (For Job Hunters) 2023 Edition
• Measurement Strategies for Agencies
• Empower Your Marketing With Private Social Media Communities
• Exploratory Data Analysis: The Missing Ingredient for AI
• How AI is Changing Marketing, 2022 Edition
• How to Prove Social Media ROI
• Proving Social Media ROI
• Paradise by the Analytics Dashboard Light: How to Create Impactful Dashboards and Reports

Advertisement: Generative AI Workshops & Courses

Imagine a world where your marketing strategies are supercharged by the most cutting-edge technology available – Generative AI. Generative AI has the potential to save you incredible amounts of time and money, and you have the opportunity to be at the forefront. Get up to speed on using generative AI in your business in a thoughtful way with Trust Insights' new offering, Generative AI for Marketers, which comes in two flavors, workshops and a course.

Workshops: Offer the Generative AI for Marketers half and full day workshops at your company. These hands-on sessions are packed with exercises, resources and practical tips that you can implement immediately.

?? Click/tap here to book a workshop

Course: We’ve turned our most popular full-day workshop into a self-paced course. The Generative AI for Marketers online course is now available. Use discount code ALMOSTTIMELY for $50 off the course tuition.

?? Click/tap here to pre-register for the course

If you work at a company or organization that wants to do bulk licensing, let me know!

Get Back to Work

Folks who post jobs in the free Analytics for Marketers Slack community may have those jobs shared here, too. If you're looking for work, check out these recent open positions, and check out the Slack group for the comprehensive list.

• Analytics-Cja Business Consultant at Adobe
• Brand Designer at Rippling
• Brand Designer, Sales Enablement at Rippling
• Customer Data Platform Manager at WGU
• Data & Product Analyst - Tagvenue at Tagvenue
• Digital Analyst (M/W/D) at Peak Ace AG
• Director, Content Marketing at Rippling
• Field Marketing Manager, Channel at Rippling
• Junior Digital Analyst (M/W/D) at Peak Ace AG
• Manager, Community at Rippling
• Senior Content Marketing Manager, Finance at Rippling
• Senior Content Marketing Manager, Hr at Rippling
• Senior Copywriter at Rippling
• Senior Copywriter, It at Rippling
• Senior Manager, Media & Sponsorships at Rippling
• Senior Web Designer at Rippling

What I'm Reading: Your Stuff

Let's look at the most interesting content from around the web on topics you care about, some of which you might have even written.

Social Media Marketing

• LinkedIn SEO Tips: Optimize Your Profile And Content via Backlinko
• Social Media Collaboration: How to Work Together on Social
• Instagram Marketing Strategy Guide: Tips for 2024

Media and Content

• 10 Content Promotion Strategies That Work in 2024 via Copyblogger
• 5 Things to Consider When Taking a Content-First Strategy For SEO via Spin Sucks
• Content Pruning: What Is It? How to Do It Like A Pro In 2024

SEO, Google, and Paid Media

• SEO Automation: 17 Tasks Streamlined for Enhanced Efficiency
• Ahrefs vs SEMrush vs Raven Tools: SEO Software Comparison
• How to Track SEO Progress and Measure Results Beyond Rankings

Advertisement: Business Cameos

If you're familiar with the Cameo system - where people hire well-known folks for short video clips - then you'll totally get Thinkers One. Created by my friend Mitch Joel, Thinkers One lets you connect with the biggest thinkers for short videos on topics you care about. I've got a whole slew of Thinkers One Cameo-style topics for video clips you can use at internal company meetings, events, or even just for yourself. Want me to tell your boss that you need to be paying attention to generative AI right now?

?? Pop on by my Thinkers One page today and grab a video now.

Tools, Machine Learning, and AI

• Mixtral of experts | Mistral AI | Open source models
• Use Amazon DocumentDB to build no-code machine learning solutions in Amazon SageMaker Canvas via AWS Machine Learning Blog
• What Happens If ChatGPT Becomes the Internet? via The Atlantic

Analytics, Stats, and Data Science

• Website Analytics: Are you use your analytics is set up correctly?
• Marketing Analytics: Attribution Is Not Incrementality via Occam's Razor by Avinash Kaushik
• Increase Analytics Influence: Leverage Predictive Metrics! via Occam's Razor by Avinash Kaushik

All Things IBM

• Seven customer service types that organizations should provide via IBM Blog
• Building cyber resiliency for your data with IBM FlashSystem via IBM Blog
• Delivering security and scalability in todays business landscape requires more than setting up a front line of defense via IBM Blog

Dealer's Choice : Random Stuff

• (3) 'Piano Man' (BILLY JOEL) Song Cover by The HSCC Feat. Danny Lopresto - YouTube
• Potassium Intake, Bioavailability, Hypertension, and Glucose Control - PMC
• Bioavailability of potassium and magnesium, and citraturic response from potassium-magnesium citrate - PubMed

How to Stay in Touch

Let's make sure we're connected in the places it suits you best. Here's where you can find different content:

• My blog - daily videos, blog posts, and podcast episodes
• My YouTube channel - daily videos, conference talks, and all things video
• My company, Trust Insights - marketing analytics help
• My podcast, Marketing over Coffee - weekly episodes of what's worth noting in marketing
• My second podcast, In-Ear Insights - the Trust Insights weekly podcast focused on data and analytics
• On Threads - random personal stuff and chaos
• On LinkedIn - daily videos and news
• On Instagram - personal photos and travels
• My free Slack discussion forum, Analytics for Marketers - open conversations about marketing and analytics

Advertisement: Ukraine ???? Humanitarian Fund

The war to free Ukraine continues. If you'd like to support humanitarian efforts in Ukraine, the Ukrainian government has set up a special portal, United24, to help make contributing easy. The effort to free Ukraine from Russia's illegal invasion needs our ongoing support.

?? Donate today to the Ukraine Humanitarian Relief Fund ?

Events I'll Be At

Here's where I'm speaking and attending. Say hi if you're at an event also:

• Tourism Industry Association of Alberta's Tourism Summit, Edmonton, February 2024
• Social Media Marketing World, San Diego, February 2024
• MarketingProfs AI Series, Virtual, March 2024
• Australian Food and Grocery Council, Melbourne, May 2024
• MAICON, Cleveland, September 2024

Events marked with a physical location may become virtual if conditions and safety warrant it.

If you're an event organizer, let me help your event shine. Visit my speaking page for more details.

Can't be at an event? Stop by my private Slack group instead, Analytics for Marketers .

Required Disclosures

Events with links have purchased sponsorships in this newsletter and as a result, I receive direct financial compensation for promoting them.

Advertisements in this newsletter have paid to be promoted, and as a result, I receive direct financial compensation for promoting them.

My company, Trust Insights, maintains business partnerships with companies including, but not limited to, IBM, Cisco Systems, Amazon, Talkwalker, MarketingProfs, MarketMuse, Agorapulse, Hubspot, Informa, Demandbase, The Marketing AI Institute, and others. While links shared from partners are not explicit endorsements, nor do they directly financially benefit Trust Insights, a commercial relationship exists for which Trust Insights may receive indirect financial benefit, and thus I may receive indirect financial benefit from them as well.

Thank You

Thanks for subscribing and reading this far. I appreciate it. As always, thank you for your support, your attention, and your kindness.

See you next week,

Christopher S. Penn