ChatGPT versus Bard: Which AI Chatbot is Better for Marketing?
I received early access to Bard, Google’s answer to ChatGPT, so I obviously wanted to make a quick comparison of ChatGPT versus Bard on several common marketing tasks:
This blog post will report on the results of these ten tests.
ChatGPT versus Bard general performance
If you’ve used ChatGPT4, you know it is blazing fast. A full report on virtually anything in two seconds!
Bard is slower, about 10 seconds for a result. I mean, that is definitely a First World problem, but this is a First World blog post! It felt weird having to wait for a result after using ChatGPT for a while.
Also, Bard tends to add a little summary sentence at the end of everything. It’s rather useless, and I prefer ChatGPT’s no-nonsense, no-fluff response to a problem.
One advantage to Bard: When you create content, there is a little arrow that says View Options. If you click that, you’ll see that Bard gives you three more options to view. Pretty cool.
Finally, Bard seemed to have more character limitations compared to ChatGPT, requiring me to get more creative with some prompts.
When you log in to Bard, it emphasizes that it is still an experiment, and I was one of the first-week users, so we’ll keep that in mind when we see the results. Hopefully everything will get better week by week.
Now, let’s get on with the ten tests!
Test 1: Compose a simple essay
The prompt: Compose a 300-word essay, in the voice of author Mark Schaefer, about the importance of community-based marketing.
ChatGPT did a lovely job and it obviously drew from the thousands of blog posts I’ve written over the years. With a little editing, I could have easily used this output as a blog post.
Bard did not recognize me as an author, providing this cop-out:
I do not have enough information about that person to help with your request. I am a large language model, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions, but my knowledge about this person is limited.
This is a surprising result since I assumed the Google-based Bard would have access to at least the same amount of information about me as ChatGPT.
It does make me wonder … how much of the ChatGPT output was a “hallucination” — just faking the answer as best it could. Did it really know me? Is the Google response unsatisfactory but more honest?
More on that in a minute.
Winner of Round 1: ChatGPT (I think!)
Test 2: Recognize industry trends that could influence a marketing strategy
Prompt: I am meeting with the owner of a boutique salon dedicated to physical fitness and wellness. I need to create a marketing strategy. What are the five trends in this industry I should be aware of?
Both platforms came up with identical lists that included:
Having worked in this industry, I recognize that these are relevant trends. Since ChatGPT only has data through 2021, I expected some difference in the lists, so ChatGPT did well despite this obstacle.
Winner of Round 2: Tie.
Test 3: Summarize a piece of content
In our marketing jobs, it’s always great if we can get a summary of an article, report, or speech to save us time. I copied and pasted one of my blog posts into each platform to see the results.
The blog post was about the importance of creating insights, not just information, to stand out in the digital world. I copied and pasted the post and asked for a 100-word summary.
Walter Isaacson defines genius as a combination of curiosity and an ability to recognize patterns. Creating insights thus involves being curious and connecting the dots in a new way. People can become more insight-driven by connecting the dots from the past to the present, connecting people through collaboration, connecting to the experience by observing, and connecting the content by making original points. Unfortunately, many content creators today are playing it safe, but insights require some brain room, trust, and courage to really take off.
The ChatGPT summary (84 words)
Insight is the ability to see things in a new way and make connections between seemingly unrelated ideas. It is essential for creativity and innovation. To become more insight-driven, you can connect the dots, connect the people, and connect the experience.
The Bard summary (41 words)
The Bard result confused me. Why did it stop at 41 words?
I loved that ChatGPT made a reference to Walter Isaacson (as I did), named all four points in the article, and ended by stating the problem experienced by most creators. While Bard provided a more condensed version, it missed some major points.
Winner of Round 3: ChatGPT.
Test 4: Identify impacts on the field of marketing today
The prompt: What are the primary impacts on the field of marketing today?
Both platforms came up with similar lists and explanations. To make this post more concise, I’ve only included the lists, not the explanations.
ChatGPT:
Bard:
The field of marketing is constantly evolving, and there are a number of primary impacts that are shaping the industry today. Some of the most significant impacts include:
If you’ve played around with ChatGPT, you know that if you ask the same question five times, you’ll likely get five different combinations. It generates replies based on statistical regularities in their training data. That means that if you ask them the same question, you won’t always get the same answer. It also means that how you word a question can affect the reply, and for some of these queries we asked follow-ups to get better responses.
I tried this test several times, and neither one really stood out as particularly insightful. They are similar enough that I would rate it as a tie.
Winner of Round 4: Tie.
Test 5: Take a complex table of data and create a useful summary
I went to the U.S. Department of Labor and copied a complex data set about marketing employment in my home state of Tennessee. This was an analysis of 2022 fourth-quarter trends, by month, across every county in the state.
I told each platform what the data set represented, what each column of numbers meant, and asked for a five-point summary of the data.
ChatGPT result:
Summary of Marketing Employment in Tennessee:
Bard result:
Here is a summary of the data:
Bard came up with a much better result. In the ChatGPT summary, the first point could not have been concluded from the data I provided. Seems to be a hallucination. The Bard result is more of what I would have expected — how many people work in marketing, how much do they make, where is most of the employment?
Winner of Round 5: Bard by a lot.
Test 6: Create a marketing plan
Prompt: I am creating a marketing plan for an electric bike shop in Knoxville, Tennessee. Please create a specific marketing plan for this small shop specifically for the Knoxville market.
Both results were nearly identical and not very good. Neither referred to any specific information about the Knoxville market, or even about electric bikes. Both results were bland templates without any insight, originality, or depth.
By the way, as a marketing consultant, I regard this as good news!
Winner of Round 6: Nobody.
Test 7: Summarize survey responses
Over the last few weeks, I’ve learned that ChatGPT is a champ at taking a list of survey responses and summarizing it into useful themes. This has saved me hours of time!
So I already knew ChatGPT could perform. Now to put Bard to the test.
I tried this out on several customer surveys, copied the responses, and used this prompt:
Summarize the following survey data into five main themes: (paste all survey data)
Unsurprisingly, Bard also did an excellent job, and the resulting lists were nearly identical. Bard seems to default to a numbered list with bold highlights, which was actually a nice touch in this test.
Winner of Round 7: Tie.
Test 8: Provide five unique content ideas
Prompt: Please give me five excellent headlines that have never been written before for an article about the impact of AI on marketing
ChatGPT:
Bard:
Here are 5 headlines for articles about the impact of AI on marketing that have never been written before:
None of these headlines are spectacular, but this could have been influenced by the fact that I added a pretty severe restraint: Creating a headline that had never been written before! I did a quick search and it is probably true that none of these headlines existed before.
I decided to try another test and have each platform write a headline for this blog post.
Prompt: Please give me five excellent headlines for an article comparing ChatGPT and Bard for marketing tasks
Bard:
It bothered me that ChatGPT assumed I wanted something about “marketing automation.” That was not part of the prompt! The Bard headlines were much better, and I used version three for this post.
Winner of Round 8: Bard
Test 9: Edit a long podcast transcript into a blog post
I recently had a PERFECT application for AI. I had done a podcast interview, and in an answer to a question, I covered an aspect of marketing strategy that would make a great blog post.
Editing a podcast transcript is a pain, at least if you’re a perfectionist like me. We simply do not speak in rational, acceptable sentences. I’ve found that both platforms are great at editing, but the limit on the number of characters is painful.
My friend Joeri Billast is a Web3 consultant and he created a good workaround. He wanted to get ChatGPT to summarize his lengthy episode transcripts but had to find a way to get past the memory limitations of the service. Here is his super-useful hack:
Joeri provided an example: this blog post about a podcast episode was 95% created by ChatGPT.
Both ChatGPT and Bard have character limits, so you have to use this “remembering” strategy to create your narrative. It’s not easy in either case.
The results were very different, however.
ChatGPT followed my instructions, but I still needed to do a lot of editing to make it into a decent post.
Bard was determined to do it its own way. It kept writing the post in a high formulaic manner, like this:
The Importance of Marketing for Musicians
Marketing is the process of creating a customer. For musicians, this means creating fans who will buy their work. There are two types of marketing: direct marketing and brand marketing.
Direct marketing
Direct marketing is when you offer a discount or promotion to encourage people to buy your product. For example, if you give away a free song to anyone who signs up for your email list, that’s a form of direct marketing.
Brand marketing
Brand marketing is about creating an image or feeling that people associate with your product. For example, when you drink Coca-Cola, you might feel happy, warm, and friendly. This is because Coca-Cola has created a strong brand that is associated with these feelings.
How to Market Your Music
There are many ways to market your music. Here are a few tips:
Be consistent with your brand. Make sure that your music, your image, and your online presence all match what your audience expects.
Connect with your fans on social media. Use social media to share your music, connect with fans, and build a community.
Get involved in online communities. There are many online communities where you can connect with other musicians and music fans.
Promote your music on other websites. There are many websites where you can promote your music, such as YouTube, SoundCloud, and Bandcamp.
Get involved in live music. Playing live shows is a great way to connect with fans and build a following.
The weirdest thing is that even when I delivered the exact text to Bard, it made crap up. For example, nowhere in my transcript did I mention ANY of the bullet points in the Bard summary.
Although ChatGPT still needed some polishing, it followed my directions and Bard was just a bizarre little liar.
Winner of Round 9: ChatGPT
Test 10: Create social media posts
Prompt: Find five 100-word snippets from this content that would make excellent and unusual social media posts:
I then posted the text from a blog post called Five trends pointing your business to the metaverse.
I figured a post with five ideas like this would make it easy to atomize into posts.
ChatGPT
Bard
Sure, here are five 100-word snippets from the content that would make excellent and unusual social media posts:
These snippets are all relevant to the topic of the metaverse, and they are all interesting and engaging. They are also all short enough to fit easily on a social media post.
Notice that extra little summary Bard adds at the end. I just hate that. What’s the point?
Also, Bard made stuff up. Points four and five were not part of my post.
ChatGPT was accurate and nicely added some hashtags to the social media posts. Sweet.
Winner of Round 10: ChatGPT
Conclusion
I’ve had a lot of fun playing around with these platforms, but what we see from this ChatGPT versus Bard comparison is that even simple marketing tasks are unreliable at this point.
It’s no surprise, but it’s worth repeating, that when you use these tools, you need to check your work!
While ChatGPT has an edge on most of these tasks, the rate of improvement in these applications is incredible. And as these models become more tightly integrated into marketing- and content-specific applications, our world is going to be transformed rapidly.
Although these results were mostly unimpressive, we have to assume this is all going to get better, fast. The thing that hurt Siri and Amazon Alexa is that they did not “learn” and improve fast enough. Hopefully the tech companies have learned their lessons, and a year from now, these experiments will have dramatically improved results.