Is ChatGPT Getting Worse?

Welcome to the 37th edition of Humanity Working. This week, things will be a little different - I’m going deep into how the AI world is clashing with practices in day-to-day work.

All About that Hype

If you are involved in technology, you’ve probably seen this:

It called the Hype Cycle, and was invented by Gartner. The Hype Cycle explains what happens when something new, particularly a piece of technology, arrives. First, people get very excited about it. Then there is something of a backlash. But all along, the technology itself is improving, so eventually more and more people figure out how to get value out of it truly, and finally it becomes a core part of our lives. In my experience, almost any transformative technology works like this, from cars to the Internet and now generative AI. It’s a fascinating demonstration of how humans respond to change.

But even though this response happens almost every time, what’s not predictable is how fast it happens. In my 35 years in technology, I’ve seen some innovations ease through the phases and others go much faster. But I have not seen anything go as fast as generative AI.

Why?

Inflated Expectations

As I’ve mentioned in a previous newsletter, ChatGPT 3.5 launched on Nov 30th 2022, and within 9 months over 50% of knowledge workers had used it. No other technology even comes close to that speed of adoption. We RACED through to inflated expectations.

Our research over the last year for our Thrive With AI program suggests that three factors made this happen:

• Voice assistants primed us to use it from day 1
• Working with the technology seemed as natural as talking to someone
• The technology addressed a pain that society feels at a deep, emotional level

I’ll touch on each one of those briefly in turn:

Y’all Ready For This?

Remember when Siri launched? It was nearly 13 years ago, and suddenly, it felt like the future was here. We could now talk to our devices as easily as to our friends.

Of course, once we got over the novelty, it was pretty clear that Siri and its cousins from Amazon, Microsoft, and Google just weren’t that smart - intellectually or emotionally. The allure of talking to someone who just doesn’t get you fades over time, and these voice assistants didn’t get most of us. Strangely, they haven’t got much better since. My son was born a few months before Siri, and he knows WAY more now than he did that day. But Siri? It seemed to stall despite a trillion-dollar company investing in its progress.

Yet?Siri and friends did accomplish?something—they created the idea—and expectation—that this was how we should communicate with computers, that Star Trek was possible and not that far away, and that primed us for something better…

I Slice AND Dice!

Siri and pals promised so much and delivered so little.

Then GPT 3.5 launched.

Now, to be clear (and fair), conversational agents like Siri are solving a different problem to generative AI. But from the day it launched, ChatGPT seemed to have no limits - to be flexible enough to deal with any question. It seemed to know a little about an awful lot and was happy to make a confident guess about anything else. Ethan Mollick called it “like a hardworking intern that lies a bit.” I called it an “overconfident overachiever.” But whatever we chose to call it, its attributes made it seem more human and, as a result, more approachable.

And it was useful as well! Quickly, word spread that this thing could actually do real things. It would happily attempt almost anything, and often, it would get a B+ that, with some effort, you could turn into an A! If you did a job where good enough was the goal, this was your ultimate partner.

So, as many more of us became generalists, generative AI was our ultimate assistant—never saying no and always giving us something we could work with. And before we went home for the day, it would write us a sonnet we could send to our loved ones and pass off as our own.

I Feel Your Pain

Technology that is easy to use and actually does things is compelling enough. Still, something else really put generative AI over the edge—it relieved (or at least appeared to relieve) a pain that we feel across all levels of society—the pain of being overwhelmed.

Today, we live in a world where there is always more to do than time to do it, and we work in a world of never-ending and always-expanding to-do lists. We have been asked to “do more with less” for decades, and nobody really knows what good looks like.

Against this backdrop comes technology that never gets tired, can take on any to-do item we ask without complaining, and often seems to compress a week’s worth of work into a few seconds.

It’s no wonder that when we asked participants in our beta Thrive with AI program what they were most excited about, 95% said “saving time.” We are time-poor, and generative AI promises to fix that.

But…

Deflated Reality

Well, that was quick.

For many, the generative AI honeymoon is already over, but it shouldn’t be surprising. When huge numbers of people adopt technology quickly, they rapidly discover its limitations.

But some specific factors also came into play

• We got frightened
• We got confused
• We got let down

Fear

Technology has been coming for many jobs since it became a thing, but let’s face it: Generative AI is different. It’s really quite good at a lot of things, and unlike most humans, it never gets sick, is indefatigable, and never loses motivation.

That’s a triple threat in its own right. But perhaps more alarmingly, it looks like generative AI will become much more capable pretty quickly. If generative AI is threatening your job today, imagine what it will be like in 5 years!

So, engaging with generative AI is a double-edged sword for many of us. Make my work easier? Yes please? Free up time? Absolutely. But take my job? Not so much.

And the anxiety doesn’t stop there. Many of the employees we have spoken to are also worried that they will just be unable to keep up with the speed of change. They’ve taken in how quickly this is evolving and wonder out loud if they can truly take advantage of new capabilities emerging not just every year but every day. At what point do they not have the energy to keep up?

Disappointment

As humans, we want more time, and AI promises to help.

But does it?

At this point, if you have spent any time with Generative AI, you will know that sometimes it takes more time than it gives. But perhaps more damningly, you will know that a prompt engineering course, the kind that teaches you what questions to ask AI, usually won’t help you figure out ahead of time when AI be a time saver and when it will be a time suck.

And that can be a huge, deep disappointment.

Confusion

If you’ve been using ChatGPT for the last few weeks, you’ve probably noticed its behavior is changing in odd ways. A little while back, it started promising PNG files and delivering web files. It’s latest trick? It takes complicated requests and promises to get back to you in several hours, only to fail to do so.

But that’s not what’s supposed to happen. Generative AI isn’t supposed to need time off.

While I don’t have proof as to why this is happening, it’s not hard to guess. GPT is probably overpromising and underdelivering to manage user demand as more people join the service. It’s creating a sub-optimal experience for each of us as individuals so it can keep the service running globally.

Whatever the underlying reasons, the result for users is not just annoying; it’s confusing. Can you rely on it or not? Will it help you to meet a deadline or cause you to blow through one when it lets you down?

Reaching Enlightenment

No wonder many people are getting disillusioned.

But not everyone is. Some people are near the end of the hype cycle and are figuring out how to use AI daily to improve their lives. Exactly how is almost impossible to explain in a few sentences, but here are four things we’ve repeatedly seen.

1. They avoid productivity rabbit holes. These people accept they cannot predict when AI will make them faster, so they rapidly try a few things. If the approach isn’t working, they attack the problem in a new way, such as asking AI to structure an approach rather than perform all the steps itself.
2. They use AI differently depending on their own level of expertise in a topic, the balance they need between speed and quality, and the areas where AI can be most helpful.
3. They focus more on AI’s ability to improve output quality rather than the time it will save.
4. They reevaluate how they use AI weekly. AI is constantly changing—getting better at some things, different at many, and occasionally worse at a few things. These people evaluate that constantly and adapt their workstyle as AI alters.

Thrive with AI

Generative AI is not a cure-all, but it can be enormously helpful to individuals, teams, and organizations if they really understand how to maximize its potential.

Thriving with AI is not just about asking the right questions; it’s about a mindset combined with a practical skillset—focused on welcoming your AI teammate into your environment and making the best of it.

The general release of our Thrive with AI program is just a couple of weeks away. If you are an existing BillionMinds customer, you can probably access it for free (ask us). But even if you are not currently a customer, I strongly recommend learning more about it and seeing if it's right for you or your organization.

This program is in general release from September 4th. If you would like to join us, please e-mail [email protected].

Thanks For Reading!

I'm Paul and I'm the CEO and Co-Founder of BillionMinds. If you are worried about how prepared your employees are for change - change in work environments (like hybrid and remote), business strategy, or even technology changes, you should talk to us. Just reach out to me here on LinkedIn and we can get a call scheduled.

As for this newsletter - please let me know your thoughts on it in the comments (I try to respond to everything)

If you liked this newsletter, chances are someone else will too, so be sure to share it with them! Oh, and don't forget to subscribe!

I also post regularly on LinkedIn outside of this newsletter - you can make sure you miss nothing by following me.

You can also subscribe to our YouTube channel and follow me on X.