As the Hype Settles, What Have we Learned About AI?

Last year, AI took the world by storm.?

Some were super-excited, proclaiming this would almost “solve world hunger” (not literally, of course, but some grandiose claims), some were super-fearful, worried about job displacement, and in certain dystopian variations, some worried that the machines would take over the world...?

Fast forward a year, and it looks like Gen AI is finding its level. I, for one, am happy to be done with the early, frothy stages of the Gen AI hype cycle.

If you look at the software vendor landscape too, it’s settling down. Almost every product vendor has some features that are powered by Gen AI. We do too.

And, of course, we have the legion of VC-backed AI “Start-Ups” that have been cashing in on the hype but are effectively no more than a thin-wrapper around OpenAI’s GPT Model…

So now that Gen AI is becoming the new normal, enterprises are now keener than ever to safely adopt & implement this technology into their operations.

Most large organizations have already dabbled with different LLMs, but in truth, the majority are still in the experimental, exploratory phases.?

And there are still major uncertainties around large-scale adoption, maintenance and choosing the right tasks to automate.

Now, I’m fortunate enough to work closely with some of the largest enterprises across various regulated industries (such as defense & space, healthcare, and financial services).

And since we at VisibleThread are also highly involved in building out Gen-AI-based extensions for our own product line, I'm hearing quite a bit about the current state of Gen AI in the enterprise.

So in this edition, I’m going to outline my observations when it comes to safely implementing and adopting Gen AI in the enterprise. Of course, these are opinions, and your mileage will vary.

So, let’s dive right in.

—

The dangerous misconception about Gen AI’s capabilities

Now, I’ve touched a bit on the “AI Hype” we experienced last year.?

Everyone, especially non-technical professionals, were blown away by the capabilities of Large Language Models like GPT.

First impressions count, and ChatGPT’s first impressions were powerful:

“Wow, this thing can literally do anything!”

But this created a dangerous misconception…?

Because Gen AI precisely can’t do everything.?

It’s great for certain use cases and scenarios, but totally useless (and potentially even harmful) for others.

The reason is simple: Generative AI is fundamentally non-deterministic.?

Now, without getting too nerdy, here’s what this means:

Traditional software (like MS Word) runs on deterministic principles. If you press “A” on your keyboard, the letter “A” will appear in your document. No matter what.?

And if you keep hitting “A” 20 times, there will be 20 “A”’s in your document.?

I know, this sounds stupidly obvious.?

But with Generative AI, it’s different.

If you ask an LLM the same question multiple times, you will get different answers. Because it’s non-deterministic.

And this specific difference led to the dangerous misconception I’m talking about.

Gen AI is great for tasks that require creativity or variable outputs. But it’s unsuitable for tasks that require 100% accuracy and 100% repeatability.

In general terms, don’t use Gen AI for tasks requiring an accurate and predictable result. For example, reliably searching a document for references to risk factors like “indemnification” clauses would be wholly inappropriate for Gen AI.

Instead, use it for creative tasks, where there are multiple potential answers. For example, creating 1st draft content, re-phrasing content or suggesting ideas for a marketing campaign.

If you could take just one thing from reading this newsletter, let it be this:

Don’t use generative AI for situations where you need 100% accuracy, 100% repeatability and if you need to know the logic behind the result.

You should always start with the job to be done and then choose the right technical approach. Whether that ends up being Gen AI, deterministic models, or some mix of both.

This brings us neatly to another mistake I saw & in some cases continue to see…

You can’t throw Gen AI at everything and expect a miracle to happen

In the earlier stages of the cycle, we saw some companies teeing up a “Gen AI technology team” or research group without considering what they’re trying to automate. And not mapping back to the JTBD (Job to be done).?

At the end of the day, Gen AI is a tool to help automate certain tasks.

And like any technology, it must serve a clear purpose. Before implementing any automation, you must have total clarity on what you’re trying to accomplish.?

Just throwing Gen AI into the mix won’t solve every business problem. Instead, it will likely waste a lot of time and money that could have been better invested elsewhere. I hope I’m not jinxing it, but I think we’re mostly behind this phase of the cycle. ??

And that wasn’t the only issue we saw last year. Some enterprises didn’t factor the critical nuances that come with large-scale Gen AI implementation.

I saw instances where excited tech teams created a nice LLM prototype that was compelling enough to get people excited.?

A few weeks/months later, the first concerns started to surface:

- Scalability & performance requirements

- Uptime requirements,?

- Enterprise quality levels,

- How to deal with hallucinations,

- How to train staff to understand the basics of LLM prompting,

- etc., etc., etc.

Rolling your own Gen AI solution comes with lots of risk and hidden costs. And before you know it, you become a glorified product company – not a company invested in your business.

This is not at all specific to GenAI, for as long as I’ve been in software (over 30 years) it's been the same. I’ve seen these mistakes over and over. And fundamentally it opens up a “build vs. buy” discussion.

So, here’s what I commonly advise people when it comes to deciding whether or not Gen AI is a fit:

1. Start by understanding the problem and what you’re trying to automate.?

2. Then, give your team license to evaluate where efficiency and quality bottlenecks exist, and then map back to the underlying tech stack.

3. Once done, choose the appropriate technology. Whether that’s Gen AI or standard computing models.

Ultimately, this whole debate shouldn’t be about Gen AI.?

Instead, it should be about how we can automate processes and improve efficiency, regardless of the technology we use to do so.

Thanks for sticking this far. DM me if you have any thoughts on this article.

Best,

Fergal

PS - Follow me on LinkedIn where I share more insights like this.