Use cases for Gen AI

Remember this diagram for blockchain use cases? Perhaps it's time for something similar for Generative AI use cases. There has understandably been much buzz and excitement about the amazing capabilities of ChatGPT this past year, but now that the dust has settled we need to assess realistically where it can help.

The key thing to remember about generative AI is that the answers it gives you are not necessarily correct, and are not even intended to be correct. You need to ask questions such as:

• Does it matter if this answer is wrong?
• Do I require the same answer to the same question if I ask it twice?

This instantly rules out a LOT of business use cases... Working in IT, I'm more interested in the use cases that have been put forward for enhancing the software development lifecycle so I'll talk through some of those.

I think there are great and valid use cases, in UX, for example, because here we are putting suggestions forward to the system user for their consideration. There's no real right or wrong answer, so correctness isn't an issue; and the user can veto anything odd. Already, design tools such as Figma have started to put this into practice.

Same in testing - I was recently reminded of this great testing joke:

A tester walks into a bar and orders a beer.

Then he orders null beers.

Then he orders 99999 beers.

Then he orders a bear.

Then he orders ytugihoij^%&*( beers.

A customer walks into a bar and asks where the toilets are. The bar explodes.

Turning up the "temperature" on ChatGPT when generating test cases, and getting some rather random test cases generated, might not necessarily be a bad thing.

What about coding? With applications such as Github Copilot getting lots of attention, this would appear to be a central focus for Gen AI. However, it's not quite panning out how some business owners have intended - you can't arm a junior developer with a Copilot license and expect them to perform like a senior developer. Why is this? A few reasons:

It only saves time spent writing code

Developers spend surprisingly small amounts of time actually writing code. Github proudly announced huge productivity gains when using Copilot, but saving 55% of the 10% of time you actually spend coding isn't as impressive - plus, most of the examples I have seen of using Copilot focus on reducing boilerplate and that's something few senior devs would write anyway. It's just as quick to copy-paste from your favourite tutorials / StackExchange posts!

It's most useful for the people who don't want it

The above point brings me on to who will find it most useful. Senior developers are less likely to use it to generate much code, as they have already found ways to avoid boilerplate, and the difficult work of fine-tuning or bug-fixing a code base is less likely to bring useful suggestions from Copilot. With one exception - how excited was I to discover that ChatGPT can write regular expressions! And Copilot will implement them for you without even being told that a regular expression is the right solution to a given problem! ??????

However, outside of regular expressions, it is junior developers who will be most likely to take up Copilot et al and use it them a crutch. And here we have to go back to our initial question: Does it matter if the code is wrong? Sometimes, yes - it does.

It's only as good as the average of the code it learns from

By training Copilot on all open source code bases, we've literally created the world's most average programmer. Which isn't what you really want for your own code base! And indeed, reports are now proving that a code base maintained with generative AI degenerates in quality over time.

It's just another programming language to learn

Copilot will do all sorts of things for you, if you ask it properly. In that way, learning Copilot language prompts is a bit like learning a programming language! You have to use the right format in the right order to get what you want.

In conclusion, there are fewer use cases than one might think for generative AI, and whilst Copilot will be a great tool for developers and will probably be as game-changing as IDEs were in the 1990s, it isn't quite the bomb people expected - developers, don't go hanging up your boots just yet.