LLMs: Feature or Bug?

To begin with, I should probably say that LLMs, like many technologies, are intrinsically neither buggy nor functional. Instead, their apparent utility or lack thereof derives from how we apply it. This seems obvious, but the point seems to have eluded some otherwise bright people over the past year.

Let’s start with some of the buggy approaches. Creating a search method that ends up recommending that we eat a rock every day and glue pizza to cheese seems like a bad idea to me. Similarly, an LLM can ingest a suitable collection of tests and answer a state Bar exam’s questions correctly, but not understand how laws work, as some unfortunate attorneys have discovered. The fact that it correctly formats case-law citations for invented cases doesn’t help.

Such examples appear to be rooted in the fact that LLMs lack any version of the mental model of the world that we take for granted. To such a system, The Onion looks like an authoritative source of information, since it has a vast corpus of articles and is also widely cited online–including a humorous article about eating rocks. It’s our model of the world that lets us distinguish satire from fact so easily we don’t even notice it happening.

So let’s cheer up a bit, and look at where LLMs might be worthwhile features. One place where LLMs shine is in summarizing or re-stating any given body of text. Similarly, they can do a useful job in pulling topics from a large collection of articles, which can save a few steps if you have to create a presentation deck by tomorrow morning. Language manipulation is what they are designed for. Just remember to edit and double-check anything you plan to use, just as with human sources.

More generally, humans need to provide the model of the world that an LLM lacks. We can define the text corpus that it works from, for example. That way, it will not interpolate an authoritative-sounding vintage MAD Magazine satire on corporate writing into a business proposal–a bad idea unless you are actually selling “veeblefetzers.”?

Whether an LLM acts as a bug or as a feature is, in the end, determined by how you manage its use, prompts, and source data. Keep this approach in mind, and you probably won’t become the next hilarious AI trainwreck.