Making Sense of LLMs - A picture worth 1,000 neurons.

This is the fourth installment in a series about LLMs. You can find the third article here:?Making Sense of LLMs - Data gets to be the new oil again!

When I first learned how to drive a car, I took my relatives for rides around town so I could get more experience behind the steering wheel. Pulling up to my uncle's driveway one day, he told me: "Lukas, you drove well. But, do you want to know how I know you are still a beginner?" -- "It stopped raining ten minutes ago, but the wipers are still on."

Steering the car through traffic had consumed all my attention, and it took me a couple more months until the interface of vehicles became second nature. Some interfaces never get to the level where the interaction is effortless, and intentions manifest without effort.

The way we interact with computers today can be traced back to the late sixties. 1968, Doug Engelbart of the Stanford Research Institute performed what is now known as the "Mother of All Demos" — showcasing the computer mouse, text that can be linked, graphics-based navigation, and more for the first time. More than half a century later, these interfaces still persist.

We intuitively understand that interfaces are hugely important.?How we touch, sense, and interact with our devices, respectively how they respond to us, creates a distinct feel and can be a strong source of differentiation for brands. Because of it, companies design everything from the?sound of a car door?to the way in which an?emoji wiggles onto the screen.

Marshall McLuhan's theory of "the medium is the message" goes even further, stating that the medium and its interface shape the content itself. And some messages would feel off if sent as a text message instead of an email, or vice versa.

LLMs are still brand new on the timescale of user interfaces, and the industry has defaulted to the most basic digital interface imaginable, the command line. The most advanced artificial technology is mediated through archaic text prompts, chiseled into bits, and executed by hitting a command button.

Soon, developers and designers will find more engaging ways for us to interact with LLMs. A highly compelling example of how radically different the future could look like was recently given by two Japanese researchers. Their paper 'High-resolution image reconstruction with latent diffusion models from human brain activity' describes how an LLM successfully reconstructed the images presented to subjects based on fMRI scans. In other words, brain scans were turned into pictures of what people saw. If this does not excite and scare you at the same time, then read the last sentence again.

We are still at the beginning of this technical revolution, and it is already wildly exciting to see the impact LLMs have. Lasting change will come about as this technology subsides from our attention and the interface fades into the background of our mental picture of the world. Just like a steering wheel after enough leisurely drives around town.

You can find the next article here:?Making Sense of LLMs - A goal without alignment is just a wish.