It's more complicated on the inside than it is on the outside

We don’t need time machines to create paradoxes in technology: they are built into the way we work. One of these paradoxes is that the simpler technology appears on the outside, the more complicated it is on the inside.

I was reminded of this recently when talking to someone who confidently told me that the more sophisticated AI models get, the easier they will be to use, for technologists as well as end users. AI would solve its own skills problem. I was surprised by this because, to me (and, I expect to most other technologists), while we understand how natural language interfaces can radically simplify the experience for end users, the introduction of the current wave of AI into our architecture makes it more complicated. We have a new set of systems to integrate, a new set of suppliers in our supply chain, a new set of risks to manage and new patterns of cost and consumption to deal with. Not least, we have to work out how to make the deterministic parts of our architecture interact with the probabilistic parts of our architecture: prompt engineering is more like persuasion than programming.

And this experience is not new: it has recurred throughout the history of enterprise technology. Back at the dawn of my career, we built systems which were complex for users to navigate. The only interfaces we could offer were green screens and keyboards. This meant that users had to use numeric menus and short key codes, had to understand the meaning of cryptic field names, and had to understand the structure of the system. When PCs and GUIs arrived, we were able to create simple, colourful, flexible interfaces navigated with a mouse. But we also had to figure out the complexities of a client / server architecture, make choices about what belonged on the front end and what belonged on the back end, and learn the languages and conventions which applied at each layer of the stack.

To end users, today’s digital world may feel overwhelming: their phone is full of apps, they manage messages across multiple platforms, and they have to remember many forms of authentication, which always seem to be changing. But, compared to the range of functions it offers, it is remarkably simple. We control systems by poking and prodding, we download software from app stores without worrying about its provenance, and we will, with increasing frequency, tell our technology what we want it to do in natural language.

Behind the scenes, the level of complexity which makes this simplicity possible is similarly remarkable. Using a single app involves traversing many layers of abstraction, passing between many protocols, invoking software written in many languages, triggering many alerts and signals, invoking many security controls. Every layer of computing architecture which we have ever invented is still in there somewhere, and we have to manage them all.

Is this paradox a problem? Isn’t it simply the job of technologists to understand and manage this complexity, and to mask it from our end users? I think the paradox becomes a problem when it places a barrier of understanding between the people who build and run technology, and the people who lead the organisations on which that technology depends. Most of those leaders don’t have a technology background, and have an experience of technology which suggests that it gets simpler over time. They become confused and frustrated when they are told that everything is very complicated, that it is delicate and expensive, and that it’s essential to spend time and money on obscure parts of the architecture which they have never heard of. Isn’t it so much simpler now? Can’t we just buy that solution from the salesperson who visited last week? Can’t we get AI to do it?

At the same time, the simplicity of an interface can constrain the imagination of these leaders. If they see AI implemented with a simple chatbot interface, they may think of all the different places we could implement chatbots. But when technologists see a chatbot as an interface to an architecture containing a model which can understand language, they imagine the infinite array of ways in which we can put this power to work. Engaging with complexity is a path to possibility - and technologists get frustrated when these possibilities go unexplored.

I believe that we can address these problems by inviting our leaders into the heart of the paradox. One of the iconic moments of Doctor Who is when a new companion steps into the TARDIS and realises that it’s bigger on the inside than it is on the outside. (It also seems to be one of those moments the Doctor enjoys most.) Less iconic, but just as important, is when the companion realises that the TARDIS can take them anywhere, in time as well as space. We can recreate these moments if we are prepared to explain technology, with patience and respect, to people who don’t understand it - but whose success depends on it.

Those of us working in technology have our own TARDISes. From the outside, our architectures look as simple as an app or a blue box. From the inside, they are complex and multi-dimensional. They can be as much of a kludge and fudge as some versions of the TARDIS, or they may be clean and elegant. We often have to hit them with a metaphorical hammer to get them to work, and they make wheezing noises when they start up. But they can take us anywhere, and we owe it to our sponsors, leaders and stakeholders to show them how.

(Views in this article are my own.)