Adventures in ignorance

Nobody knows anything.

That’s the number one rule in Adventures in the Screen Trade, the book by the late screenwriter and author William Goldman (Butch Cassidy and the Sundance Kid, Marathon Man, All the President’s Men and The Princess Bride, amongst many others). Goldman’s insight was that, even though the film industry was getting on for a century old when he was writing in 1996, no-one had been able to figure out what would be a hit and what would be a flop. Nearly thirty years later, it appears that it’s still true that nobody knows anything: despite algorithmic targeting, despite new ways of reaching audiences and measuring their reactions, it is still possible for juggernaut franchises to stutter and stumble, while unknown newcomers charm and delight.

Enterprise technology is rather less glamorous than the film industry (although it involves far more money), and is about half a century younger: we got our start in the1940s rather than the 1890s. However, I think that we would do well to remember Goldman’s rule: nobody knows anything. This is not just because we have our own share of flops and hits, and struggle to predict which will be which, but because we have our own special nexus of ignorance.

Consider the initiation of a major technology transformation programme. We are setting out to implement new systems which will automate, digitise and improve the performance of an important part of our organisation. We have a sponsor whose success will be measured through business outcomes. We have a technology product owner and a team of technical specialists. We have some project delivery professionals and a PMO. We have an ecosystem of product and build partners who are going to provide capabilities and capacities which we lack. We are all set!

Except that none of these people understand each other. The technology people aren’t experts in the business domain: even if they have worked for the organisation for years, they have never done quite the same job as their colleagues. The business practitioners aren’t technology experts: even if they have worked on many projects, they don’t write code or run systems for a living. The project delivery people understand their methods - but they aren’t true technologists or true business people. And the partners have only just walked through the door.

Furthermore, all of these people are likely to be suffering from an extreme case of the Dunning-Kruger effect: the phenomenon that people who lack expertise in a field overestimate their level of understanding. I don’t understand your business? How hard can it be? I don’t understand the software development lifecycle? How hard can it be? I’ve never worked with you as a client before? How different can you be from all of the other clients? It’s easy to have reassuring kick-off workshops where everyone convinces themselves that they have formed a team, developed empathy and achieved mutual understanding, even though the actual result is false confidence (I know: I have participated in and occasionally run such workshops).

And, even if through some miracle of comprehension and self-awareness, the team really does achieve mutual understanding and respect at the beginning of the programme, it still faces the greatest ignorance of all: ignorance about the future. Anyone who has worked on a major delivery programme knows that plans are always derailed by reality: we just don’t know exactly how they will be derailed. And yet we continue to make plans as if, this time, it will be different.

Of course, none of this means that we should give up. Despite his realisation that nobody knows anything, Goldman went on writing screenplays and books, some of them successful, some of them less so. In an illustration of his rule, The Princess Bride, one of Goldman’s works regarded with special affection today, was a poorly marketed flop on release, and took years to find its audience. Goldman was not counseling despair, but humility, realism and healthy scepticism about all the frantic activity in the film industry intended to provide certainty and predictability.

Fortunately, in the technology industry, we have our own set of tools to tackle our ignorance. We don’t have to release our products in one go and wait for the box office returns or streaming stats to tell us whether they work or not. We can conduct user research, organise our work into sprints, release frequently and test continuously. These tools are most powerful when we approach them with humility, and remember that they are not just rituals, but exist to help us navigate through our nexus of ignorance and discover what works, what doesn’t work and what matters.?

Just like in Goldman’s book, we start in the dark, where nobody knows anything. Fortunately, we don’t have to stay there.

(Views in this article are my own.)