The beauty and danger of simplicity

We all love simplicity. The ease of remembering, the way it makes us feel smart, the safety of having the answer. But not all simplicity is created equal.

At one end of simplicity is the naive, magical place of not knowing what you don't know. A place where things will be easy because you want them to be, not because you actually know what you're doing. In this zone lives the Dunning-Kruger effect. A fascinating cognitive bias that leads us to overestimate our abilities when we know very little about something.

Naive simplicity is dangerous. When we know so little that we think there's not much to know, a lot can go wrong. The new manager who thinks that people will do what their told because 'I'm the manager now'. The junior developer who thinks they can fix that bug and deploy it without a code review because 'it's a simple one', and crashes the system. The new product manager who thinks that focussing on estimation will mean work becomes more predictable.

At the other end of the curve lies elegant simplicity. The place where you can be unconsciously competent. Where things feel easy and your results are excellent. This is the zone of wisdom. The place where you've worked through all the complexity, and distilled things down to a few key points.

Elegant simplicity is like magic. When we know so much about something, and have such deep competence, we can get results with apparently minimal effort. It's evident in Shakespeare's writing that was so good we are still quoting him 400 years later ("too much of a good thing", "seen better days", "break the ice" and many more). In Darwin's theory of evolution that explains so much. And in Francis Bacon's articulation of the scientific method.

The trick of course is that elegant simplicity is not magic. Its simplicity is borne out of deep experience and critical thinking. It comes from going through the complexity and coming out the other side; having distilled the most critical components and how they fit together.

Elegant simplicity can allow others to 'understand' something complex without going through the complexity themselves. But this 'understanding' is an illusion. Being able to describe key principles, or use the common phrases of a discipline does not mean that true understanding lies beneath.

And therein lies the danger of elegant simplicity - it can allow a non-expert to convince themselves and others that they are an expert. And it can hide the amount of work required to get to real mastery.

In business this is perhaps the more insidious danger. Naive simplicity will be found out quickly - a few deeper questions, or a small project, and the (lack of) results will speak for themselves. Mimicry of mastery through borrowing from elegant simplicity is harder to discern. A mimic may be able to talk the talk, and even get decent results in simple projects. Their mimicry will enhance their performance (to a point). That's actually part of the value of elegant simplicity - it allows fast access to better understanding.

But mimicry will fail in all but the simplest of situations.

The true value of elegant simplicity is that the key elements carry a wealth of complexity. it makes complex topics easier to discuss, and keeps the focus on the most important elements. But without a strong base of practice & experience to sit on, elegant simplicity can lose its meaning and start to resemble naive simplicity.

This problem is a big one for startups. How do you tell if someone knows what they are talking about if you're not an expert yourself? How do you know whether to hire this person or that one? Or whether to rely on the advice of the people around you? When should you invest in expert advice, and when is it safe enough to wing it?

One way to solve this problem is to rely on others' perspectives to determine expertise. This is what referee checks are supposed to be doing - you ask a person who should be in a position to know, and then you take their advice. This is also what we are using when we rely on someone's reputation, or markers of status to determine their level of expertise ("he worked at Google, he must be good").

Unfortunately, relying on others' perspectives assumes that they know what they are talking about. Which of course leads us back to our problem of not being able to determine expertise. How can I trust someone's perspective when I don't know if they know what they are talking about?

A better way to solve this problem is to look first at the area someone claims to be an expert in, and assess how rare real expertise is in that domain. Once you have an idea of how rare expertise is, you can more accurately determine how careful you need to be in your checking process.

Some domains are much easier to gain mastery of. That's because some domains naturally have better feedback available to learn from. Mastery of anything requires disciplined practice at getting it right. In order to be able to practice getting it right, you need to be able to determine IF you are getting it right. Therefore, domains that have better feedback available will be able to produce more masters.

For example, sports tend to have excellent feedback available. If I'm trying to learn to shoot hoops in basketball I don't need anyone to tell me if I got it in. I can simply throw the ball at the hoop and see if it goes in. Certainly I will make faster progress if I can learn from others what kinds of things tend to make the ball go in the hoop reliably. But I can always tell when I did it right.

On the other hand, leadership tends to have terrible feedback available. If I'm trying to learn how to lead, it's hard to even figure out what 'right' looks like. Even at a more micro level, the feedback tends to be delayed, noisy and difficult to interpret. Are the goals I've created 'good'? Is the feedback I've provided 'helpful'? Is my team 'engaged'? For any new leader one of the biggest challenges is getting useful feedback. In fact for any leader, no matter how experienced, getting useful feedback is still a challenge.

Domains where accurate, objective and timely feedback is available are easier to learn. These domains tend to produce more masters. These are also generally domains where the problems have been reduced to the level of complicated, rather than complex. Most of the traditional trades sit in this space - your average plumber is able to fix most household plumbing issues. The problems that occur are certainly real, and take skill to fix, but the feedback is relatively good. The tap is no longer leaking, the toilet works, the gutters flow properly when it rains.

Domains where feedback is imprecise, subjective and delayed are hardest to learn. These domains produce very few masters. They are also the domains where problems are the most complex, or even wicked. Politics is a classic example. What makes a politician 'good'? Most people's day to day definition of good politics would be something like "I agree with most of what they say" or "I like them".

The closest politicians get to objective feedback is election results, which are infrequent, extremely imprecise, and barely connected to the results of any espoused policies. Imagine if basketball games were played with no hoops, and the winner was determined by a vote of the crowd after 60mins of play & a speech from each team about how fabulous they were. It'd be pretty hard to tell whether the best team won, or even what 'best team' would even mean.

If the domain is more complex and the feedback trickier, then determining expertise will be harder, and you should expect that there are many mimics and few experts. In these domains, one of the best methods for determining expertise is actually to ask the supposed expert about the feedback they use to determine success.

A true expert will be aware of the fact that their feedback is noisy, and they will have methods for trying to distil a useful signal from out of the noise. Any answer they provide about understanding 'what worked' should acknowledge the complexity they are dealing with, and have methods for seeking a useful signal anyway. They should be able to swing easily from describing the principles they are using, to describing examples of putting this into practice. They will also usually be able to tell you about how they determine expertise in someone else. In short, an expert is much more likely to know what they don't know, and what is unknowable.