From story to science

One of the persistently recurring themes of my reading and experience has been that there is a continuum of knowledge and behavior that spans the way from cowboy to checklists. Cowboys being any type of activity that there are no rigid guidelines for (yet) and that only people who have 'the knack' for it can succeed at. Checklists being an activity which, through trial and error, we have found a correct way to do and that requires specific steps and operates independent of personal ability or attributes. I originally got this distinction from Atul Gawande's book The Checklist Manifesto but I've seen related ideas elsewhere (clocks and clouds, deterministic systems vs stochastic ones, nascent sciences versus mature ones).

As I imagine it, this relationship between behavior when we have low information and behavior when we have high information, also describes a continua of bodies of knowledge and attendant behaviors. That is to say that not only are there cowboys and checklists but there are rodeos and bureaucracies. I think that we see these models all the time, everywhere, and that we know which one is which very early on when seeing it. We know when a business or organization doesn't have things together, when things are being held up with duct tape and bubble gum, and we know when we're dealing with a well-oiled, stainless steel T-1000 machine.

I think often about the transition point between one state to another. As most people that have any professional or personal interaction with systems relating to safety (of any kind) know, all regulations are written in blood. For those unfamiliar with that saying, it means that every time something goes bad, someone writes a new rule to prevent that situation from happening again. Many of us will look on with puzzlement as we read and evaluate rules that appear to be written for children, such as "Do not stick finger inside of meat grinder while it is on" but those of us with experience realize that things like that are never written for no reason. Someone, at some time, somewhere, stuck their finger in the meat grinder to fish out their wedding ring (extreme benefit of the doubt) and were forevermore known as "4-finger Todd".

I'm going to talk a bit about the lifecycle of bodies of practice and knowledge. First, let's talk about the far poles.

Why Story?

Someone may be asking themselves why story represents the low information end of the pole of bodies of practice and knowledge. Simply put, stories are the basic unit of human information for anything more complex than a plain fact. Anything that involves any sort of interaction, any dynamics, will involve a story.

Here's an example. I can tell you a rock is hard. That's not a story. That's a flat fact. There's no dynamism there. You don't expect the rock to change. It's a rock and it's hard and there's nothing more to understand there. However, if I wanted to convey to you that sometimes this rock is hard and sometimes it's soft, you would almost immediately demand a story. I would then have to tell you that "Well, when I went to sit on it in the morning it was hard but I went back in the afternoon and I practically fell over because it was too soft to support my weight".

We use stories to convey complex bundles of data that do not fit neatly into flat facts. I'm also going to point out that the paragraph you just finished reading is also a story. Anytime someone gives you an example, they're telling a story because the information they're attempting to convey has too many dimensions for a simple and clear and easily understandable statement.

In brief, storytelling is our best (evolutionary) solution to the communication trilemma. You can be brief and simple, simple and complete or complete and brief but never all 3 at the same time. Storytelling is our way of breaking out of that by taking turns (more on that in another article)

Why Science?

Science, in my mind, is our best attempt to declare the rules that reality runs by, unlocking the ability to do things that would not ordinarily occur in nature. Nature has never made a computer by itself. It's made brains, which are much better than computers right now, but never a computer. Brains need food. Brains need air. Computers need electricity. Totally different thing and only happened because humans took the process and practice of creating devices of this type and brought it up to the level of a science. Rule-based, consistent, repeatable, very little performance deviance.

How do we transition?

There's an abstract idea that Temple Grandin touched on when she said the following.

"The first step is you have to realize different thinking exists. I've done a lot of talks to corporations and they ask me what the first step is. The first step is you have to realize different kinds of thinking exist. That's your first step."

Basically, you've already done the first step because you realize this continuum exists (by reading this article or on your own) and now you just have to figure out the way from one end to the other (Mapping the Maze writ large). So what's the second step?

Like an athlete's lucky underwear, stories allow us to boost our performance without having to think too hard about the real reasons for those performance gains. The second step, the first active step, is to notice what the drivers of performance are in a given situation. The ideal way to do this is to aggressively investigate and document everything until you figure out what is important. I'll give you an example. <- See? You went right into complex information ingestion mode on that cue, didn't you?

A few years ago a science lab was working on attempting to replicate a relatively simple experiment. The researchers were reaching a point of frustration because they were following the methodology listed in the paper exactly and yet getting wildly divergent results. So, one researcher gets so frustrated that he calls the principle on the original paper to ask some questions. As it turns out, while the paper was specific about the methodology, it was actually not specific enough. It said they stirred some chemicals but it didn't say how they stirred it (manually, magnetic stirrers), how vigorously, or for how long. That coupled with a couple of other minor methodological differences accounted for the entirety of the failure to replicate.

In other words, the view that they had on the data that they needed was not precise or comprehensive enough to be useful. Read that line as many times as you need to because that is the point of this article and practice.

Well-posed problems

In math, there's the concept of a well-posed problem. When porting it over to other disciplines, it sounds a lot like "the problem has been defined at a sufficient level of detail and accuracy that the solution is singular and self-evident". Well-posedness is a concept I wrestle with a lot personally when it comes to task analysis, especially for tasks I have a low familiarity level with.

The game of isolating variables and creating systems that guarantee a high level of performance with a high level of consistency is one of triangulating on well-posed problems. This is actually a game I play quite a bit in my professional life, so I'll share some vignettes with you to illustrate my point <- Different language but, did you notice this time or did the cue work like it normally does?

Someone calls my office and tells me "the system is broken". By the way, I have effectively come to take that phrase to mean "I have no idea what to say to you but I need your help" in about 99 out of every 100 utterances. Anyway, someone says that to me and my kneejerk response is to ask some variation on "What do you mean?" However, over many trials, I've discovered that asking "What do you mean?" to someone that has effectively already signaled to me that they have no idea what to say is not helpful. The specific phrasing I use (coordinating on a well-posed problem here using specificity) is usually something more like "What exactly are you trying to do and what are you seeing on the screen?" This allows them to describe their end goal, which there will usually only be a couple of paths to arrive at, and also it allows them to describe their obstacle, which may be a mole hill they're making out to be a mountain.

In that story, I take the user from vague "Things are bad, man!" to specific "I'm having this difficulty in the course of attempting to navigate these specific set of steps." Do you see how much easier it is to address the latter than the former? I use this exact same technique when doing what my father would describe as making the impossible merely improbable. 1 thing I want you to notice about that story is that language use mattered there. This is one thing that I have to emphasize or else it may be missed. The words you use to describe the problem matter because the concepts you use internally to represent the problem are the key to solving it. The words are just a proxy but they're the best proxy we have, so you must be careful in their use.

Surfaces and Essences

In a book (that I highly recommend) called Surfaces and Essences, Douglas Hofstadter takes a close look at the relationship between symbols and meaning and examines how we use analogies to help us think. I enjoyed reading the book so much I created a card game based on it called Scatterbrain (shameless plug). Anyway, this seems like an appropriate moment to wheel out some of these concepts now.

Ordinarily, I prefer to think of labels and objects separately. That is to say that when I think about the world, I see what a thing is and I see the symbolic representation that is associated with a thing as separate entities. To put this in perspective, let me ask you a question. How many apples are in this picture?

The correct answer is none. Why? Because an image of an apple is not an apple. The word "apple" is not an apple. The mouth-sounds I make that you associate with the string of letters "apple" isn't an apple. Only an apple is an apple. Everything else is a symbolic label, whether that label be in the recreation of its image, in symbols we associate with it, or with other sensory impressions that remind us of it.

The case of the well-posed problem and coordinating on it is a special case for me, though. I abandon my usual separation of ideas from the signs and symbols we use to describe and manipulate them and treat the words as the ideas. In this case, how you say it almost matters more than what you say. I'll give you an example.

I've always hated orange juice with pulp. It's the consistency. It drives me up the wall. One day in my early 30's, I decided that I didn't want to be the kind of person who would be uncomfortable drinking orange juice with pulp anymore. It was childish and I wanted to be rid of it. So, I used a trick that I learned (from some book no doubt). I reframed the concept of orange juice pulp to make it not disgusting for me. I've always hated orange juice pulp but I've always loved applesauce. I had to ask myself "What is really the difference between these 2 things?" From that moment, I considered the pulp to be "orange sauce". I've never had any issues with it since.

Using a specific term is a part of the secret sauce in this method. Reframing the problem involves using manipulation of the concepts, which sometimes involves working with the words we associate with them. You see this a lot in positive psychology and self-help circles, and the reason you see it is because it actually works (or so my therapist tells me).

This idea is most often leveraged in the form of jargon. People create symbolic shorthand for their findings in a space and it allows them to construct elaborate systems with relatively low cognitive overhead .

Actionable insights

Aside from coordinating on the reality you're dealing with in any realm of observation and in leveraging language to that end, the rest of the work is connecting the dots. It's figuring out what levers to pull to make the widgets come out the way you want them to. What this translates to in practical terms is running experiments, first to confirm (or disconfirm) your understanding, and next to find out how much you can manipulate the outcome using the methods (/tools) you've built (/discovered).

More on Sensors and Handles in another article.