Order out of Chaos – How to create processes that help, not hinder

Order out of Chaos – How to create processes that help, not hinder – D M Goldstein, Sept 2022

In the beginning

Let’s discuss a hypothetical start-up company. You have Product people designing the Next Great Thing, Engineers building it, Salespeople getting you your first customers, and Support and Services teams at the ready. If those early adopter customers have any problems, they might talk to Support, or Services, or Sales, or even directly to an Engineer. Customers calling Engineers directly? Not a problem! “We’re small, fast, agile, and customer-driven.”

But as you grow you start to notice there is a lot of chaos going on, with too many independent lines of communication and too little coordination. Mathematically, every time you add one more person you add “n minus 1” possible lines of communication.

• 2 people = 1 line of communication
• 3 people = 3 lines of communication
• 4 people = 6 lines of communication

It might not sound like much, but at 100 people you technically have 4950 possible lines of communication. That’s an awful lot of noise and interruption. How do you get order out of that chaos? Through building processes that control and prioritize communication.

Note that communication is not the only situation that can benefit from process; it’s just a common-enough situation so it makes for a convenient example. We’ll get back to this later in this article. But first…

Traffic Control

Analogy time! Let’s consider the intersection of two streets of equal size and volume. If they are low-volume residential side streets, you might be able to get away with having no traffic controls, but you are going to have accidents. Your town considers putting up Yield signs. That might slow some cars down and reduce the frequency of accidents, but it’s not really a foolproof control. The town decides to upgrade to Stop signs. Do you put them on either side of just one of the streets, leaving the other one unimpeded? Or do you go four-way (one sign on each corner)?

Nearby is the intersection of two multi-lane boulevards, with shopping centers on each of the four corners. Stop signs could work there, but it seems intuitively obvious that traffic lights would make a lot more sense and get cars through safer and quicker. It also seems intuitively obvious that you would not use traffic lights on the residential intersection; there is not enough traffic there, so the lights would unnecessarily slow drivers down. These variations on traffic controls are like the processes you would consider for the communication problem, or any other problem. Too little and you have chaos; too much and you are imposing unnecessary limits.

Gilroy, California

Two two-lane highways meet in Gilroy, California. Highway 152 is a mostly East/West corridor which many people use to get between the coastal cities and inland cities, and to access major North/South routes like Interstate 5 or Highway 99 instead of coastal Route 1 or Highway 101. Highway 156 has its northern end at 152. Many years ago, a problem would occur when people on 152 West wanted to exit on 156 South. That intersection was like going up the base of a Y and choosing to go up the left side of the fork, crossing in front of the traffic coming down the right side of the fork. Like the fictional town in my Traffic Control example, they chose to put a Yield sign for those folks. That worked well-enough during light traffic times.

Until Thanksgiving. With no real traffic controls and no police presence to keep traffic moving, drivers decided to treat the intersection like a set of Stop signs. One car from 152 East would continue on 152 East, then one car from 152 West would turn onto 156 South. One at a time, with all other cars stopped and waiting. One Thanksgiving it took me four hours to drive the 12 miles East from Highway 101 and get through that intersection. That’s what inefficient processes do to your business. Note that they eventually built a “flyover” bridge, so that problem is gone, and cars can safely and quickly get through that interchange; a good process can provide that kind of improvement by eliminating bottlenecks.

Back to the Process stuff

What makes a “good process”? A good process typically is consistent, reproducible, measurable, and efficient. It must address a problem and resolve it. “Process for Process’ sake” is a dangerous thing. Imagine a teacher telling an elementary school class to, “line up in height order so we can march to the restrooms.” What value does height order provide? That’s an unnecessary process, it adds no value, and there is no apparent problem being solved. If instead the teacher said, “Everybody who needs to use the restroom, please line up at the door,” that would make sense; it identifies those who need the restroom and provides a mechanism for controlling how many go at a time. When I was a kid, we had fire drills where the students would march onto the tar playground and stand in assigned lines. One time it was so hot that kids started fainting from the heat. What purpose was served by having us stand in line for the whole drill? Once we were all accounted for, why not let us sit down or seek shade? The teachers or the administration did not understand what problem they were trying to solve with the process they designed.

Next, the various parties with roles and responsibilities in the process should be involved in its design. If I’m designing a process for the handoff of something between two teams, both teams need to participate in its design. Each must make sure their requirements are met and that they all understand each other’s objectives. Think about the playbook for a professional sports team; every player knows what is expected of them, and what the other players are expected to do.

A good process must include room for exceptions. You don’t want to be in a situation where the process requires that, “Billy must approve all requests.” What if Billy is unavailable for an extended period? There must be a clause for what happens in Billy’s absence, or a means for Billy or somebody else to delegate that approval. Things that are too rigid tend to break, and people who feel blocked by an overly restrictive process will tend to find ways around it, introducing that chaos again. This applies to my fire drill example above; due to the heat, an exception should have been made to remove the “stand in line” requirement.

I have yet to meet a process that couldn’t benefit from Continuous Process Improvement. That implies several things: a mechanism for measuring the process’ success; a “process improvement process” to review and improve processes; and, if the process is mandated by law or governed by some external authority, there must be a means to be audited and approved if necessary.

5000 Lines of Communication

Let’s return to that 100-person startup and their 4950 possible lines of communication. A customer posts an issue that is seen by individual contributors and managers in each of Support, Services, Sales, Product, and Engineering, for a total of at least ten people. That’s still 45 possible lines of communication. With the six people in the first three groups each only reaching out to the four people in Product and Engineering, that’s still 24 lines of communication. And three of the four in Product and Engineering are each reaching out to the single Engineer working on the problem, who now has the nine others constantly interrupting the actual work to ask for status that they can pass back to the customer, with messaging lacking coordination and consistency. I sense an opportunity here to build a process.

What if we required that all customer issues route through Support? Customers would be directed to report issues to Support and not to whatever method got the attention of the ten people above. And, if any of those ten got a direct customer report, they would advise the customer and pass it to Support. Support, as the nexus, would be the only one discussing status with the Engineer, and then broadcasting status back out to the customer and any other interested parties through a tool (CRM) that everybody has access to. The Engineer gets to focus on the work, and the rest of the world gets consistent information. Now think about the benefit when we apply this “at scale” with a lot of customers, a lot of issues, and a lot of people across those five (plus) teams. We just built the equivalent of that flyover bridge at 152 & 156 to replace the bottleneck.

Beyond just communication

Lines of communication was a comparatively easy problem. Let’s continue the customer case scenario but make it a bit more complicated and requiring an Escalation Process. Support is working on an issue with a software product, and determines they need to escalate to Engineering, as it presents as a bug. They submit an entry in the bug tracking system saying, “The Blue button doesn’t work.” Is that sufficient? Well, it depends on what the “Blue button” is supposed to do. If it controls some binary thing, like switching between two views, that description might be sufficient, but probably not. If the “Blue button” does something more complex, like translating between languages, then more information is certainly warranted.

When Engineering is looking at an issue, you want to keep their attention and use their time productively. Forcing them to ask for clarification is counterproductive at best. Here is another opportunity for a well-written process to make the job more efficient. Perhaps Support and Engineering can agree on a minimum amount of information that should be on every bug report: Software version, environment/platform (PC, Mac, etc.), date & time of problem, any log information that can be collected, any screen shots of the problem happening, customer name, the last time it worked as expected (if at all), and any recent changes that were made. That kind of information can help, even in the “switching between two views” scenario. Additional information might still be required for the translation issue, such as source and destination languages and text.

Now we have an escalation process that reduces the time the Engineer spends playing “20 Questions” with Support, allowing them to work on the issue. Earlier, I asserted that, “A good process must include room for exceptions.” What if Support is unable to collect some of the “required information”? Should the Engineer reject the bug outright until the information is available? In some cases, that might be so, if the missing information is critical to understanding or reproducing the problem. If that’s so, when Support provides the missing information, no time should be lost in resuming Engineering’s investigation – don’t make the issue wait in the queue multiple times. But sometimes the missing required information does not materially impede the investigation; in that situation, the process must be flexible enough to have the Engineer start working despite the missing information.

The Factory Floor

In many scenarios a good process functions like a factory floor. Things flow smoothly from one workstation to the next, with appropriate planning for time and materials at each stop. On some factory floors there is a means to stop the production line when something anomalous occurs, such as pulling on a chain or pressing a big red button. That is an exception process, but it can be an expensive one if it stops the entire line. In your processes, think about how you can “pull the chain” without stopping all other work. I was at a candy factory where the tooling automatically sorted out the candies that were not the standard size; no need to stop production, and even the rejected candies had a future use.

Order out of Chaos

I like to use the phrase, “Order out of Chaos,” to describe the application of well-designed processes. I am not a hard-core time-and-motion scientist like Frank and Lillian Gilbreth (“Cheaper by the Dozen” was based on them), but I abhor bad processes. A good process can gain you massive efficiencies in your workplace.

To recap, a good process is consistent, reproducible, measurable, and efficient, and is designed with the following guidelines:

• It must address a problem and resolve it
• The various parties with roles and responsibilities in the process should be involved in its design
• A good process must include room for exceptions
• You should practice Continuous Process Improvement