Chaos - When Humanity Meets Technology

Thank you for subscribing to Work Thoughts. I am an Enterprise Architect with almost two decades' experience in IT. I share my experience - both hard and soft skills - in fun ways through these articles. Share the fun. :)

I am probably going to write a book about this. But for now, I will keep it short. The article is about my foreboding, finding and fixing issues in a technology environments I have worked in. I should add here that while my newsletter is mostly written from the African perspective, the chaos that ensues when technology meets humanity is universal. You might argue with this if you have not read The Phoenix Project and sources.

Why is chaos the result of this relationship between machines and humans? I think one reason is our profound levels of sublime creativity. Machines tend to follow simple rules which the newbie must learn in order for harmony to be maintained but humans are a little different. We are creative, adventurous and have a tendency to want to be in control, or better put, feel in control. More like, "Type that command and hit the return key! It feels good!"

In the following sections, I offer three proposals that can help manage the potential chaotic relationship between humans and machines.

1. Systems Should be Designed with "Failure" in Mind

Anticipate Failure with a High Degree of Paranoia

One of the basic principles of building a new system is the need to anticipate failure. The whole hullaballoo about high availability, disaster recovery, fault tolerance and the likes is the need to make sure that service continues to be delivered when components fail. One common gap in this practice of preparing for fracas is that we are often focused on failure on the part of machines. While we must ask questions like: "What if this server goes down?" "What if this link fails?" "What if there is a DDoS attack?" "What if there a fire breaks out in the data centre?" "What if there is an earthquake in Accra?" which are all valid questions bordering on the reliability of technology, we must also ask questions about humans.

"What if Amewu gets drunk?" "What if Olawale resigns?" "What if Harrison's new wife annoys him?" "What if Tonie punches his boss in the office on Monday morning?" Outrageous questions you might say but that is because you assume humans are more reliable than technology. But you see, machines obey rules strictly, humans (typically) do not. Besides, one of my big bosses used to say (I think he still believes this) : "Be paranoid when building a system" (paraphrased).

Avoid Single Points of Failure from All Angles

The first set of "What if?" questions seek to address a single point of failure scenario among other things. One variant of this SPoF problem is something called a "Key Man Risk". This is a situation where only one person knows what to do in case of a severe catastrophe. We have to wait for that person no matter what. He could be in traffic on Third Mainland Bridge, on holiday at The Royal Senchi, or in the loo. But of course, having to wait for him makes him feel important so its fine. We have to preserve his or her self worth and value to the organization, right.

While single points of failure in machines are easy to spot, we often gloss over the KMR issue because there is more complexity when we have to take decisions that impact people. Maybe we don't want to hurt someone's feelings. We are scared they may get upset and resign. Or we just like them. Truth is, people can be as much single points of failure as machines. In fact, in many cases, humans may even be the SCoF - Single Cause of Failure.

Don't Always Count on Outsourced Risk

In dealing with these challenges, we often resort to playing the political card - another complexity created by our humanity. Simply declare all your IT resources incompetent and outsource operations. That way, when there is an issue, you can simply tell the board the vendor is working on it. If the big boys (and girls) get too upset, change the vendor! The vendor may be a managed service provider or a cloud services provider. Outsourced risk helps but there could be a better step that or a little more robust - building a culture of simply following rules.

2. Architecture Should be Described as A Published Set of Artefacts

Problem Resolution, Impact Analysis

Using a variety of artefacts to document a system means we have captured and codified everything we intend to build. Publishing this design means it is available for use in troubleshooting when things go wrong and in impact analysis when we need to do something else. It also means we do not have to rely on a single person stuck in traffic to give us information about the possible causes of a failure or the potential impact of a change. We simply look at the documentation as long as it is accurate and accessible (hint: democratization of data, right?)

When the Chips are Down

If documented architecture is going to be much value in the scenarios mentioned above, we must take time to ensure the documented architecture is actually the implemented architecture - a design process that captures all aspects. We must then validate the build, comparing it with the architecture - design assurance. These activities are often time consuming and boring. Most techies are busy putting out fires from the last build. They simply don't have time to "write" and "draw", there are things to fix! But when the chips are down, we tend to ask the same set of questions: "How does this connect to that?" "Which port is this listening on?" "Do we have DR?" "Who is managing this system?" and so forth.

What About the New Design Patterns?

The advent of cloud native architecture patterns brings a new paradigm to the fore. Resilience is better as long as it is gotten right and the application likes it. But the truth is that quite a few organizations particularly in Africa still rely on on-premises infrastructure and layered architecture. So while we are still on the "everybody to the cloud" carnival, we have to embrace a new culture - following processes we agreed on ab initio. This culture will still be relevant even when we move to outer space.

3. This Chaotic Relationship Should be Balanced by Process

Machines follow rules. Humans should follow rules too if the relationship is expected to be less chaotic. The word "process" is the third element in what is called the PPT Framework or the Golden Triangle (I like the sound of the "Technology Value Triad").

Code tells machines what rules to follow. Policies, processes and procedures tell people what rules to follow. Trust me, machines are more obedient than humans. It may be good thing - humans are more creative, able to take decisions by themselves etc. But often, stability is facilitated by simplicity. Simply follow the rules.

Processes define how ideas are conceived, designed, deployed and validated in a corporate enterprise. The end game is ensuring there is more harmony in the relationship between humans and technology.

Conclusion

People, Process and Technology (PPT). "Process" sits in between as stated, balancing the relationships, improving stability. If we all follow the same set of rules, then we all know what to do. If the rules are not working, then we sit at the round table and modify the rules rather than simply break them. If we need new rules, then we follow the rules about how to make new rules. People, Process, Technology.

How is this balance working out in your organization?