Do Governance Like a Shepherd – Speed and Stability with Automated Governance

Introduction

In today's rapidly evolving digital landscape, maintaining a balance between governance and innovation is crucial for large companies. Without clear governance guidelines, digital teams can quickly descend into chaos, jeopardizing the entire organization. However, achieving robust governance does not mean sacrificing innovation speed. This article explores how Automated Governance, exemplified by the metaphor "Do Governance Like a Shepherd", can provide a scalable and efficient governance model that enhances productivity and creativity without compromising safety or stability.

The Shepherd Metaphor illustrates how to do Governance in large, autonomous systems effectively. Imagine you are responsible for several herds of sheep: your digital teams. Constantly interrupting them with cumbersome governance processes hinders productivity. Instead, by implementing Automated Governance, akin to a well-trained sheepdog, you can ensure rules are followed seamlessly without stifling innovation.

TLDR

Automated Governance, inspired by how shepherding works, allows companies to maintain robust governance while preserving innovation speed. By using a metaphorical sheepdog to ensure compliance and security, organizations can prevent chaos without hindering the productivity of their digital teams. This approach transforms governance from a burdensome process into a streamlined, integral part of the digital value chain.

How would a Shepherd do your Governance Job?

I often use this metaphor to explain the potential of 'Automated Governance' to my clients because it clearly illustrates the challenge of managing a large, autonomous system.

Imagine you are responsible for several herds of sheep. These sheep would be stressed if you constantly interrupted them from their activities (eating grass, sleeping, or other tasks).

You can't be with the sheep all the time while they are in the pasture. As a shepherd, you have many other responsibilities besides watching the sheep.

So, let's explore what this metaphor can teach us. I'll try to explain it using the concepts of Team Topologies .

?? The Sheep (Stream-aligned Teams)

The sheep need a safe, nourishing environment where they can work efficiently without constant distractions. Flow-blocking governance processes and continuous attempts to train the sheep about risks would hinder their productivity. Instead, they need simple, clear rules and an environment that allows them to work creatively and productively without compromising the safety of the shepherd business.

To be successful and profitable, we need to scale the number of productive sheep on our pasture while keeping the number of people managing the pasture, including shepherds and other workers, as low as possible. Having one shepherd for each sheep would ensure maximum stability, but it would be impossible to sustain that business model.

(Side note: A few years ago, one of my engagements with a governance unit was canceled because they felt offended when I discussed this topic! What about "Don't shoot the messenger"? A few months later, that governance unit was dissolved for being just a cost factor that did not provide value to stream-aligned teams.)

?? The Pasture (Platform as a Product build by Platform Teams)

The pasture is a platform where the sheep graze and are productive. It provides all the necessary resources and a solid foundation for productivity. The pasture/platform should be kept as simple as possible and, besides the essential components – nutritious grass and flowers – contain no distracting elements.

Sometimes the shepherd participates in building the pasture platform because he is skilled enough to do it. However, most of the time, the shepherd does not have enough time or the proper skills to do this, so he delegates the activity to specialized workers (comparable to Platform Teams).

?? The Fence (Facilitation of Governance Guidelines)

A pasture without a fence poses both external risks (e.g., wolves wandering around) and internal risks (e.g., improper consumption, security breaches, "useful illegality").

The fence provides an intuitively understandable framework to address these risks. Given the complex nature of potential threats (detailed in hundreds or thousands of pages of documentation and guidelines intended for shepherds but not easily accessible to sheep), it's impractical to teach all sheep every detail. Therefore, the fence serves as an abstraction that the sheep can grasp with minimal effort. Our autonomous herds are here to graze and focus on their tasks, not to be trained as shepherds.

However, there's always a chance that someone will attempt to break through. In such cases, we can expect extreme chaos on our pasture. The fence cannot fully protect against sabotage or ignorance!

?? The Sheepdog (Automated Governance)

This is where the sheepdog comes into play. Think of it as a form of "Automated Governance," perfectly trained to send important signals to the sheep, intruders, and the shepherd, thereby influencing everyone's behavior.

This guardian constantly remains on the pasture, ensuring the sheep are not disturbed as long as they follow the rules and intervening immediately if a rule violation occurs. Under normal circumstances, the sheep would instantly adjust their behavior to stop the dog's barking. If the dog continues to send alarming signals, the shepherd knows that his intervention might be necessary.

This system ensures autonomy for the sheep, decoupling their daily business from the shepherd's schedule, and provides a scalable solution (e.g., 1,000 sheep split across three pastures, watched by three dogs, managed by one shepherd).

???? The Shepherd (Governance Enabling Team)

The Governance shepherd ensures that the herd follows the rules and boundaries. However, he can't always monitor if the rules are being followed. Constant monitoring would be too stressful and distracting for both him and the herd, negatively impacting business outcomes.

The challenge? Keeping track of many teams while ensuring a safe and productive environment. How can one shepherd manage hundreds of freely roaming sheep as an audit organ?

The shepherd must ensure the pasture/platform is safe by design. Intuitive guidelines and alarm signals are needed when rules are ignored. To achieve this, the shepherd has many responsibilities:

• The pasture/platform must have a proper fence that represents his guidelines (Shepherd as an Enabler/Facilitator for those who build the platform).
• A 24/7 watchdog must be trained to intervene when rules are not followed on the pasture/platform (Shepherd as an Enabler/Facilitator for those who build/train the shepherd dog).
• The sheep must learn to understand the meaning of the fence (Shepherd as an Enabler/Facilitator to teach about the characteristics of the fence).
• The guidelines must be kept up to date, requiring constant effort to ensure that the fence, watchdog, and sheep are also up to date. It's crucial to find the right level of abstraction. Ideally, changes should cause no or minimal interruption and retraining (Shepherd managing the product lifecycle of his content).
• If rules are broken, as noticed by the dog not stopping barking or showing escalating signals, the shepherd must prevent chaos by intervening with appropriate force based on the situation (Shepherd as a Crisis Manager). However, this should not be the default behavior except in chaotic situations.

Team Topologies for Automated Governance

The accompanying diagram illustrates how this principle can be practically implemented within an organization, aligning with the vocabulary and framework of Team Topologies.

In this model, the Stream-aligned Teams are depicted as the primary work units focused on delivering value. These teams operate with minimal distractions and maximum autonomy, thanks to a supportive governance framework.

The Governance Enabling Team acts as a facilitator, providing guidance and support to the Stream-aligned Teams. This team ensures that governance processes are non-blocking and integrated early in the workflow, building a culture of trust and respect. By facilitating rather than obstructing, the Governance Enabling Team helps maintain a steady flow of productivity and innovation.

The concept of Governance as a Service is central to this model. It represents the evolution of governance practices into a scalable, digital platform that automates compliance and governance processes. This service is seamlessly integrated into the work of the Stream-aligned Teams, reducing manual interventions and allowing for continuous, real-time governance.

Finally, the Platform Team provides the necessary infrastructure and tools that support both the Stream-aligned Teams and the Governance Enabling Team. This foundational layer ensures that all teams can operate efficiently and focus on their core tasks without being bogged down by governance issues.

By evolving traditional governance into a service-oriented, automated model, organizations can achieve a balance between innovation speed and governance rigor, ensuring that both can coexist and thrive.

This visual representation underscores the importance of facilitating governance to support team productivity and highlights the role of digital platforms in streamlining and automating governance processes.

Comparing Blocking Governance and Automated Governance

With the implementation of Automated Governance, modern organizations achieve a balance between flow/innovation speed and safety/stability while gaining the ability to scale and create new value streams. Teams can focus on their work while the platform actively protects against misuse and risks. Here are some of my usual arguments for why this investment pays off!

The Issue with Blocking Governance

Classical governance processes, characterized by a collaboration model based on mistrust and the four-eyes principle, have many unpleasant characteristics despite their value. They are blocking, occur too late in the process (shortly before delivery), and are often viewed as distrustful, dogmatic, and unpopular. These processes are frequently referred to as extrinsic cognitive load (distractions from essential tasks) in qualitative surveys. While qualitative assessments can be debated, in high-pressure engineering environments, these processes are often measurably unsustainable due to their throughput, speed, and error-proneness, as shown in value stream analyses.

Why Invest in Automated Governance?

Transforming to a digitized and automated governance landscape not only reduces the perceived burden and improves the image of governance units but also accelerates governance processes from the perspective of stream-aligned teams. It resolves blockages, provides self-help tools, and reduces the error rate in the governance process due to the frequency and reproducibility of the processes.

The power of combining both systemic models

As a shepherd on the governance pasture, facilitation and "help for self-help" may not always be enough. The changes described here significantly contribute to flow and stability, but there may be instances where stream-aligned teams or platform teams deliberately ignore critical signals.

Despite the "autonomy" and "responsibility" given to these teams, a governance unit must be able to intervene quickly in these situations. This intervention may involve blocking and issuing clear hierarchical signals. The primary goal is to protect the company from harm as quickly as possible, not to maintain "innovation speed."

In these situations, the governance team can deliberately switch to blocking mode, being disruptive, assertive, and cautious. The rest of the organization can continue working undisturbed with a good workflow while the governance team addresses the individual case. Although it may seem slightly contradictory, a governance unit in a digitized landscape should master both approaches and be able to switch between them quickly.

Transformation Through the Eyes of Governance Units

This transformation process can be relatively painful for many governance units, as it requires numerous simultaneous changes. These necessary changes encompass various levels, including behavior, role interpretation, processes, technology use, and hierarchy.

The following visualization based on Team Topologies highlights some systemic aspects of this change. But remember, we don't want to abandon the ability to perform "Blocking Governance." Instead, we ensure that we focus on impacted teams in alarming situations while leaving everyone else autonomous when no intervention is needed.

Behavioral Change: Adopting a Customer-Centric Approach in Governance

All interaction partners (Stream-aligned Teams, Platform Teams, Complicated Subsystem Teams) must be treated as customers. While the governance needs of the organization are important, the needs of the engineering teams must also be acknowledged. Governance units must learn to be perceived as valuable enablers with empathy for their customers—the engineering teams.

Addressing Hierarchical Challenges in Governance: Essential Steps to Prevent Agile Transformation Failure

Unfortunately, I have had negative experiences with the behavioral change when governance units were allowed to act like the final boss in a computer game, throwing around stones like mad, shouting at everyone and firing some unnessecary escalation-nukes.

The needed behavioral shift aims to place all teams on an equal footing when intervention is not needed, while still allowing any form of hierarchy and escalation if warning signals are ignored.

To achieve this, governance units either need to learn to interact without hierarchy in these situations or, in many cases, must be directed (and sometimes forced) by upper management to adopt this expected behavior.

The latter often results in conflicts between the CIO/CTO/Head of Engineering (responsible for digital value delivery) and the Chief Compliance Officer. If they don't agree on this, you can forget any ideas of removing this bottleneck. It doesn't matter what cloud technologies you use or how agile your organization pretends to be if you operate in a regulated ecosystem and top management cannot align their expectations with modern industrial engineering principles. I have seen many senior engineers quit their jobs because of exactly this scenario.

Enhancing Flow with Improved Interaction Modes

From a systemic perspective, several forms of interaction need to be adjusted or added to the portfolio of a Governance Team. The audit process itself should be standardized, digitized, automated, and continuous (e.g., with every code change).

Here's how we can achieve all that:

1) Enabling and Facilitation for Platform Teams to Build a Governance as a Service Offering

Governance units are absolute experts with deep knowledge in their field but usually lack the engineering skills needed to offer continuously available governance self-services. These self-services are best housed within a platform team. However, the governance team must first ensure that their subject is properly understood by the platform engineers. Governance experts act as requirements engineers for the "Governance-as-a-Service" project in an enabler role, providing "help for self-help" (facilitation).

2) Governance as a Service & Governance Engineering in the Platform Team

In the future, the audit process will be digitized and automated instead of relying on error-prone, lengthy, bureaucratic human processes. Critical signals (warnings, blocking issues) from these self-services should be sent not only to the stream-aligned teams but also to the governance units to maintain transparency about boundary violations.

This approach falls under the domain of Governance Engineering. Platform engineers must first understand the meaning and value of these governance topics, but they have the expertise to integrate scalable, digital solutions into their platform. They are generally more empathetic to the needs of software engineers in stream-aligned teams and know how to establish and develop self-services with a good developer experience (DevEx). For more insights on Governance Engineering, you can refer to posts of Bill Bensing on Linkedin, take a look at his book and/or join his GovEng community.

3) Enabling and Facilitation for Stream-aligned Teams to Understand Governance Signals

We have now achieved the automation of signals being sent to the software engineers in the stream-aligned teams via the platform. However, these signals must also be understood to initiate the appropriate changes. For this purpose, the governance teams continue to provide their services to the stream-aligned teams through optional interactions. The governance team remains responsible for creating target-group-oriented documentation and continuously improving it (Documentation as a Product). Additionally, various support services based on this documentation can be developed, such as training sessions, short learning videos (learning nuggets), and GenAI-based assistants to review documentation and code. With the help of these self-help assets, the stream-aligned teams can now decide when they want to seek additional advice.

Conclusion

This article is not about a change in the sense of "old versus new" or "good versus bad," but about a necessary development and increase in the governance maturity level. This development is, in my opinion, indispensable in organizations with digital value chains.

Governance is not an end in itself and must align with the company's goals. These goals have changed drastically in many organizations over the past decade. Stability and security at the expense of adaptability are no longer acceptable in the digital competition.

Automated Governance is currently the best response to this shift.

#PlatformEngineering #GovernanceEngineering #DevOps #EngineeringCulture #DevEx #ValueStreamManagement #VSM #FlowEngineering #Digitalization #DigitalTransformation #AgileLeadership #EngineeringManagement #EngineeringLeadership #Governance