Separation of Concerns and Agile Development

There is a notion that is popular in the agile community from the @agilefant tweeter feed. This notion appears when corporate governance is not part of the conversation. The role of?governance is to define the?decision rights?of the participants in the organization -?IT Governance: How Top Performers Manage IT Decision Rights for Superior Results , Peter Weill and Jeanne W. Ross, Harvard Business School Press.

When small teams of individuals take the Agile Manifesto literally and apply it to the?business?of writing software for money, those decision rights of the organization are usurped, sometimes without the formal organization realizing what is happening.?

Let's look a little closer at a non-de minimus corporation that is writing internal software for its operations. Or the same corporation writing software?for sale to others who will use it for internal or external operational purposes.

For the internal software?systems, the?user makes the assumption that the software works according to the needed capabilities of the organization. This means that the business can conduct its?operational processes without consideration of the state of the software. And the capabilities to accomplish the business mission or fulfill the strategy are available.

For example

Running your retail inventory control system for 147 warehouses around the United States will not be successful with a?Beta?version of the software. Or a version of the software that is?emerging?in its capabilities every week.?How to develop and deploy such a system is another topic, an important topic for another blog, but the operational aspects - what is needed, when it is needed, and the order of the needed capabilities - of the deployed system are defined by the Users of the system, not the developers of the system.

In this governance paradigm, the Product Roadmap development from a Capabilities-Based Plan (what capabilities do we need to accomplish the mission or fulfill the strategy?) and Release Plan defining these Capability outcomes, connected to the business strategy (usually something like a Balanced Scorecard) used to fund the development - all using the 12 Agile principles. By the way, let's review those 12 Agile Principles in light of standard corporate governance

So how does a?production-grade application get into production??First, let's look at the IT Operations structure of a?production-grade?system and the motivations for?separating?the concerns for the development of the system.?Non-functional concerns such as time and space predictability, dependability, safety, and, more recently, security, have an increasingly large incidence on system development in high-integrity application domains [1]

In our current software development paradigm, data is separate from the application?that use the data. The?classic?systems in the 1970s had data access, and the data itself was often embedded in the application. COBOL programs use IMS?databases, but access to that data is only available when the COBOL program is running. This motivates separating?the data into a Database (Oracle, SQLServer). So it is not to the corporation's advantage to have the management of that data spread across teams of developers. The data belongs to the?corporation, not the development teams. To do this, we need Data Based Administrators to manage this?corporate asset.?

In large-scale IT systems, a vendor often defines the system's architecture. SAP, Microsoft Dynamics, and other ERP systems. The vendor defines the architecture of these systems.

Here are a few words on this topic for a domain I worked in"

• Architecture Centered Design
• ERP systems in the publishing domain
• Architecture-Centered Publishing?Systems

So who manages the architecture of these Enterprise?Production systems?

• Developers
• Testers
• The?agile team??Not likely.

The Corporate asset?of the Enterprise IT system is similar?to the corporate asset of the buildings and facilities, supply chains, and communications systems. Architects do the management of the architecture.?

If developed?internally, these?corporate asset?IT systems require specific levels of integrity, reliability, defect-free targets, performance service level agreements, regulatory compliance and reporting.

Who is going to?ensure?these systems meet these requirements? The developers, probably not. They have day jobs writing code. Same for a purchased system. The?responsibility?for assuring the integrity?of these systems is part of the governance process as well. Cybersecurity, Database, and Application performance management and assurance, and software quality measured against the requirements is a full-time job.

This is just a sample of the issues that must be addressed through?the Separation of Concerns?in an Enterprise-class,?Corporate Asset?governance model. We can add to those formal governance models like ISO 12207, COBIT, ITIL, Sarbanes Oxley, ISO 38500, and similar frameworks.

Governance is about Decision Rights. Who gets to decide? The developers? For some decisions of course, for all decisions, only likely the case for de minimus projects.

Does this mean large, governance based, organizations can't be Agile? Can't write software with Scrum, Kanban, XP, Crystal, DSDM? Respond to emerging requirements? Not in the least. Of course, they can and they do, and I work several programs with 100's of developers and 100's of millions of dollars of budget, all using Scrum and Kanban.

But there is a separation of concerns needed in any sufficiently complex system. The Cyber Security staff does not write business production code. The Database architecture and performance management staff do not write User Interface code for the web services. These are standard governance management separations. Numerous examples of cyber breaches have been in the news recently for an obvious lack of?separation of concerns.

Separation of Concerns is a well-known concept in application architecture as well. Over the years, application structures have evolved from monolithic to modular, using encapsulation and abstraction techniques to reduce coupling and increase cohesion. The purpose of doing so is quite simple - it yields software systems that are easier to understand, change, and enhance.

At a minimum, separating cybersecurity from software development is mandatory in any corporate environment. There are rules and regulations in the financial, healthcare, and retail industries. The?NIST?Framework ?defines most of these first, then they are adapted to specific industries.?

Here's an example?process?for developing, testing, V&V'ing, cyber assuring, performance assuring, and releasing to production for an enterprise software system used by millions of customers in a regulated domain. So before accepting the notion that all those?roles and responsibilities?in the picture at the top of the post should be combined into a single team, and that's what's needed to be?agile, take a look at all the steps here. These are the?top-level?processes to get software from an idea to production in ISO 12207 processes for a healthcare firm.

There are similar guidelines for Data Governance. HIPPA and the?Health Care Analytics Adoption Model ?is one we all encounter. Ask Equifax and Target about cybersecurity and Data Governance. These are NOT roles for development, they are a?separation of concerns?for individual roles.

So when is it appropriate for the picture at the top of this post to be replaced by the agile communities notion that there is?No?Separation?of Concerns?

The answer is simple...

What is the Value at Risk?

If you mash up all those roles into a single self-organized team, with no separation of concerns, no governance processes, and no independent isolation of the processes, what could possibly go wrong, and what is the cost of that outcome? Answer that first before getting all excited about how Agile is going to remove the silos in your organization. Those Silos may be there for very good reasons.

Trust But Verify?

"Доверяй, но проверяй, Лучше перебдеть, чем недобдеть"

[1] "A component-based process with separation of concerns for the development of embedded real-time software systems," Marco Panunzio and Tullio Vardanega,?The Journal of Systems and Software,?96 (2014) 105–121

[2]?Production Systems and Testing Separation

[3] "Enterprise Architecture as Capability: Strategic Application of Competencies to Govern Enterprise Transformation,"?Janne J. Korhonen and Wolfgang A. Molnar,?IEEE 16th Conference on Business Informatics (CBI), 2014.

[4]?"Tripartite Approach to Enterprise Architecture," Korhonen, JJ & Poutanen, J 2013,??Journal of Enterprise Architecture,?vol 9, no. 1, pp. 28-38.