Answering the Question, How Does Your Solution Solve My Problem With Actionable Infomation I Can Confirm?

When we hear about a "solution" to the problem of agile development, any solution, from simple and possibly simple-minded, to complex business systems, we must ask ourselves "what is the problem that is being solved by this solution" it rarely if ever comes with a step-action-plan and a set of units of measure for assessing the Effectiveness and Performance of those actions in Increasing the Probability of Success for the project.

In the agile world, the Team is usually allowed to come up with their own processes as they see fit. Those are nice words, but if that team is spending other people's money, it is very unlikely they can choose to work in a way they decide without some oversight from those paying for their work.

This is one of the core myths of Agile - self-organizing teams does not mean self-directed teams, unless they're spending their own money or the project they are working on is de minimis - meaning if it's late, over budget, and the needed capabilities are not produce no one cares.

The Seven Principles of Disciplined Agile

I've come in contact with the Disciplined Agile concept being "sold" by PMI. By sold I mean you need to take the course to become certified or at a minimum buy the book or download it if you're a PMI member. I've gone through the book several times and it appears like the Agile Manifesto and 12 Agile Principles that DA is fully buzzword compliant.

1. Delight customers
2. Be awesome
3. Context counts
4. Be pragmatic
5. Choice is good
6. Optimize Flow
7. Organize around products and services
8. Enterprise awareness

Now I use fully buzz word compliant is a phrase I learned from one of my many mentors. After sitting through proposal presentations at the Tennessee Valley Authority, where we were helping the IT department develop an IT strategy that included acquiring an ERP system, a Document Management System, and integrating those into the Nuclear Power Station Watts Bar. After listening to a vendor give their pitch, my boss said that's a nice presentation that is fully buzz word compliant, now could you tell me in specific units of measure how your solution solves the problems we stated in the RFP?

Let's look back to the 12 principles of agile first before looking at DA

These 12 principles fall into several categories

• Restating the obvious
• Missing units of measure
• Simple Fallacies

1. Customer satisfaction by early and continuous delivery of valuable software

This is one of those don't do stupid things on purpose buzzword phrases. What credible management method would NOT want customer satisfaction. But then comes the bigger question:

• At what rate can the customer accept this valuable software?
• Does the customer have a business rhythm for acceptance of new, changed, updated, fixed software? Without that, the unit of measure foe early and continuous is unanswered

2. Welcome changing requirements, even late in development

This is one of the foundational root causes of project failure talked about in Death March. Changes on all projects, agile or not need to have a Change Control Process. Without change control there is no way to assess the impact of the change on the cost, schedule, technical performance, stability, security, maintainability - if fact all the ...ilities of the resulting product or service.

When you make changes in a codebase for all buy de minimis products, those changes are going to ripple through that code.

To properly manage change we need to first assess the impact of the change and ONLY then approve the change.

3. Working Software Delivered Frequently (Weeks rather than Months)

This is restating the obvious. Why would you NOT deliver working software in exchange for being paid to deliver a working product or service? The challenge is what are the units of measure of working?

Without defining what are the deliverables, there is no way to assess how often they can or should be delivered. The notion of incremental capabilities and their delivery intervales is completely dependent on what these capabilities are.

• I need the capabilities to find where my package is in the supply chain and when it will be arriving at the warehouse, is not the same as ...
• I need to capability to track all the packages in the supply chain, find the redundant orders, confirm all the order comply with the ISO 31000 standards for our procurement, match those orders with the authorized budget for those materials and confirm I'm not going to have over-stock or under-stock in the warehouse over some extended period of time.

4. Close Daily cooperation between business people and developers

This of course means the business people have two jobs now. One working at the assigned job and the other working with the development team. So the question is how can the business person convey what the software system needs to do in terms understandable to the developers? The answer is Capability-Based Planning.

Capability-Based Planning identifies the levels of capability needed to achieve the strategy, a problem common across many defense forces. With the assistance of scenarios, CBP explicitly connects capability goals to strategic requirements.

In a business context, capability-based planning is an approach that ensures that changes in an organization are aligned to the overarching strategic vision. The approach has its origins in defense and military planning in the US, UK, Australia, and Canada. CBP has become popular in the business domain, particularly for developing systems and IT-related strategies.

5. Projects are built around motivated individuals, who should be trusted

Where do you work where it is acceptable to have unmotivated individuals as an acceptable practice?

6. Face-Face Conversation is the best form of communication

This may or may not be correct. If the problem to be solved can be explained in words, then yes F2F is fine.

Now let's look at the problem of interfacing two systems, here are examples from hands-on experience using agile in Complex Adaptive Software Intensive System of Systems (CASISoS)

• In a public health insurance state system - developed with Scrum - interfaces to the IRS, FBI, INS, DOS, DHS, and other Federal systems are needed for the State of Colorado to grant ACA insurance process. Those interfaces are defined in documents, and formal process and data specification. Face-2-Face talking about managing these interfaces is pretty much a bust
• The Command and Data Handling (C&DH) system and the Guidance Navigation and Control (GN&C) system for the Mars lander is developed using an agile process - since the technical requirements are emerging and no single requirements specific will get a correct product. These two systems must talk to each other so Perseverance doesn't crash on the surface of Mars. No amount of Face-2-Face communication is going to flesh out all the details. The interface is defined in SysML formal language, that is testable and verifiably correct - with a proof of correctness

7. Working Software is the Principles Measure of Progress

OK, but what's the definition of working. This is one of those restating the obvious principles, with NO actional outcomes or meaningful units of measure.

First, the units of measure must be

• Measures of Effectiveness - are operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in the operational environment, under a specific set of conditions.
• Measures of Performance - are measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions.
• Key Performance Parameters - are measures that Represent the capabilities and characteristics so significant that failure to meet them can be cause for reevaluation, reassessing or termination of the program
• Key System Attributes - that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal
• Technical Performance Measures - that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal

Here's an example of MOEs and MOPs for a Stylish Coffee Cup

Without defining these the phrase working software is the measure of progress is just a buzz word

8. Sustainable development, able to maintain a constant pace

Yes, restating the obvious

9. Continuous attention to technical excellence and good design

Nice, but how do you do this? To what level of detail? What is the measure of excellence? What the definition of good. All just buzz words.

10. Simplicity - the art of maximizing the amount of work not done - is essential

Yet another example of a unit-less buzzword. What is the measure of simplicity? Does simplicity have any connection with effectiveness and performance or any other of those measures above?

11. Best architecture, requirements, and designs emerge from self-organizing teams

This is completely domain-dependent and therefore of little use as guidance.

TOGAF and DoDAF guide or work. If you work where there is no external architectural compliance, then you can make up anything you want, no one will likely care. Another example of a de minimis project.

But if your system has external connectivity with other systems, even those no guided by ToGAF or DoDAF, say the architecture for the integration with SAP or any other large complex COTS product, architecture, requirements, and designs emerging from self-organizing teams is likely to lead to a smoking hole in the ground where you project was.

Here's an Enterprise IT architecture

And an embedded systems architecture

Both designed for accomplishing a mission or fulfilling a strategy across a wide range of problems, with many, sometimes dozens to many dozens of participants in defining the needed architecture to accomplish the mission - Capabilities Based Planning drives the architecture

12. Regularly, the team reflects on how to becomes more effective and adjust accordingly

yes, good business and project management and restating the obvious.

This is the example of me asking my project managers, and they asking their teams what have you done for me lately to move this project toward success?

Standard, out of the book, project management governance process.

So Now Let's Look at Disciplined Agile in Light of the Issues with there 12 Agile Principles

1. Delight customers - who would not delight the customer?
2. Be awesome - got any units of measure of being awesome
3. Context counts - yep, context and domain are always the starting point for ANY process or practice. But the principles of increasing the probability of success are not stated in DA
4. Be pragmatic - when would you NOT be pragmatic in the presence of uncertainty?
5. Choice is good - in the presence of uncertainties that created reducible and irreducible risk, choices are always needed.
6. Optimize Flow - when would you NOT optimize flow?
7. Organize around products and services - what other organization structure is there in software development? The product can be internal or external. The service can be internal or external. What is can there be?
8. Enterprise awareness - So once again, a unitless measure. What are the units of awareness? DA says teams should strive to do what is right for the organization. But who defines what is right? The team? Not likely unless they're spending their own money. This is one of those buzz words that ignores the principle of Corporate Governance. The team may be self-organizing, but they are NOT self-directed.

The Way of Working processes that are the basis of Disclpied Agile appears to not have any understanding of Systems Engineering, Systems Management, and Program Governance. This seems to lead to restating the obvious. If you start with a Sytems View, the Programmatic and Systematic architecture of the work efforts define units of measures of the 8 principles of DA, where DA itself doesn't have those units of measure - or at least I;ve yet to find them.

DA is essentially saying Do Good Work

Some More Analysis in Light of Buzz Word Compliance

• Collaboratively developed vision - The “governed” are actively involved with the development and evolution of the organization’s vision. Goes against the principle of separation of concerns in any non-trivial project. The Chief Engineer for fly to Mars or Developing and Deploying the Process Safty Management Systems at a petrochemical firm doesn't actively involve the governed. They have their role he has his.
• Collaborate with other teams. Our team is only one of many within the organization and we often need to collaborate with other teams to achieve the outcomes that we want. Brings us back to the separation of concerns.
• Educate staff - Our organization must educate, train, and coach staff members in enterprise-level concerns such as security, our business vision, our technical vision, and many other critical issues. Yes, an educated staff is necessary for success.

What is Disciplined Agile in the End?

I was one of many Vice Presidents at a nuclear weapons plant (you had to be a VP to have enough money on your Government Purchasing card not to have to go to purchasing anytime you wanted to buy something. In ICT (Information Communications and Technology, everything IT) we had $250,000 spending limit on our P-Cards).

One time there was a complete site shut down or our 7 years $9 Billion program when some Steelworkers were sealing doors on an elevator shaft in a Nuke-Building with several floors below grade. They went to Home Depot and bought cans of foam sealant, sprayed it around the edges of the plywood they install where the elevator doors used to be. Didn't read the directions, sprayed on a second coat, and set fire to the plywood. Throw water on the fire, which was from an exothermic process, and made more fire. Called the fire brigade which put out the fire.

There was a complete site shut down for all 5,000 staff on-site and a very large banner appeared in the lobby of our building. We never went on site, it was much too dangerous. The banner said ...

DON'T DO STUPID THINGS ON PURPOSE

I'd suggest that DA is a handbook for not doing stupid things of purpose that you have to pay money to learn and get certified, but then you have no verifiable experience test, external validation from senior mentors that you can actually do your job with that training and nice new certificate.

This is the core problem with PMI

• You can take a class to get a PMP or DA certification by just enrolling in a course and taking a test.
• The certification does validate you have had the experience, but just the duration of the experience, not an assessment of competency over that period of experience and doesn't call out the domain or assess the competencies from that experience nor assess the proper principles, processes, and practices that have been properly applied during that experience to solve problems with tangible evidence that problems have been solved. Just that you have a certificate of completion.
• In other professional organizations, certified experience, certified recommendations, and then the exam is needed. INCOSE, AACE, ICEAA, FAC-P/PM, and other Professional Engineering societies require you to show not only can pass tests, but you are also competent and experienced in the domain to be considered a Professional.
• This is why we don't see many PMPs in our CASISoS domain.

Why Did I Write This?

I see DA going the way of the PMP and its goal to sell training and certifications, along with external training organizations that produce certificate holding students with little or no experience requirements for having put that knowledge to works on non-trivial projects.

PMI has become a business of getting people certified but provides no assessment of their actual competencies after certification to accomplish the mission of successfully managing projects in ways other professional certifications do.

Is this a narrowly focused criticism of DA and other PMI certificates? Likely.

I've encountered Certified Scrum Masters, who never worked on a complex project.

Competency Assessment

Competency is the application of knowledge, skills, and behaviors used in performing specific tasks. A competency assessment is the assessment of someone’s capabilities against the requirements of their job. Those requirements are defined in a competency model. To be valuable, competency models should contain only tasks and skills that are critical to success in the role, not every activity they perform in their job (which comes from a traditional job task analysis).

Successful projects, in any domain from Agile to Traditional depends on identify gaps in the process, procedures, and practices creating problems and taking corrective and or preventive action to address these issue before they affect the probability of project success. A competent PM must be able to apply those processes, procedures, and practices in a manner that increases the probability of project success.

Competency assessment methods include:

• Direct observation helps identify and assure performance in the role of a Project Manager
• The techniques are watched during the application of the principles, processes, and practices to observe if the PM is following the guidance learned during any training or course work.

The Final Unanswered Questions

• What are the units of measure for each of the principles and practices of DA?
• Without these units of measure, how can the business adopting DA, know they are making progress to plan toward each of the 8 DA topics?

[1] Competency assessment, a process for identifying, developing, and assigning high performing project managers

[2] Understanding of Project Manager Competency in Agile Software Development Project: The Taxonomy

[3] Competences in Agile Development: Exploring the social, functional and cognitive requirements of a systems developer