An Incremental Paradigm

I spoke at a Department of Homeland Security (DHS) with a colleague on the topic of Estimating and Managing Agile development in the Federal Government, and a senior director brought?up a paradigm that resonated?with me.

The?infrastructure?for enterprise IT has a critical impact on the operational aspects of projects. The notion of continuous?delivery in an enterprise?like Homeland Security is an interesting discussion. That discussion is deep and broad, but a simple?cartoon?drove home a point.

We live in a?golf course community, where our wastewater is processed locally inside our neighborhood. The Wastewater plant can be seen from our deck. That non-potable water is used to water the course, so complete recycling is the result. The grass is green, the solid waste is hauled away for further processing, and we benefit from lower costs.

Agile and?Incremental, and Iterative Development

Like the agile incremental house cartoon, or the bicycle morphing into a car, discussion of software's incremental or iterative development can only occur with a Product Roadmap and?Release plan. With these two plans, the?Product?Backlog?and Sprint Plans can be developed incrementally and iteratively to produce the needed Features for the customer.?

But only sometimes, if ever, does a Feature?standalone?unless it is a de minimis project. Here's the paradigm we use for our enterprise development programs.

In the residential wastewater development paradigm, the Concept of Operations (ConOps) describes the capability to use the toilet?(the Feature) only to the homeowner if the?infrastructure?of taking that waste from the toilet?to reclaimed water is in place. The Integrated Master Plan (IMP) of the neighborhood builder shows the sequence of connecting the bottom of the toilet to the wastewater plan. The Integrated?Master Schedule (IMS) shows the build sequence of the touch labor work, install all the equipment, and make all the connections.

The same is true of any agile enterprise IT product or service development. The ability to?pay vendors?is of little use without the ability to?Invoice vendors. The Product Roadmap would show that.?Along the way, each?feature?in the value proposition chain has to be in place in some way.?

So when you hear Continuous Integration (CI) and?Continuous Delivery?(CD) as?buzzwords, ask?if you can show me your Product Roadmap and Release Plan (either Capability release plan or Cadence release plan). In these plans, the needed Capabilities the customer expects will be shown properly, with the predecessor and successor dependencies to the Capabilities?delivered?before?and?after?the Capabilities needed by a specific person or business entity.

This goes along with the nonsense idea of?starting with the most important feature and keeps developing until told to stop. This means that someone else is prioritizing, sequencing, and funding the effort needed to produce value. The developers are then just labor, being told what to do by outside entities. That's fine, but expect little consultation with the developers since they are not in the loop of making uncertain decisions.

No? Then those words are just hollow buzzwords.?

Incremental Commitment Spiral Model

The Incremental Commit Spiral Model provides an iterative and incremental framework for complex software intensive system of systems.

This model has its roots in the Iterative and Incremental Development (IID) methods in use since the mid-1970s, as described below in Craig Larman's paper.

It's a popular misconception of the Agile community that?Waterfall?is used in any domain outside their own personal experience, and Big Design Up Front (BDUF) and Big Bang Development (BBD) are the only alternatives to Agile.?

The Classic Misunderstanding between Principles, Practices, and Process

As an early user of (a FORTRAN 77 developer on Software Intensive System of Systems for Ballistic Missle Defense systems), the principles of Iterative and incremental development, emergent requirements,?the customer in the loop, and other principles, practices, and processes found in?Agile?started long before the Agile Manifesto, in 2001.

Let's start with a?history paper, “Iterative and Incremental Development: A Brief History,” Craig Larman and Victor Basili, IEEE Computer, June 2003.

And here's another "Iterative Enhancement: A Practical Technique for Software Development,"?Victor R. Basil and Albert J. Turner,?IEEE Transactions on Software Engineering, Vol. SE-l, No.4, December 1975.

And another from early on in complex systems development. "Design, Development, Integration: Space Shuttle Primary Flight Software System,"?William A. Madden and Kyle Y. Rone,?Communications of the ACM, Volume 27, Number 9, September 1984.

These are a few of many papers describing how iterative and incremental development was used - long before the Agile Manifesto and eXtreme Programming.??

Then let's look at the actual description of how iterative and incremental development took place in?“Managing the Development of Reliable Software,” R. D. Williams, Proceedings of the International Conference on Reliable Software, April 21-23, 1975. You'll need an ACM membership?(which I doubt the OP has) to read this one. But notice in the Larman paper that IID started in 1936 and has evolved ever since.

Another paper, where I have hands-on experience is "TRW's ADA Process Model for Incremental Development of Large Software Systems," Walker Royce,?TRW Systems Integration Group, Space & Defense Sector, Redondo Beach, CA, describing the development of the Command Center Processing and Display System-Replacement (CCPDS-R) project's success in with over 300,000 lines of Ada source code executing in a distributed VAX VMS environment. The VAX is my favorite real-time?computer, by the way.

I've since lost my?handbook?issued at TRW for the Software Development Lifecycle processes used on the programs we worked on, but the?loop?approach of?the plan, do, study, act?was at the heart of that method. And of course, the classic?misused and?misquoted?paper that agilest use as the stalking horse for their claim?iterative and incremental?is the replacement for the dreaded?Waterfall?process is Winson Royce's "Managing the Development of Large Software Systems," WESCON, August 1970. By large, Royce means?large. Defense systems.

Is this early work the same in form as Scrum, XP, or other Agile development methods? Not exactly. Are the principles the same? Yes, pretty close.

But of course, the Agile Manifesto could be more actionable in a practical?way, as the 12 Principles are. And By The Way, the 12 Principles of Agile are simply restatements of good project management.

Those that objected to the early XP approaches - and I had an email spat with Ron over how to release?partial?features to a production system where those?partial?features were missing the core elements of the needed Capabilities -?were right to criticize?the lack of governance and actionable outcomes.

You cannot make an informed decision in the presence of uncertainty without?estimating the impact of that decision.

To claim otherwise willfully ignores the principles?of managerial finance and probabilistic decision-making.

This misinformed, sometimes intentionally misinformed, idea clouds the conversation. Agile is more common now and has moved to Software Intensive System of Systems where I work, but IID has been around for a long time. The latest anti-establishment advocates need to read some history before claiming?we've discovered?new and better ways of developing software, a sample of how it was done, perhaps long before those critics were born.

• "Iterative and Incremental Development: A Brief History," Craig Larman and Victor Basili,?IEEE Computer, June 2003.
• "A Software Lifecycle Methodology and Tool Support," Leon J. Osterwell, Department of Computer Science, University of Colorado at Boulder, Boulder, Colorado, 80309
• "Managing the Development of Large-Scale Software: Concepts and Techniques," Winston Royce,?Proceedings, Wescon,?August 1970.
• "Iterative Enhancement: A Practical Technique for Software Development," Victor Basil and Albert Turner,?IEEE Transactions on Software?Engineering, Vol. SE-1, No. 4, December 1975.
• "Real-Time Software Engineering in Ada: Observations and Guidelines," Mark Borger, Mark, Klein, and Robert Veltre,?Technical Report, CMU/SEI-89-TR-022, September 1989.