Agile waterfall

The first project on which I led a sub-team was a software development for calculating employee taxes 1984-85. It concluded quite successfully on time and quality, I have no clue on the budget though (and was not in charge of it).

The project was overall a 4 year effort with more than 150 people on it, creating a new employee accounting system for 30K German IBMers incl. retirees. Technically it used the transaction engine of IBM IMS and each employee report was run as one transaction (this way the backup and database rollback features of IMS could be used - a hugh benefit).

For sure the project and sub-projects were planned as what we today would call waterfall: a series of phases producing interim results that were used in the following phases, until a system test and cut-over would put the whole system into operation.

When I came on board as a new hire, I already had developed applications on IBM systems for 8 years, for a bank in parallel to my studies at university. So I was given the lead for the sub team developing the tax features. The requirements and design phases had produced voluminous documents which I was asked just to implement (code and test). In reading the documents, I came across some questions and asked a key user (a subject matter expert on employee taxing) to help me understand and clarify.

This started a year long process of frequent meetings, re-designing the application and features, adopting to new rules and jointly deciding on how to implement them. These days this was sometimes called JAD - joint application design, though we did not use fancy terms. The overall project manager and lead architect gave us free hand, and they were proven right. We met the milestones in the waterfall schedule and I remember the team celebrating in the B?blingen city hall.

This was my first agile waterfall project and I led many since.

Waterfall sometimes is described as a predictive life-cycle for creating a product. It is not a feature of project management itself, which can support any type of life-cycle. Waterfall life cycles divide the time given to a project in phases and each phase ends with a stock taking debrief and a decision to proceed to the next phase or not (phase gate). This might be important to release the next funding from the company coffers, or to review the relevance of the strategic targets for which we started the project at the beginning, or to just acknowledge the good quality of the results of the past phase.

Predictive means that we think ahead and try to find out what the future might bring. We do that all the time, our brains are using prediction as a successful strategy to help us survive. If we hear a rustle in the bush, we may predict a sable tooth tiger and are prepared to run or fight, adrenaline is pushed in our blood. If we commute to work again, after Corona, we predict (estimate) the time it will take based on current weather and traffic, and we might decide to leave 30 minutes earlier. If our boss asks us to take on a new urgent task, we immediately predict how long it will take and which other tasks will be delayed. We cannot NOT predict.

Especially in complex and large tasks, like the 4 year software project I was getting on, or if you climb a summit in the Alps, or if you eat an elephant, you automatically break down the task into smaller chunks and put them on the timeline. You have a long-term target and you try to set short-term targets. We often plan with the end in mind and from the end (backwards path in CPM - critical path method).

The German General von Moltke said 150 years ago, that no plan survives the first contact with the enemy. Strategy is won on the battle field, not in the Board room.

And here another capability of our brains sets in: being responsive to changes (aka agile). We hear the rustle in the bush, expect the tiger but discover a deer. We change plans (running away) and hunt the deer for feeding the tribe. As we drive to work we recognise that the fuel gets low and decide to stop at a gas station (thank god we added a risk buffer which helps us with this unknown unknown). In looking at the task our boss gave us, we recognise that it will help save us a lot on two other projects, as we bring in the top notch expert for some time, we just need to adjust the portfolio a bit to gain a huge benefit.

At the beginning of each project journey the uncertainty is highest, as we do not know too much about our long-term target (what) and the approach to get there (how). Some even are uncertain about the purpose of the project (why). As we travel, maybe we read or experiment to reduce risk as we find out more about the what, we find specialists who climbed that mountain before to help with the how, or as we get closer to he summit we just recognize more details and might question our why we want to go there anyhow. The plan gets adjusted and detailed - it is permanently changed and progressively elaborated.

The final plan is the one you have after the project ends.

Luckily our brains have a set of hardwired capabilities. Prediction and responsiveness are two of them, they are not mutual exclusive and we use them on a daily base. Others are the ability to recognize patterns, being curious and creative (when we are not under stress), our strive for growth (which can be detrimental), or our compassion for others (we enjoy giving more than taking). We need to utilize (and be aware of) all of them in order to survive and thrive and should not limit our use of them by imposing artificial mindsets (belief systems). Except that we should limit our actions by our shared human values (but this is another topic).

We are humans and use these capabilities since thousands of years. That’s how the pyramids were built. That's how we continue to ride the waves of disruption. Panta Rhei.

Let’s continue to do agile innovative growth waterfall projects.