Projects and the Agile Ceremony Map

Project Managers. I have some non-news for you: Projects Fail. Everything you strive to do as a PM tries to counter this truth. All the processes and controls that Solid Project Management requires are to try and stop projects failing. But fail they do. Why?

I wanted in this article to look at Change and how it manifests itself through the project landscape and then how Change is viewed in Agile practices. I wanted to do this from the view point of the ceremonies, artifacts and meetings the two 'styles' put in place. I want to explore if Change is the reason why projects fail and if there is a mapping between project ceremonies and agile ceremonies that is explianed by the approach each method has to Change.

A Traditionally designed and run "Waterfall" Project doesn't want to fail

Projects, and Project managers start off with a determination not to fail. To do this some very specific actions and activities are undertaken and some very specific ceremonies happen to manage the delivery.

All projects start with a Mandate then a Business Case that contains an initial RAID log, Financial Budget and the statements on Return on Investment/Benefits. These things are poured over and taken through various Stage Gates to check they are sound. Everyone is clear on what the project will do, how it will do it, what it will cost and how it will be measured. Excellent idea. Then a Detailed Requirements Document is written up-front to be absolutely clear to everyone what we’re going to do. A full Project Plan is created with Resources Allocation so everyone knows when they're going to do everything. Again solid idea. A Steering Board is put in place to inspect progress and the project kicks off. How can we fail - we know exactly what's to be done, by whom and when?! Oh wait... We'll put in place a Change Control Process just in case, but we won't need it right?

Then things do change...

After a while the project comes across something unexpected, something that wasn’t in the Requirements Document. Not a problem, this is real life after all. A Change Request (CR) is raised. Then the Plan is redrawn the RAID log updated, and we move on. After a while though we find that we’re raising quite a few CR’s. The rate of CR’s increasing as the project progresses. Then a Risk we'd identified materialises and becomes an Issue and we raise a further CR to change the scope/cost/timeline. Soon we have a PMO analyst managing the RAID log and resulting actions due to the volume of the CR’s we’re creating.

Our Senior Sponsor becomes concerned and asks for the monthly Steering Board Review to now be every two weeks. Our Head of Delivery, hearing about the sponsor’s concern asks to attend the weekly project meeting to offer some 'help'. 

Suddenly the CR's you are facing become daily and the issues list is growning alarmingly. The project is suddenly going badly off track and not holding to the Requirements Document and Plan at all.

Enter the War-Room!

The Project manager (You!) instigates a War-room, puts up the list of issues on the wall and requires the whole team to meet in this war room to prioritise and address the most pressing of issues on a daily basis. Issues are priority ranked with input from the Sponsor, Engineers, Delivery leads and others with regards to which ones can be delivered, mitigated and are material to actually going live with something.

After the project gets delivered (or cancelled!) the war room is dismantled and a Lesson’s Learnt report is written. It’s rarely read by anyone as the whole thing was too painful to ever think about again.

Sound familiar? Bet there’s a wry smile on your face at this point. Been there? Me too.

Enter the War-Room 2

As a quick aside: You may have run a perfectly lovely project. All the design and development, and even your SIT testing went well but then you made the mistake (a few weeks before launch) of giving your lovely product to actual users after many months of development and testing - The dreaded UAT phase of a project! This, without fail raises a whole heap of feedback that the team have never considered and that needs to be urgently addressed before you can ship your product and close the project. Enter the war room again! Stack rank the user’s concerns, update the RAID log, go back to stakeholders with a possible slip to go-live and run around!

The impact of Change

You see why Change is often the root cause of being unable to control a project. We live in a Volatile, Uncertain, Complex and Ambiguous (VUCA) world, and have been for quite a while now. Change is part of our everyday lives. I suggest therefore that our project management style and method needs to be fundamentaly designed around accepting change as part of our delivery landscape.

How does Agile map to this?

How does “Agile” map onto this? Why might “Agile” have the answers?

An Agile mindset and Agile delivery methods assume, from the outset, that any endeavour WILL go wrong. An Agile way of approaching the world accepts the truth that: Change is the only constant.

Agile delivery processes are some of the strictest delivery guidelines I have ever come across. You have to hold to them to be successful. They are designed around the truth that change will happen and so it is how we manage that change and include it in our delivery that we can control that delivery and not decend into chaos. So change will happen but we are in control of that change.

The Ceremony Map

Reverse the project timeline above. If the standard project meetings and artifacts, as outlined at the start of this article, start top left of the image below and proceed over months towards the right, then an Agile project starts from the bottom right and proceeds, in increments of two weeks, to the left:

At the top left of the diagram "Waterfall" projects start and have a Governance Board every month (several of these are probably stage-gates) and a Steering Group or Project review every 1-2 weeks - and maybe even that daily War room review of issues when the change is over-whelming! Agile endeavours start from dealing with change, at the bottom right of the diagram, and mandate a Daily Stand-up then a Show-and-Tell every two weeks (every single one of which is a stage-gate) and a Roadmap/Backlog Review Monthly.

The high-level ceremonies of the two project styles map across. The difference is that the Agile process Starts from the point of view that Change is inevitable and you have to manage change on a daily basis.

Some further mappings.

Both styles have very similar artifacts, ceremonies and reports and they can easily be mapped one to the other. The fundamental difference is based on acceptance that: You can not know everything at the start of a 'project', that the development of the project will uncover further knowledge and that there will always be more to do than time and money will allow! In short; change is inevitable

War Room: Every Agile team has a daily stand-up around their list of issues. They assume a war-room mentality from day-1. All issues are transparent to the whole team and the entire organisation. They assume the project will go wrong and will change – from day-1!

Change Control Process & CRs: A change process is not an after thought for an agile team, it is the first thought! Every Agile team knows that things change, they accept this and build it into their process. They do not raise CR’s. These changes are just another thing to be pulled into the mix, evaluated against everything else, prioritised and moved on from. It things don’t change then they’ll be happier, but they never assume stability. It’s real life after all!

Steering Board: Every Agile team delivers outcomes and meets with their senior stakeholder every two weeks to show what has been produced and to check that they are doing the right thing. This is called a Show-and-Tell. In Agile teams the Senior Stakeholder is called a Product Manager. In fact the Agile team will deliver valuable outcomes every two weeks to Users (their Customer) - as it is only the Customer (not the Product Manager) who can tell them if what they are doing is valuable! A truely agile team must produce ‘potentially shippable product’ every two weeks, even after the very first two week period!

Stage Gates: The Agile team hold themselves accountable to delivering value every two weeks at the show-and-tell (Stage gate). If they’re not delivering value to users then they accept that their ‘project’ could be shut down. Ever worked with a team that could be closed down every two weeks? That’s an Agile team. It sharpens the mind and really drives a focus on delivering excellence and great and valuable outcomes for customers! Being Agile requires courage, but that’s another subject.

Project Plan: Every Agile team, together with their Product Manager will be looking ahead on a monthly and quarterly basis to check on the road ahead and will hold reviews with Senior Sponsors (Senior Product Managers) to check on their strategic direction towards the desired end state. They will create a plan for two week periods (sprints if you're using scrum) in the context of a Roadmap (a longer term plan for the delivery of valuable features to customers)

Lessons Learnt: Every Agile team will produce a lessons learnt report every two weeks and feed it back immediately into their own process (it’s called a Retrospective in Agile terms). It is a core part of an Agile team – to reflect and learn and to improve regularly.

Business Case: Of course every Agile team also starts with a Business case for the challenge they have been set to solve. Sometimes called an Initiative or Feature of a Product. This is created and owned by the Product Manager. This will always have solid measures as to how their progress will be measured and the return on investment established - these things however tend to be couched in terms of Value delivered and Outcomes not output or deliverables.

Detailed Requirements Document: These exi[st throughout an agile delivery. Each issue must be specified as to What is required and Who requires it and the Acceptance Criteria (tests) so we will know it's been delivered - but not How it is to be delivered. The scale is smaller. The issues are deliverable in two week itterations to control change. Requirements are specified in high level terms and 'just enough' to get the project started and an initial product into a customers hands. The detail is documented as the team deliver on the requirement, they document How the requirement was met and attach test outputs to the requiremnts document. The later requirements are documented in more detail as we learn and as the project progresses, folding in all the lessons and changes as we go to make the requirements more relevant and delivering constant value to the customer

RAID Log: The entire list of things an Agile team have to do is a Raid log - a list issues that need to be solved, in priority order focused on delivering value to their customer. Agile teams call this a Backlog.

Financial Forecast: Agile also requires a financial forecast. It is prudent to estimate, as with a traditional project what we expect to spend. The fundamental difference is in the certainty. An Agile team can provide more certainly as it's process is designed around the control of change and therefore on delivering value often and checking with customers and stakeholders on that value. Whilst we may have estimated a six month period to reach and end state and full value (and it'll probably be the same cost as a traditional project) we may stop sooner than that as what we have is 'good enough' or our return is not worth the continued spend, or (and this is usually the case) we want to continue as we are delivering huge value to our customer and uncovering new opportunities - but let's not get too excited here!

Resource Allocation: As with projects the Agile team is a group assembled to solve the business case. Same as a project team. The difference comes in the desire for teams to be given a set of tasks within a domain and focused on continuously solving issues in that domian and not dis-established and reformed into new projects and teams. There is great disruption in tearing down and reforming teams.

Summary

Waterfall and Agile project delivery process and artifacts are remakably similar. Each is designed to provide control and certainty of delivery. Each process is strict in its requirements to follow the prescribed process. The fundamental difference is in their design principles; Agile accepts change is inevitable and must be the cornerstone of the process to control delivery - Specify enough to get started, lock that in for two weeks, control the priority of issues and lessons learnt as you proceed and check-in with your customers, pause when they are happy enough or your costs outway the value you are delivering. Waterfall projects are designed to control and minimise variation and change - Specify everything up front, lock it into a plan and control scope, cost and timeline before delivering to the customer at the end.

Accept Change as inevitable and design your delivery process around it. Take all the project artifacts and ceremonies you have and refocus them around acceptance and inclusion of change. In fact you don't need to design a process - excellent processes already exist - any number of Agile delivery processes!