Could A New ISO Standard and Old Foe Solve the Gordian Knot of Agile/DevOps Sizing and Estimation?

The following comes with a warning! I am about to suggest something that many in the agile world will hate. Even as I type, I can already imagine friends in the agile community shouting, “Dave, what are you doing, we thought you were one of us?” I can also image people who are anti-agile taking the following as justification of the failings of agile. Well, they would both be wrong. I am still an agile evangelist, but I am a realist, and we do have a problem with agile when it comes to sizing and productivity measurement.  

 Agile uses a Wide Band Delph technique for sizing, which is based on gaining consensus around a set of requirements, usually user stories, which are relative to each other. Humans are naturally good at relative sizing, for example, who is taller, which piece of cake is larger, etc. We are poor at absolute sizing, i.e., tell me how tall that person is in feet and inches, or how many grams is that carrot cake. 

 Let look a bit deeper with a simple example. Team A is a 5-person team who use story points to size the backlog to help plan the sprints. Note I said size and not estimate – they push back when someone asks “how long”. They have settled down on a velocity of 25 SP (Story points) per sprint.

 Figure 1- Team Level Story Sizing

Team A is part of a larger product group, Product X, which has 4 teams, all doing story point sizing relative to their own backlog. Product X product owner (PO) is OK with the relative team sizing approach. The PO works with the teams on feature release and sprint planning and does not interfere with how they size. The PO does not ask for a single velocity for all 4 teams.

Figure 2 – Product X Multiple Teams

But there is an ill wind blowing, that will impact Team A, indeed all agile teams in the organizations, and it’s a problem that has nothing to do with agile – it’s the human desire for certainty, a need to know when or how much.

I have that desire, you have it, we all have it. Don’t believe me? When was the last time you asked a Uber driven when they will pick you up ? or a waiter when will the food arrive, or a builder when will the work be done and how much? See we all ask for estimates, which means someone has to think about productivity.

So we should not be surprised when the day comes when the CEO in our case study asks the CIO what is IT productivity level?.

Now the Product X product owner has a problem, as does every product owner, application managers, just about anyone in the organizations who looks after agile teams; how are they going to measuring productivity across teams if they are all using relative sizing?

At this point, the cry will go up from the agile teams, the CEO does not get agile, and if the CIO ask us to supply productivity data, they don’t get it either. This may be true, but telling the CEO or CIO he or she is wrong to ask and just doesn’t get agile and DevOps it is a CLM - Career Limiting Move. 

So, Team A has to move away from agile relative sizing, that works, to agile absolute fudge metrics – “fudge verb: a vague or inadequate way, especially to conceal the truth or mislead.”

Let leave the example there. However, before we go any further I need to put my hand up to a blunder I made when I was a Gartner analyst, yes you did hear that right – an analyst got it wrong. The above scenario was so common I wrote a paper on it. The paper talked about normalising story points. Sadly, the paper was used by management not for the teams benefit, but to get senior management of middle management backs. I had created a Frankenstein monster and burden for the team, Mea Culpa.

The issue of agile sizing and estimation is not limited to the scenario just described where a CEO starts asking the awkward question on productivity. It also comes up when application managers are dealing with distributed development centers, when scaling agile using frameworks. It is also a significant issue when vendor and sourcing management have to access suppliers regards productivity. In short, its an enterprise agile challenge.

An interesting variation of the problem is when agile improvement initiatives run into the issue when trying to show how a new tool or practices can benefit the organization. As one DevOps Community of Practice lead put it “We were so sure of ourselves, and so green,  that when the CIO asked for evidence before signing the cheque for $3M of new tooling, somebody in the room actual said, but why? don’t you trust us?.”

Currently, organizations use a mix of strategy to deal with this challenge. Some, a small few, have changed their attitude at the Cx level regards internal teams and with suppliers and stopped asking for absolute productivity measures electing to focus on outcomes instead. Some organizations have developed their own story sizing based on absolute sizing frameworks, but these miss the point of the benefit of relative sizing to the teams. Sadly, most organisation are fudging agile productivity metrics.

But what if there was a way to having our cake, relatively sized, and eat it? It is time we introduce the villain of the piece and my old foe – Function Points Analysis (FPA).   

A long time ago in a galaxy far, far away.... I did my MSc thesis on “Function Point Analysis in Real-Time Systems.” I was actually going to do a project on “The Dangers of Fly-by-Wire Without Pilot Feedback” but the project crashed two weeks before I was meant to start – oh the irony.  

To say my thesis project was boring would be an understatement, but it did earn me a distinction and 1inch space at the British Library Archive – yay me. When my MSc supervisor asked would I like to turn it into a Ph.D. I politely declined saying I’ll rather gnaw my own leg off. 15 years later, I end up trying to use function point analysis and Scrum – the result of foolishly boosting to my CIO about my MSc.

 FPA and I have had this love-hate relationship until I joined CISQ where at last, I could see how to rehabilitate function points, make it less boring, less of an overhead – by not doing it. Well to be more precise, making it invisible, automated and part of the toolchain. To have our absolute sized FPA cake and eat it. 

This is where the new Object Management Group? ISO standard, developed and published by the Consortium for IT Software Quality?, comes in ─ “ISO/IEC 19515:2019 Information technology -- Object Management Group Automated Function Points (AFP), 1.0”

We can use a hybrid of relative and absolute methods using ISO 19515 embed into the team's development or application portfolio tools. Allowing relative story point sizing to be used correctly to help plan sprints and work with the product owner. While standardised functional size productivity data can be collected in a fair and consistent way automatically across the enterprise.

 Figure 3 – Hybrid Measurement Model

 This would not be duplication as the approaches would be aimed at two different goals with real benefits to the team and management. 

 ?   The team can use story points correctly; for sizing their backlog and planning their sprint without having to normalize or fudge the story point process to allow management to compare across teams. 

 ?   Teams can stop story sizing altogether if they want to, if they feel there is no value in it, for example, when the story size has coalesced. But productivity data can still be collected.

?   It can help see the big picture, defect density across domains and multiple teams, or average release effort of the DevOps toolchain. 

?   Gives CoP/Guilds common datum for improvement and experimentation they can use to form part of a business case for a new tool, skills training, etc

?   Allows teams to compare maturity using a common and consistent datum, for example, the percentage AFP that were tested automatically or the AFP through-put 

?   3rd party sourcing team can focus on value without having to provide endless metrics to the bid team on productivity to win a contract.  

Of course, this approach is open to abuse, all measurement can be gamed, and all KPI can be misused. But its harder to do when the measurement is automated and based on a standard. 

Management can, and some will use the AFP to pressure the teams for higher productivity; however, they likely do it anyway – at least now there would be reliable data to have the conversation around. 

And we have to clear code size does not equal value. The best coder I ever worked with, was amazing with C++ pointers and cut about ? of the code as the rest of us and delivered about 4 times the value. But this is the beauty of the hybrid approach, by combing methods we can cross check. Teams trying to game the system by cutting extra code will find the actual release less value compared to the teams that maximize the amount of work not done –– Agile Principle 10 Simplicity.  

You may have noticed I keep refereeing to the team. Management should never use productivity measure to single out the individual – once you start doing that you no longer have a team. If a team wants to use AFP data internally with individuals for mentoring and improvement, that’s a team decision.

In summary

Agile and DevOps is going through its teenage angst years, and question on productivity are not going to go away – no matter how many times we say, “but we are agile.”  Some organization will listen to their teams, but only a few, so what about the rest? 

 Agile and DevOps is mainstream and living with the realities of the cooperate world – and that’s not always pretty. 

My view is its better we in the agile and DevOps community take ownership of the problem rather than have a non-agile approach pushed onto us.  Going to the PMO or production function and saying we have a solution that is consistent, automated, and fair puts us in the driving seat – and that’s the best place to be on your agile journey.    

Dave

Come and see us at