Software Development - To Measure or Not To Measure

OK, this was humbling! Last week, I pulled together a group of the fractional CTOs that Pilgrim Consulting integrates into our software development projects and opened with a conversation about measuring project success. As a backdrop, I’d just written an article about metrics as an absolute requirement for project success two days earlier (Measured Success).  

The conversation started where you’d expect it to - defining the business goals and justification of the project before diving into the “What?” and the “How?” of the project. So far, so good - my little glass house of biased perspective was unthreatened. Mendy Donin from Meticulous Growth succinctly spoke to the importance of understanding the goals for the project in order to ultimately build something that meets an actual need.

Another of my favorite Fractional CTOs, Craig Utley, spoke briefly about the challenge some project stakeholders might experience if they’re tasked to undertake a project which is outside of their core strengths and competencies. He basically made the heart and soul of the case that Pilgrim Consulting uses for why we consider our leverage of Fractional CTOs to be central to our differentiation and why we get things done and deliver quality on-time, on-budget, and on-scope. So far, so good. My little world remained comfortable up to that point, and my model remained affirmed. I remained happy.

Then...it happened.

Paul Irvine invaded my mental safe space with an idea that contradicted where I wanted the entire conversation to go!

Success metrics, sure….but not really. “I prefer outcome based management - this is how much we want to spend, and this is the outcome that we want to achieve. Then, we work together, collaboratively, to achieve that outcome.” [The Pentagon calls this “design-to-price”... “I want $10 billion worth of aircraft carrier. I don’t care if the $11 billion version has twice as many features. Give me $10 billion worth of aircraft carrier, or I’ll find someone who can.”]

Oh NO! PMI would just not agree! I must steer the ship in another direction! I turned back to Craig to ask (implore really) - Craig, don’t corporate stakeholders need a structured means of measuring and tracking progress on the project?

“Of Course,” Craig began to my relief, “CFOs want everything measured!” Still, for the overall health and management of the project, I agree with Paul. If they want to measure velocity, then fine - I don’t really care one way or the other. At the end of the day, the organization wants to achieve an outcome, and an outcome based management strategy will be much more effective than a bunch of reports and charts. Outcomes are what they look for FIRST. Metrics are what they look for AFTER they’re sure the outcomes meet their expectations.

Mendy concurred, indicating the importance of staying aligned with project stakeholders through constant feedback. His thoughts on feedback from stakeholders aligned with similar thoughts shared by Craig moments before.

So, what gives? To measure or not to measure?

So, The Bambino (Babe Ruth) was famous for nearly perfect bat mechanics. His swing was an artform. His instinct for what pitch is coming next and where it will be was out of this world. When asked how he hits home runs, he replied - I swing as hard as I can …” All the disciplines of great hitting had become second nature to him. The analysis of the game, situation, players, everything had reached an instinctual level.

If a rookie comes to the plate and tries to follow The Babe’s formula for baseball success and “Swing as hard as he can,” nothing but humiliation will follow.

The same is true in software. Too many project managers attempt to operate “agile”, when, in fact, they are truly doing little more than being slobs. They try to operate outcome based management, but they only manage to create a mess.

At Pilgrim Consulting, we have what I consider the best of both worlds. We measure. We track. We do burndown charts and velocity reports. This happens at the team level with the scrum master or project manager.

We augment these “field commanders” with fractional CTOs - “Jedi Masters” - technology and operations leaders with decades of battle wounds and trophies. Every project at Pilgrim Consulting Group incorporates the oversight of fractional CXOs... CTOs, CIOs or COOs, depending upon the specific situation. Coordinating the two strategies has elevated our game and dramatically increased our batting average.