God said: project uncertainty is good brother!

Don’t get me wrong. A sensible and thorough set of project objectives along with comprehensive requirements gathering and proper risk management is good for you.

Not just good, they’re essential to any project.

But that’s not so much the point.

I would say it’s quite impossible to remove uncertainty in the sense that’s impossible to know how much of it you’ve managed to prepare yourself for.

And that’s something surely not alleviated by the commonly, overly stringent and short-sighted measures of project success: the plain deliverance of a scope on time and on a budget. I would say there’s more to it in it. Let’s call it business value, for short, and picture it for the long haul, please.

Malcolm Gladwell points it out very nicely in an article for the New Yorker[i]. Resorting to Albert O. Hirschman’s Hiding Hand Principle, he recounts the issue of digging a mountain in a railroad project, featuring a majestic cost overrun. But this failure also became one of the drivers of the colonization of the West and, ultimately, kept the economic and political connections with the East and the cohesion of such a vast territory which, historians say, helped turn the US into a world power:

“Everyone was wrong. Digging through the Hoosac turned out to be a nightmare. The project cost more than ten times the budgeted estimate. If the people involved had known the true nature of the challenges they faced, they would never have funded the Troy-Greenfield railroad. But, had they not, the factories of northwestern Massachusetts wouldn’t have been able to ship their goods so easily to the expanding West, the cost of freight would have remained stubbornly high, and the state of Massachusetts would have been immeasurably poorer. So is ignorance an impediment to progress or a precondition for it?”

Of course, there are scholars who took issue with Hirschman’s Hiding Hand Principle, arguing for a more pervasive one, the Malevolent Hiding Hand[ii]. They have a point but, to me, they’re mostly jealous.

Let’s take the cherished example of software projects and the Standish Group CHAOS Report - chastising software projects outcomes since 1994. The first report made a considerable splash. Of the more than 8,000 projects studied only 16.2% were completed on time and budget, and with all the features and functions as specified. 52.7% were completed but over cost, over time and lacking all the features and functions originally specified, and a disturbing 31.1% of projects were abandoned or canceled.

Fortunately, things have much improved since then.

Now, let’s take a look at the 2015 report[iii]. For this report, the Standish Group studied 50,000 projects around the world. They’ve also released what they are calling the “Modern Resolution”, which defines project success as on time, on budget and with a satisfactory result.

So, delivering a project solely with all the features and functions originally specified ceased to be good enough to qualify for successful completion. One has to be pleased too.

Apparently, things were improving too fast. This 2015 report accounts for 29% of successful projects, 52% challenged, and 19% failed ones.

This change, of the definition of success, really brings forward the matter of business value, but in a kind of different way. What business value, the report’s business value? I digress.

Anyway, even in this report it’s funny to catch a glimpse of Hirschman’s Hiding Hand Principle when the Standish Group representative is asked why the figures about goal setting seem counter-intuitive, in that projects with precise goals are less successful than ones with vague goals, and answers like this:

“Only that which is unknown produces real value.  Having precise goals means you know everything in advance, which is usually not the case. IT is also always trying to align itself with the organization. However, as Chairman Jim Johnson would say, if IT and the organization are aligned, one or both are not going fast enough.”

Well, well, well, even the guys at Y Combinator say that “ignorance”, in the sense of an innocent lack of awareness of the trials that come with doing a startup is a prerequisite for success. Very much like the case of the railway project, it’s best for the founders not to know what’s ahead or they might not even try it.[iv]

As a corollary to this rambling piece I can’t help adding my own stanza to the “God Said” song from Leonard Berstein’s Mass, one that never ceases to amuse me:

God said, let there be plenty of hazards

to bring sunk cost in all of its forms

Forms such as overruns and slippages and blarghs

blarghs that have names and numbers and blamestorms

And it was good, brother!

And it was good, brother!

And it was good, brother!

And it was goddam good!

[i] Gladwell, Malcolm. 2013. The Gift of Doubt. https://www.newyorker.com/magazine/2013/06/24/the-gift-of-doubt

[ii] Flyvbjerg, Bent; Sunstein, Cass R. (2015). "The Principle of the Malevolent Hiding Hand; or, the Planning Fallacy Writ Large". Rochester, NY. https://www.nationalaffairs.com/doclib/20080516_196700602theprincipleofthehidinghandalbertohirschman.pdf

[iii] Shane Hastie, Stéphane Wojewoda. Oct 04, 2015. Standish Group 2015 Chaos Report - Q&A with Jennifer Lynch. InfoQ. https://www.infoq.com/articles/standish-chaos-2015

[iv] Stross, Randall. The Launch Pad – Inside Y Combinator. 2012. Portfolio Penguin. https://www.ycombinator.com/ . https://paulgraham.com/mit.html