How To Make Sure Your Software Project Succeeds

This is Part 1 of a 2-part article.

If you are involved in the creation of software you will know that it is notoriously unpredictable.

Whether you are a Developer, a Software Project Manager, or a Business User for a software project you will very likely know the experience of a software project that is running late.

Maybe some Sprint goals get missed.

Or a release doesn’t contain the expected functionality.

Or the software is buggy and unreliable.

It’s intensely frustrating and, sadly, really common.  The larger the project, the more likely it is to go wrong (as this article makes clear).

Why Do Software Projects Fail?

There has been a lot of research into this question.  Since so much of the world now relies on software, It would be very helpful if we could reliably develop it.

Firstly we need to define what we mean by success (and failure).  There is no definitive answer to this, but the most commonly used definition of success is software which is delivered approximately on time, within budget and which fulfils the stated requirements.  Some commentators add that it must also be accepted by its users.

Accepting this definition, the consensus is that the following are important factors required for success:

• Having a committed project sponsor
• Having clearly defined goals and objectives
• Clearly defined requirements, or adequate time allowed for requirements elicitation
• Having a project manager to oversee the project, working from a well-defined plan
• Utilising best practice software engineering principles (such as CMMI)

This is a summary, and many, many words have been written about this.  That said, it is a not unreasonable assertion that we know why software projects fail, and yet they still do, in droves.

To put it more simply, the old joke is that software is 90% complete, 90% of the time.

Why Do Software Projects Still Fail?

Despite all the research, the books, the professional bodies advocating best practice, and the clear negative impacts of failure, software is still a problem.  In the innovation space, so many founders view software as one of the biggest uncontrollable risks to their organisations.  If you’re not in the software industry it just seems like a black art.  A force of nature, as unpredictable as the wind.  Maybe your startup will ride gentle zephyrs to software nirvana, but maybe you’ll be tossed by storms of software overspend, or even dashed on the rocks of business failure.

Even if you’re an established business, software projects can throw up nasty surprises, wasting many hours and dollars to deliver disappointing benefits, unworkable interfaces and frustrated staff (and management).

There are three possibilities:

1. Software is inherently unmanageable.  No matter what we do, projects will continue to fail at a similar rate
2. Software development is a manageable process, but we have not yet determined the best practices which can lead to predictable success
3. The key factors for success are known, but they are not being implemented consistently.

I’m going to go out on a limb here and say it’s 3.

(if you don’t believe me, the available research pretty much backs up what I’m saying)

So why don’t we do what we know is the right thing?

Working on my assumption, that we know in principle how to manage software projects, but we fail to apply that best practice consistently, and this leads to failure, let’s look at why that might be.

Let’s also assume that most of the people involved in the industry, who are working on projects of this type, are not crooks.  There is a genuine desire to develop software within the allocated budget, that aligns with the required timescales and achieves the stated objectives.

Therefore, with the best of intentions, people are engaging in software projects with:

• Inadequate Senior Stakeholder buy-in
• Poorly defined or non-existent objectives
• Unclear requirements, or not enough time allocated to determine what those requirements are
• No project manager, or someone attempting to fulfil the project management function who is not able to do so
• Teams not utilising best practice

Not necessarily all of the above, but some of them.

In my experience, we get into these situations for a variety of reasons, but very often underpinning them is an inadequate budget or an over-ambitious scope.  And the root cause of this is that software is extremely difficult to estimate accurately.

The sponsor wants the best software for the smallest possible budget

The developers want to write the software, and they don’t really know how long it will take

The project manager doesn’t want to say the project shouldn’t go ahead, because they’ll be out of a job.

So we embark on projects and then find there isn’t enough time or money to do everything that’s required.  And furthermore, software projects often start with a design phase.  That design sets expectations on what ‘done’ will look like.  The developers are then expected to develop whatever the UX/UI designers have come up with.  Then as the end date looms and the software doesn’t seem to be finished according to the design, the pressure grows to ‘just get something developed’.  Standards go out of the window, people work long hours, things aren’t checked properly and we’re into the doom spiral of another software project going out of control.

How do we end the Doom Spiral?

What we need is a way to give us a realistic sense of where we really are.  A way of assessing how secure our project is, that allows us to measure our project, not just what percentage complete it is, but how confident we are about that assessment.  An evolving metric of how likely it is that the project will complete on time and within budget, and still meet its objectives.  Something which lets us make informed decisions about what to do next.

The answer, in fact, is quite simple.

It’s all about managing risk!

No, hear me out on this.  Most people’s experience of risk management is an occasional session with the project manager where everyone looks at a risk register and tries to think of things that could go wrong, and what could be done to stop them happening.

In truth, risk management is a vital tool in the Project Manager's arsenal.  But only if it’s done properly.  Not just a dry exercise that people pay lip service to but otherwise ignore.

There needs to be a way of assessing project risk that allows us to focus on fixing the things we can fix.

We need to give senior stakeholders a realistic assessment of what could go wrong so they can make informed decisions.

We need a way to push back on decisions that might seem reasonable but push us closer to serious problems.

Stay tuned for Part 2, coming soon.