Five reasons why large IT programs fail

Introduction

Only 16.2% of IT projects are delivered successfully and on time, reports The Standish Group, a research advisory company. If delivering large IT programs on time and on budget is such a common challenge, what are the factors that lead to project mistakes and how can we learn from them??

There are many reasons, but they can all be traced back to human mismanagement. Based on my reading and experience in the field, I have grouped them into five categories:?

• Project and business misalignment.
• Humans overoptimism.
• Not enough planning.
• Not reusing common concepts.
• Neglecting software modularity.

When I began researching the common reasons for delays in IT projects, I was surprised to find parallels with other industries. My realization was influenced by the book ‘How Big Things Get Done' by Prof. Bent Flyvbjerg . In his book, Flyvbjerg examines both successes and failures of megaprojects from around the world, using the world's largest database that he and his team created. The database includes a wide range of projects, from IT initiatives to infrastructure projects like dams, airports, and railways. An intriguing insight from the aggregated data, is that when IT projects do overrun, they tend to do so by an average of 447 percent.?

As an IT leader, how do you avoid your project to overrun and result in budget increases of 400 percent or more?? The following paragraphs answer this question by exploring the most common mistakes and some actionable recommendations.

Project and business misalignment

Misalignment around business outcomes is one of the most common reasons for project failure. As a leader, you are responsible for assessing whether the project is connected with the organization's overarching vision. If there's a lack of alignment, the chances of success are significantly diminished.

Another crucial responsibility of a project leader is to foster a sense of belonging among contributors. This is achieved by communicating the project's mission and how it aligns with the organization's strategy. When individuals understand the project's purpose and its connection to the organization's goals, they are also enabled to make the right decisions themselves.

Recommendations:

• Connect your projects to a “Why”. No matter how big or small the project is. Ask yourself, what is the purpose of this project? How is it contributing to the wider purpose of the business?
• Involve senior or executive leadership in the review process to add an extra layer of validation. This ensures the project is aligned with the broader vision of the company.
• Communicate the purpose of the project regularly to all contributors. Repetition does not hurt!

Humans overoptimism

Chances are you’ve been in product planning meetings where you noticed that you and your colleagues overlooked many challenges and risks associated with the plan? It is remarkable how optimistic people are!?

Human optimism and overconfidence is a well-documented phenomenon, and is an important factor in project overrun and failure. When people plan projects, they tend to underestimate the time and resources required, and on the other hand overestimate their ability to overcome challenges.?

But presenting a delivery plan with low effort can also be deliberate. In the book Flyvbjerg discusses this phenomenon as “planning fallacy”. People keep cost and time down during the initial phases of the project to influence the decision for the project to be approved or go to the next stage. The Sydney Opera House is a perfect example of the planning fallacy, it took 10 years longer to complete.

Recommendations:

• Use historical data to estimate projects rather than “human estimations”. Empirical data from similar projects can also be valuable to correlate estimations.
• Use experienced professionals, they play an important role in producing accurate estimations as they bring their lessons learned and know-how from previous projects.
• Stop trying to slay the mammoth and break your projects into smaller chunks. This will make it easier to estimate time and resources, and to track progress.
• Do not underestimate potential risks and challenges in the project, identify and assess them earlier on. A straightforward method to identify risks during the design phase is to brainstorm with team members potential risks and challenges based on their expertise and past experiences, log any dependencies the project has with external individuals and systems. Additionally document assumptions and factors that could hinder the project’s successful execution.

Not enough planning

In the fast-paced world of IT, the idea that a project can't start without a meticulously crafted plan is often dismissed. The very act of creating a plan is, in itself, a crucial step in the project's development. This fundamental principle is frequently overlooked in large IT programs, where leaders often underestimate the value of comprehensive planning.?

Another misconception in detailed planning is that it kills innovation but who doesn’t see creativity in animation movies like Toy Story, Ratatouille or Monsters?? Pixar Animation Studios follows a methodological approach of planning, designing and testing their project without impacting the creativity of their films.?

An interesting example the book refers to in a parallel sector is the work delivered by the architect Frank Gehry. Gehry has a reputation for delivering projects on time and on budget, even when handling complex and innovative building structures. Key factors of Gehry’s success in delivering megaprojects is understanding and challenging the purpose of the project, planning the designs to the detail before starting the delivery. This approach of following a thorough design reduces the probability of unexpected events that could derail things during the implementation phase.

Recommendations:

• Don’t be a victim of the sunk-cost fallacy: Project leaders are sometimes forced to forgo the creation of a thorough, detailed plan in order to kickstart the project quickly. This urgency is driven by stakeholders to secure funding and resources.?
• Avoid delays and cost increase by identifying and validating project assumptions before initiating the delivery phase of the project.
• Similar to the previous point, identifying risks and their likelihood to avoid surprises later on.

Not reusing common concepts?

The Sydney Opera House is unique for its design and location, but what few people know is that its construction went monumentally over-budget. The failure to reuse concepts that were already employed in other similar architectures, such as the use of precast concrete shells, contributed to delays in construction. As a result, the Sydney Opera House experienced extensive delays from an initial estimate of 4 years to a staggering 14 years for completion.

Don’t recreate the same mistakes as the Sydney Opera House project team did! When designing a new software solution, be sure to carefully consider all of the options available to you, including the possibility of reusing concepts and patterns implemented in other similar projects. This can help you to avoid costly delays and overruns.

Another area where reuse becomes very useful is when analyzing large programs' cost and effort estimates. The IT industry has spent years standardizing estimation models and still today there is not a universal way to measure projects. A reliable and accurate technique is to estimate projects based on previous similar projects execution timelines.

Recommendations:

• A common mistake is to design every component of the solution without reusing existing concepts or patterns, this often happens when there’s not enough research during the design phase or personal vanity to reinvent the existing.
• Decouple the cost effort into different and more manageable modules of the solution. It makes it easier to identify the most expensive and time-consuming parts of the project.

Neglecting software modularity

What is common in solar power, wind power, fossil thermal power and electric transmission? These projects are modular in nature. For instance in a photovoltaic power station, solar panels can be installed independently from each other, allowing for an interactive and incremental approach to building the power station.

According to the Mr Flyvbjerg database they have the lowest likelihood of being overrun compared to other 25 project types available in the database. This parallel can also be found in IT, modularity in a software solution facilitates adaptability and risk management, also generating value from small pieces of work is now the new norm.

By breaking down the solution into different software modules, risk of failure is lower than having to wait for the final result of the complete solution. This adaptability is an advantage of software delivery, that can be more challenging to attain in other sectors that deal with physical projects.

Recommendations:

• Break down the project into different delivery modules, learn and take the lessons from previous repetitions, if something works keep it in the plan, if it doesn’t, simply remove and carry on with a new design.
• To limit the unpredictability of things going wrong, deliver value faster to customers by shortening the project release cycles in different iterations. The longer a project takes to deliver value, the more risks it encounters. As in other sectors, large programs are often canceled for reasons including market shifts, regulatory changes or employee turnover.