Why Software Projects are SO difficult?

If you are an analytical person, you will have a idea on the variables we have when we develop a software. If you are a business person, well, here are the words of a Senior Engineering Manager.

The idea behind this article is to show why it is complicated to develop great software in timely-manner and why we probably will never be able to fully optimize it to its maximum, but we can always thrive to be better in it.

It's important to note that all ideas here came from my learning over the years and critical thoughts on how things operate

Understanding the Variables

As someone responsible for coordinating multiple leaders, squads and projects, over time I created a solid idea on why some projects are more prone to success and others are faded to failure.

We tend to think that as software is based on top of more analytical and problem-solving skills, it has to be a exact science like mathematics. This could not be more wrong.

One of the more pleasure parts of creating software is the ability to mix these analytical characteristics with creativity. And as you may know, creative is not exact, it is the result of our minds wandering.

Said that, let’s started our conversation

Usual Project Dimensions

When we estimate project, we have a triad axiom that almost everybody knows:

“You have to choose two out of the three options: time, cost and quality”

This means that:

• If you want it fast and with low cost, don’t expect high quality
• If you want it fast and with high quality, don’t expect low cost
• If you want it low cost and with high quality, don’t expect to be fast

But that’s not the point of this article.

The point here is to explain why we are not able to optimally deliver a project at all

Before we started

All variables are expecting normalized values between 0 and 1 where 0 is failure and 1 is success

The objective is to understand its effects on each other and you can place your own weights

Software Delivery

When we develop a software, we have two major variables involved to be successful: skills needed and scope clarity

You can understand skills needed as the ability to solve and resources needed (tools and people)

For scope clarity is the knowledge to give directions on what is the problem and what result is expected

So here, we have our first mathematical expression:

Where,

SD = Software Delivery
NS = Needed Skill
SC = Scope Clarity        

Simple, huh? Let’s continue

But as I mentioned before, when you estimate a project, you have to consider the time, quality and cost associated with it, so here is the next expression:

Where,

SDE = Software Delivery Expected
SD = the result presented in the first equation

time, cost and quality with its own evaluations based on the decision-making before the project start, I’ll give an example using the ‘time’ variable, but it can be applied to all others

Time is equal to 1 if you deliver as expected. If you deliver later, than it should be less than 1        

Value Generation

Nothing should be developed other than to generate value to its client/customer/audience. So it is important to measure if what we delivered generated value based on its objective.

I define two types of Value Generation: Real and Expected

Value Generation (Expected) is what you understand before you delivered something that your audience will gather benefits after you deliver your piece of software

Value Generation (Real) is found when you check based on a series of metrics if your audience is gathering the benefits with your software delivery. So it’s the same to say

So if we are going to evaluate what is our value generation, we have the following expression:

Where,

VG = Value Generation
VGr = Value Generation (Real)
VGe = Value Generation (Expected)        

Project Delivery

Finally we can concluded that a project delivery is the sum of multiples software deliveries:

And the Project Delivery Success is the sum of multiple Software Deliveries Expected

Other Factors

As we are all humans, there are more forces that we can map or control in the world.

And in the world we live in, these factors influence directly in our results

Social and Emotional Factors

Fortunately (or unfortunately) we are not machines, so part of being human is that our social environment and emotions reflect in our results, actions and choices.

And as you know, software is basically developed (until now) by humans.

So we have to consider the effects of our cultural fit, group fit, internal values, individual satisfaction and individual interest.

All these characteristics certainly affect if we will be effective in solving problems and delivering software.

Other Variables

As wild as the world can be, there are variables we can control:

• Macroeconomy
• Death
• Sickness
• Conflict of Interest

And they impact directly in the results we want and cannot simply ignore that they have their part on it

Deep Diving

Now my idea is to get a little deeper into how all these factors influence our results together

Deep Diving into Needed Skills

When we have the skills, we can assume that this value is 1, but it’s unreal to expected that we know everything.

Usually we have an idea of what it’s needed to solve a problem and then we go from there. So basically our needed skills index will always be incomplete.

And here it becomes fun. To enhance this index, you have to understand:

• In which part of the spectrum you are
• What is your learning curve to get better in it
• If you are interest in learning this
• What level of information access you have to learn it

All these factors will impact if you will be able to learning and in which pace (quality and time)

Deep Diving into Scope Clarity

For scope clarity, it takes the same steps you saw above in a business context, but we will add a little pepper here, communication

Communication between two groups that have brains wired differently and different professional skills is one of the most complicated areas to solve.

If the communication between Business People (who knows what they want) and Software People (who knows how to do it) is not clear and aligned, well.. the project will not be successful.

Software Understanding != Business Understanding → Total Failure
Software Understanding ~= Business Understanding → Some Success, Some Failure
Software Understanding = Business Understanding → SUCCESS!        

Final Equation

As I mentioned in the previous section, our emotional and social are directly correlated to our results.

We also deal with scenarios where some aspects have more weight than others in terms of business and technical decisions.

So I created 3 weights to consider in the mathematical expressions:

Wt = Weight (Technical) - The weight of the technical aspect (needed skills) in the results
Wb = Weight (Business) - The weight of the business (scope clarity) in the results
Weight (Emotional) = The weight of the emotional factors in the results        

Needed Skill, Scope Clarity and Skill Development

Assuming that we always have something to learn and that we don’t know everything, we can use this expression

Skill Development

Where,

SkD = Skill Development (Technical)
LC = Learning Curve
IA = Information Access
PI = Person Interest        

Needed Skill

Where,

ND = Needed Skill
SkDt = Skill Development (Technical)
We = Weight (Emotional)        

Scope Clarity

Where,

ND = Needed Skill
SkDb = Skill Development (Business)
We = Weight (Emotional)
Comm = Clear Communication        

New Software Delivery Equation

Where,

SD = Software Delivery
NS = Needed Skill
Wt = Weight (Technical)
SC = Scope Clarity
Wb = Weight (Business)        

Wrapping-Up

All this article is an attempt to explain why we probably won’t be optimal in delivering software and why estimates fail so often.

When you add the human factor into account and it is so relevant in the results, it’s impossible to be extremely assertive and successful in terms of quality AND time AND cost.

And to make sure that after all these math and extensive writing, I'll try to summarize in a way easier to understand:

• If you have the skills needed, but you don’t have scope clarity, you have nothing
• If you have the scope clarity, but you don’t have the skills needed, you have nothing

In real world you will probably have none of them in completeness

If your decision is more based on Technical Aspects than into Scope, the weight of the business is lower - You can see that in refactors, performance optimization, etc.

If your decision is more based on Business than into Technical Aspects, the weight of the business is lower - You can see that in easy tasks that have tools created to solve it like excel calculations for example.

Besides being to complicated to deliver software projects, it does not mean that we should not try our best to achieve what we aim for and to make the world a better place (with software help)

Itiro Inazawa