Scrum Is A Danger To Your Product.

Scrum is, as Scrum.org proudly points out, the “de-facto standard for how teams deliver software”.

It is blindly followed by 12m+ adherents on a daily basis, but where did it all start? 

And why is it potentially preventing your product from achieving Product-Market Fit?

Well, it's not that obvious at first, as Scrum’s principles make a lot of sense:

1. Individuals and interactions over processes and tools, so that teams & different stakeholders prioritised collaboration & communication.

2. Working software over comprehensive documentation, to focus on delivery — actually shipping software quickly to customers.

3. Customer collaboration over contract negotiation, to ensure that teams were speaking to the customer & understanding their problems & frustrations, rather than just mapping out solutions managers had come up with themselves.

4. Responding to change over following a plan — self-explanatory.

A Fatal Flaw

Yet Scrum has one fatal flaw.

A flaw that all its practitioners, Scrum trainers, Agile coaches & the like conveniently ignore.

It fails to address the one thing that really matters:

Helping us determine whether we are building the right thing.

Consider the following facts:

?? Only one third ?? of the ideas tested at Microsoft improved the metric(s) they were designed to improve (Kohavi, Crook and Longbotham 2009). 

?? Success is even harder to find in well-optimized domains like Bing and Google, whereby some measures’ success rate is about 10–20% (Manzi 2012)

?? Fareed Mosavat, Slack’s Director of Product and Lifecycle tweeted that with all of Slack’s experience, ?? only about 30% ?? of monetisation experiments show positive results; “if you are on an experiment-driven team, get used to, at best, 70% of your work being thrown away. Build your processes accordingly” (Mosavat 2019

?? Avinash Kaushik, Product at Google, wrote in his Experimentation and Testing primer (Kaushik 2006) that ?? “80% of the time you/we are wrong about what a customer wants.” ?? Mike Moran (Moran 2007, 240) wrote that Netflix considers 90% of what they try to be wrong.

(Source: “Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing” by Ron Kohavi, Diane Tang, and Ya Xu)

In my experience, these figures sounds about right. Strong customer insight can improve success rate - particularly at the start when it is easier to find bigger wins - but every time I have run a digital experiment 80-90% fail, or the results are entirely unexpected. 

Consider also the fact that we are talking about mature, well-established companies with known markets & known customer bases.

This isn’t just a startup rushing around trying to work out its target customer, the problem space & what their business model might be (where we would expect failure rate to be even higher!)

These are top-performing technology companies with unparalleled experience & talent.

Therefore, how can we use a framework that fails to directly address this reality?

Why We Can't Just Use Scrum + Something Else

Unfortunately, humans tend to be good at finding a simple solution to a nuanced problem. 

This is usually the case with Scrum: 

That whatever Scrum as a methodology may promulgate (e.g. “Don’t just focus on velocity”), the original purpose of Scrum as a tool for delivering useful software has been bastardised and misinterpreted to simply help teams deliver software (whatever the impact - or lack thereof - that software may have on the customer and on the business).

If we add to that the fact that anything from 70-90% of our product ideas will fail, it is downright dangerous for you as a product leader & as a business to not focus relentlessly on answering the question:

Are we building the right thing? 

And how do we know we are building the right thing?

And that, if we don’t know we are building the right thing, to put all of our resources into answering that question so that we aren’t adding useless features to a bloated product and, as a consequence gradually alienating our customer base?

6 Steps To Improve Your Product Process

In my experience, that means taking a clear stand on the following:

1. Stop measuring developer velocity. “What gets measured gets done”, as the saying goes (usually at the expense of the business metrics that really matter: usage, retention & revenue)

2. Question every ritual or practice you employ as a team e.g. retrospectives are useful to create a culture of continuous improvement, but do your retrospectives deliver this outcome in practice? Or do you just go through the motions?

3. Define a clear company-wide strategy to create a framework for your product team/s to make decisions about what they should - and should not - be working on (what are the broad “guardrails” for them to operate within). Watch our step-by-step guide for doing this in action here

4. Define a clear hypothesis (expected outcome) before you work on anything, so that there is always a conversation about whether the feature is worth it, or whether there is an easier way to deliver the same outcome. Watch how our students craft OKRs & hypotheses here

5. If you don’t have the architecture for running accurate experiments (for example, multiple product teams being able to run A/B tests across reliable cohorts at the same time), then find a qualitative way of measuring results

6. Review the outcomes & share those outcomes with other product leads to share your learnings & encourage a culture of experimentation (and the inevitable failure, coupled with the patience required for success) that is an integral part of product success

-

You could call this “being Agile”, but - then again - Agile tends to be translated into Scrum. And Scrum tends to be translated into meaning “build lots of things quickly, regardless of the outcome”.

Best to just call it nothing. Or simply your "product process".

And pick what makes sense for your specific context, focusing always on that one question that really matters:

Are we building the right thing? 

---

Want to learn our 6-step product playbook for building the right thing?

You can start with Step 1 - by crafting a product vision in practice - by watching our free training here.