The pitfalls and antidotes to drive business benefits with data science

Throughout my career in data and analytics roles, I have always cared deeply about the ‘so what?’ for the projects I was working on.? Don't get me wrong, I love the data and the creativity involved in solving problems with data and algorithms, but I also yearn for my work to be useful and really make a difference.

Over the years, we have seen data science and analytics become as critical as a Finance function and very few executives would question the need for our specialist skills and investment in technology to support our work.? But one challenge is harder to solve - what is the value data delivers to an organisation?

And, do senior leaders really care enough to want to attribute benefits to the work done by the data and analytics team?? We are after all an enabler, and rarely are we directly in control of how the outputs of our hard work are used by the customer facing business teams.

I know I certainly want to know how my team has contributed to the success of a business.? I want to be accountable for outcomes that drive positive change in the company I work for.? I want to feel the satisfaction that comes with knowing that I have made a difference and there is real value in my work.? And I don’t think I am alone.

The Challenges

I want to firstly explore the areas that make measuring the benefits of data work a challenge, before I reflect on some techniques and approaches that could help others in the pursuit of data benefit realisation.

Challenge 1 - Measuring benefits/changes

The first issue I often observe is that the business problem is not clearly described and hence the scope and direction of the work is not clear.? How can you determine what you are measuring if you cannot clearly articulate the business problem you are trying to solve?? If you can clearly identify the problem you are trying to solve and what impact the issues have, it becomes much easier to define the benefit, even if just as a sentence to begin with.

Even once you can define the business problem, you probably need to know what the current state performance is.? This is where you need to define the baseline or a benchmark so you can determine the level of change the solution contributes to the outcome.? This is a challenge I see regularly.? The issue is even more prominent when you are trying to develop a new capability or test something that is completely new to the business.? What is the current state or baseline for something that does not exist today?

And finally, I see people set the measures at the start of the work but are then reluctant to adjust or revise the measures as the work progresses.? It's common for analytics projects to have new insights emerge as we analyse the data, and this can lead to a change in direction for the work.? The initial hypothesis of the root cause for a problem may be disproved and hence the measures of success need to be adjusted accordingly.

Challenge 2 - Getting people excited about measuring outcomes

It's challenging to be an enabler in the business and for the most part, Data Science & Analytics teams aren't seen as directly customer facing with a P&L for accountability.

We need the business teams with the P&L accountability to work closely with data teams to ensure outcomes are defined and measured, especially during and after the ‘data product’ has been implemented.? High engagement is required between the many teams that need to be involved during the entire project or initiative, and that can be difficult to achieve with everyone busy and having other priorities at the same time.

The ultimate challenge is who is accountable for the benefits and measuring them?? Is it the team who owns the P&L?? Is it the team that developed the solution?? Is it the team who have embedded it into their work practices or deployed the solution to the end customer?? The answer is not straightforward and hence this challenge often stalls the measuring of outcomes and benefits for data driven projects.

Challenge 3 - There needs to be a change

One thing I have heard many times over is that we need “Actionable Insights”.? Have you ever really thought about what that means?? ‘Action’ means that someone does something proactive with the outputs of the work.? To me that means that someone does something different to what they are currently doing.

This is ‘change’ and I see frequently that all parties involved in data initiatives do not take enough time to consider the potential changes required for the implementation AND adoption of the solution to ensure that the benefits actually transpire.? We need to achieve a change in the way someone is working or a change in how a customer interaction takes place.? Now, I am not saying that change is easy but without a change of some kind, the outcome can only be sub-optimal.?

While there are probably other challenges people can point to, these are the key ones that I have seen consistently impact on benefit realisation for data and analytics work.

Potential antidotes to these challenges

Now let's look at what you can do to address these challenges and make some progress in demonstrating the value you can deliver with data and analytics.

Antidote 1 - Defining Measures

While the saying goes, “You can’t improve what you don’t measure”, it can be challenging to find a way to measure an outcome that is a combination of data science and business change.??

Defining the measures can be complex and direct attribution is not always possible.? There is nothing wrong with soft and fuzzy measures of success!? This is especially true when you are testing hypotheses or iterating on a solution to find an optimal balance between science and how people work.? If you are starting small and seeking to build knowledge and credibility, there is nothing wrong with measures such as “Positive feedback”, “Improved engagement” or “More willing to test an alternative work flow”.

And while qualitative benefits are valid, we often want to have quantitative measures in place to be confident of the benefits achieved.? One tip is to try and align your quantitative measures with business objectives to give them extra punch.? It is more meaningful to the business teams who will leverage the solution and it can make it easier for them to take ownership of measuring the benefits.

My three key suggestions when it comes to defining measures for data opportunities are;

1. Don’t try to measure everything!? Select 20% of the most likely opportunities that will deliver favourable benefits.? Choosing those that are really important to the business and could add measurable value to them will have a better chance of success.? Even better, they can tell a story about the work when it's hugely successful !;
2. Defining measures and measuring outcomes will be so much easier when you work closely with the people who will realise the benefits.? Collaborate not only to define the problem and objectives but as you iterate on the solution and need to refine the measures; and?
3. As you define the business problem and the potential solutions, aim to define 2 qualitative and 2 quantitative measures - no more and no less.? As you iterate on the solution, test and learn, these early measures will converge into clear measures that you will have confidence in as you move to implementation.

Antidote 2 - Focus on People and Change Management

While people are critical to a successful data science based solution, how the people collaborate is vital to enable benefits to be fully realised.? I know people will say ‘Duh!’ but I feel that we do need reminding that data driven business benefits are only achieved if it's a team sport.

If you can optimise the interactions between the many teams and people that need to contribute to the outcome, you will share knowledge at the appropriate points during the project and also build trust between people (as they feel involved when it's relevant to them).? We know that everyone is busy and we all have our own ‘day job’, however people like to be part of a winning team!? Share openly all aspects of the opportunity, good and bad, as I believe this accelerates team ownership of the outcomes.

When it comes to change management, it's an area I see often underestimated or overlooked completely.? Let's be clear, a successful data initiative means that someone has to do something differently as a result of the work (that’s the ‘actionable insights’ bit!).? Make sure there is some level of change as the data product is implemented.? If there is no change in how the business works, then the benefit is sub-optimal.

While the change management does not need to be complex or overly time consuming, there are a number of things that you can consider (for example);

• What would adoption of this solution mean to how people work?
• What issues could prevent this initiative from being adopted?
• Do we know ALL the people who could be impacted by this solution?
• How do we keep stakeholders updated and enthusiastic about the initiative?
• Could data quality impact the success of this initiative?
• Who is going to monitor, follow up and look for continuous improvements when the solution is implemented?

Antidote 3 - The Approach to Execution

I like to think of the benefits realisation journey as a process that can be embedded into how you scope and deliver D&A projects but with one difference… There is not a defined ‘end’ to the work but more a ‘measure and refine’ approach with continuous reviews as all teams involved learn what works and what does not achieve the desired outcomes.

Benefits realisation is rarely a pure ‘it's done’ activity.? Think of it more like a ‘measure and refine’ approach that engages people over longer periods of time.? Again, you don’t need this approach to all your initiatives but focus on the 20% that are really important to the business and most likely to deliver measurable value.

One final point in this approach is to ensure that you have at least 1 person from the business team (but 2 is much better) to take ownership of the outcomes, with you.? You can keep each other motivated, accountable and with a focus on continuous improvement.

Final words

The outcome is important (you want to feel like you are contributing to the business! ) but what is more important is the journey and trust you create with people through the process.? Benefits realisation is a team sport that needs trust and accountability to be successful.

My final recommendations on this topic are;

• Collaborate with many, many people - discuss openly all aspects of the opportunity, especially what outcomes you are striving to achieve.? This will ultimately accelerate knowledge and build trust between people;
• Have a Test & Learn mindset - there is no better way to drive to an outcome than learning together while you engage with data?
• Iterate on problems - make it safe to be wrong and empower people to ask questions and think outside the box when it comes to defining and measuring the outcomes.

Thanks for reading to the end ! Please feel free to contact me if you have comments or suggestions.