3 ways to overcome the challenges of outcome measurement

At Resurgo, we’ve spoken with 50 charities so far this year about the challenges they’re facing when measuring their outcomes. ?

The same themes crop up again and again. They’re challenges we’ve navigated ourselves during our own journey of growing the Spear Programme, and they’re challenges that might resonate with you. ?

Here’s our top three - and our suggested ways of tackling them.

1) Begin with your Theory of Change???

You might be collecting data, but is it the right data???

Many Impact Managers are working with inherited data sets, and lots of them.?

Maybe that’s you: handling data sets designed years ago for an outdated Theory of Change, perhaps to report to a certain funder or to conduct a specific piece of analysis. ?

That was fine at the time, but now, repeated exercises of this sort have accreted your data sets into a many-headed beast. What’s more, this beast now doesn’t yield the insights you need. Your data sets lack validity; they don’t shed light on the outcomes you’re trying to measure - they can’t quite be cut in the right way. It’s difficult to sort through what is a useful signal, and what is merely noise.?

In this instance, our advice to charities is to start by refreshing your overall Theory of Change. We do this through a coached team workshop, helping you establish clear outcomes that are held jointly across your organisation. Then we help you consider what data to collect, so that you’ll know whether and how these intended outcomes are being achieved. Our hope is that this process simplifies what you need to measure and how... a great relief for all involved.?

2) Balance thoroughness and completion???

You might be collecting the right data, but do you have enough of it??

Have you ever had the nagging feeling that the results of your client survey were a bit of a fluke? The responses showed a 98% client satisfaction rate – but did your organisation just get lucky? The?figure might well be (much) lower next time... and it feels uncomfortable seeing your colleagues touting your high success rate, because the data lacks reliability.?

Countering this feeling can be a delicate balancing act. We want to ask enough questions to assess whether we’re achieving our intended outcomes, while keeping the data collection process simple enough that sufficient people respond, and our?measurements are truly representative.?

This is more than just keeping surveys short. It’s also about how and when you ask people to fill them in; what incentives you offer; whether the questions are the least intrusive ones possible for what you need to know, and so on.?

At Resurgo we’re proud of our 92 percent data completion rate for young people 12 months on from finishing the Spear Programme – and we’re obsessed with keeping it that high. If you’d like to operate that way too, come and talk to us.?

3) Buy your team in???

You might be collecting enough of the right data, but who cares???

You’ve finally wrangled your Theory of Change and measurement framework into a state that you’re happy with. You’re collecting data that actually tells you how your organisation is progressing towards your intended outcomes. This is great for your impact report and funding applications – hooray! ?

But here kicks in our third challenge: your frontline colleagues remain strangely unmoved. They are spending precious time filling in forms for you, instead of directly serving the people or cause they are so passionate about, and are left feeling jaded - ?lamenting that it’s all a bit of a waste of time. ?

At Resurgo, we talk about data having ‘a seat at the table’. It doesn’t make decisions for us, but it needs to have a voice in our decision-making, from our frontline coaches to our Board.

That means two things: ?

• Investing in ways to make data easily accessible to lots of people beyond just the Impact Team, and, (arguably more importantly);?
• developing a culture where data is valued as a catalyst for undefended discussion.?

Having these things in place allows us to make outcome measurement both motivating and useful, and so to buy in our entire staff body. Whether our frontline coaches looking at their Spear Centre dashboard, or our Comms team perusing our latest data for an infographic, both know that our ultimate shared purpose is to make more of a difference to the young people we serve. ?

If you’re considering how to collect enough of the right data for your organisation, and have the crucial conversations that take colleagues with you, our Skills for Outcome Measurement workshop in September 2024 can help – we’d love to see you there.

Resurgo’s social impact consultants have been evaluating and improving the award-winning Spear Programme for over 20 years. This employability programme has been independently evaluated by the Government Data Lab to be highly effective. We offer impact consulting to a wide range of organisations across the social sector.?