Measuring our impact

In our 2022-23 fiscal year, Farm Radio International measured that the radio programs broadcast through our projects had 24.1 million listeners. Our calculations showed that 4.8 million of those listeners were taking some form of action because of the radio programs they listened to — they were improving their farming practices, or trying something to improve their health, or simply planting more nutritious food in their kitchen gardens.?

As it turns out, we might have been underestimating our impact.?

This fiscal year, we measured 45 million listeners, with 22 million of those making a change in their lives.?

That’s a huge difference, so why the big jump in numbers??

First, let's explain how we get these numbers.

Measuring listeners

Farm Radio has pioneered a way of mapping the coverage of radio stations. Using the power of a station’s transmitter and the location and height of its antennas, we determine geographically where a station’s broadcasts reach.

We then overlay these coverage maps with population data maps based on each country’s latest census using an open spatial demographic dataset by Worldpop. These allow us to visualize using 3D maps how many people live in what areas, and estimate how many people live within reach of the station’s broadcasts. When we look at the people who live in that area, and are over the age of 15, that’s what we call our potential audience. This is also known as coverage.?

In short, it’s the total number of people who can listen to our programs.?

Then, using either household surveys, where our staff travel to various communities to ask specific questions within that broadcast zone, or mobile phone surveys, where people living within those communities are called and surveyed individually, we are able to determine exactly who is listening, and what they are learning from those programs.?

These include simple questions like how many episodes of the Farm Radio program they have listened to, or questions that dig into the program’s impact, determining people’s knowledge on the topics broadcast or what changes they have made because of the programs.?

We compare those questions about knowledge and practice to surveys we do before the programs begin, as well as to surveys we conduct outside of the listening areas, to measure our programs’ impact.

Using the mapping data we collect from the stations, we can then extrapolate to the entire area to get an estimate of how many people are actually listening, and how many people have made a change.

On average, we find that 40 to 60 per cent of potential listeners tune in to our radio programs regularly, and of those listeners, 20 per cent apply at least one new recommended practice.

2023-2024 results

So why the jump from 24.1 million listeners to 45 million listeners this year??

The answer boils down to a couple things.?

Usually, to determine our numbers from the previous year, we download a list of our partner radio stations who worked on radio programs with us over the past year.

We then pull the maps that we have produced for each of those stations, and control for any areas where those maps overlap, so that we aren’t double counting anyone.?

That gives us our coverage, or potential audience. We then apply the formula above: 40 per cent of potential listeners tune in to our radio programs regularly, and of those listeners, 20 per cent apply at least one new recommended practice.

It’s an accurate, if somewhat conservative, view of how many people are listening and making a change. And one backed up by more than a decade of research.?

But we’ve been making some changes in terms of how we collate our data.?

Before, we weren’t really able to take a close look at all of the 30-some projects we run in a given year together. And we weren’t able to compare projects when aggregating them on such a large scale.?

In some projects, like our groundbreaking Nature-based Solutions project, we are seeing upwards of 65 per cent of community members in certain countries reporting they are listening to the programs. In other projects, like the Platform project in Uganda, 67 per cent of the listeners we surveyed (which extrapolated means 3.5 million people in Uganda alone) are telling us they took up a new practice because of the programs.?

Now, thanks to a new tool in our Uliza Log (a web-based audio and data management system that we use to both track and evaluate radio programs), we are able to accurately report the actual results of each project.?

This “global results framework” auto-generates mapping data (using the same information as before), making mapping the stations easier, so we’ve been able to map more of our stations’ coverage than we ever have been. Then based on our surveys we can upload listenership, change in knowledge, and change in practice data.?

And, the tool can add them all up, so we see the real numbers of people we have impacted based on actual surveys we have done.?

This gives us a more accurate look at the impact of our programming.

And, as it turns out, that impact is bigger than we thought!

Learn more here