Monitoring and Evaluating the Just Transition in Africa

Author: Cara Hartley (Director and M&E practice area co-leader)

I recently had the privilege of attending the AfrEA conference in Kigali, Rwanda. AfrEA is the African VOPE (voluntary organization of professional evaluators) and this was the first time I got to attend the biennial conference.

I was presenting on the Monitoring, Evaluation and Learning Framework for the Just Transition in South Africa (soon to be launched by the Presidential Climate Commission!). The feedback from fellow conferencegoers – from Africa and beyond - was very instructive.

Here are three lessons for those of us who are doing M&E pertaining to the Just Transition.

1. What do you hear when I say “Just Transition”?

It has become clear that different people hear different things when one says “Just Transition”. When the term originated in the United States, it was focused on protecting workers’ rights in the transition away from carbon intensive sectors. And yes that is part of it. But for South Africa, we define it more broadly.

For South Africa, “just transition” is about the social and economic consequences (so: people) of both mitigation and adaptation to climate change. It is not just about avoiding harm, but about using the transition to make things better: transforming our wrecked society to a more just one. See https://www.climatecommission.org.za/just-transition-framework. (You can criticise our definition in various ways too. Which is a topic for another day.)

A quick Google suggests that South Africa’s more all-encompassing use of the term is commonly used internationally, but we should still communicate it deliberately and clearly to make sure we are properly understood when discussing this work internationally.

I’m sure a lot of people know this full well, but I was surprised, because misunderstandings seemed to persist even though I explained what we mean by the term as part of my presentation.

2. What do you hear when I say “evidence”?

In drafting the Monitoring, Evaluation and Learning Framework for the Just Transition in South Africa, we were dragging along some baggage.

• In the South African public sector, many associate M&E with compliance-oriented monitoring – or punitive, counter-productive accountability.
• In NGO circles, many associate M&E with the inappropriate demands of international donors who don’t understand the context and but must be appeased to keep the money flowing. I’m told that some refer to the M&E staff as “enemy” – and for good reason! And using alternative acronyms – like MEL to include learning – does not necessarily mean a real difference in approach.
• The private sector – a really critical audience for Just Transition MEL –use different terms and methodologies to answer the kinds of questions that the development sector answers with M&E.

Furthermore, we were at pains to ensure that a broad range of data and information can be deemed potentially “valid” for understanding Just Transition progress. Not just monitoring reports or evaluations. One of my colleagues has been using the word “tracking”, but to me that sounded too much like monitoring without evaluation.

So we went for “evidence for decision making”. Our audience is “those who are working with evidence about the Just Transition – from generating it, to interpreting it, to using evidence for decision making. It promotes a shared framing of the issues for which evidence is sought, ensuring robust evidence for well-informed action.”

I was surprised however, to hear from an AfrEA audience member what she hears when I say “evidence” rather than M&E. In her context – mainly the German development sector - they assume “evidence” means exclusively quantitative evidence. Indicators. Even experimental / randomised controlled trial-based impact data.

So whereas we (authors of the Just Transition MEL framework) are attempting to shake the narrow understanding and baggage associated with the term “M&E” by rather talking broadly about evidence – in her context that would backfire. They would not realise I am absolutely including qualitative evidence – more critical than ever for M&E of the transformational change required for the Just Transition (see https://www.cif.org/tclp).

3. What do you hear when I say “complex”?

As an evaluator, I’ve walked a journey with complexity. My first steps were a paper shared by @Matodzi Amisi called “Taking responsibility for complexity” (Harry Jones, 2011), and the 2008 article in the journal Evaluation, in which @Patricia Rogers explains complicated vs. complex aspects of interventions so clearly.

In our M&E practice at PDG, our articulation of a theory of change for the LandCare programme was very enthusiastically received by our clients in the Western Cape Department of Agriculture. The theory of change simply showed that the programme has several mechanisms that produce outcomes, depending on implementation decisions in the context. The theory of change diagram is a map with the three outcome pathways, not all of which are relevant in each case. The LandCare implementers assessed what is needed in each context and activated one or more mechanisms with the intention of achieving the desired outcomes in each chosen pathway.

The complexity-aware monitoring approach that we developed for the Future Cities South Africa programme (see my earlier LinkedIn piece) felt like a breakthrough. Because it allowed for regular adaptation of goals, and was flexible about means of verification, it suited the realities of technical assistance to a complex institution, much better than conventional M&E.

However, complexity in the Just Transition is at a whole different level. It is intersectional, multi-sectoral, whole-system, multi-disciplinary, and more. The way the Just Transition has been framed feels like a pendulum swing reaction to oversimplification of development. Riding the pendulum will give the most level-headed M&E practitioner whiplash.

But M&E is only useful if it brings stakeholders along. So we need to start with something simple – as simple as a linear theory of change. Woefully inadequate to express the complex reality! But we must start there. Something that non-experts can get behind. So we can build consensus on the basics, through a participatory approach. And we build up complexity as we (and stakeholders) gain appetite (and courage).

Let’s engage

It took me a while to write this. I am so aware of how little I know about this. It is harder than usual to put myself out there. But that is the fourth lesson – most of us are new to this intersection of M&E and the Just Transition. Very few of us have broad AND deep expertise in this. And the need is so great. If not us, then who? So we need to share as we learn; and we need to seek out resources – including each other – so that we maximise our collective wisdom.

I would love to hear others’ inputs and lessons. Join me in the SAMEA Community of Practice for M&E for a Just Transition, or post something in the comments.

Big thank you to Timothy Leslie Leslie who provided valuable inputs on the draft of this post.