aes23 Meanjin - Day 3 Reflections
Katherine Trebeck , a political economist and advocate for economic system change, kicked off the final day with a challenge to evaluators to harness our unique role to drive system change. She cautioned us that while GDP grew over the last decade the spoils of this wealth were not shared. She stressed that we need an economy that meets the needs of the people and planet, and called this the wellbeing economy. She talked about the need for "predistribution", things like worker cooperatives and community wealth-building agendas, and more jobs that provide enough money and more dignity and purpose. So where do evaluators come in? Evaluators need to hold the current system to account and share the alternatives. Map the theory of change to a new system!
In the first breakout session I popped in to see Kate McKegg , Julian King , Laurie Porima , Samantha Togni , and Keira Lowther discuss the need to adapt and embrace new evaluative practices and conceptions of rigor that align with the complexities of today. I loved the painting that Sam shared, which was developed over a year through the collaborative act of evaluation. The symbols in the painting tell a rich and complex story that was also captured in Language - now that is rigour!
In the afternoon my colleague Shani Rajendra and I ran an interactive session to present and seek feedback on a set of principles we've been working on to help drive principled MEL practice. Look out for my article on the revised ones in the next few days!
The conference concluded with a panel with Andrew Leigh MP who recently established the Australian Centre for Evaluation (ACE) in the Australian Treasury. Prior to the panel, Minister Leigh presented a plenary in which he explained what a Randomized Control Trial (RCT) is and gave us an example of how RCTs can bust myths about medical interventions. Here are my early thoughts a) RCTs certainly have a place in our smorgasbord of evaluation designs b) However, linking in with the main things bubbling up this week [system change and centering Lifeviews] I just can't see how they are going help us get upstream of systemic problems C) Getting upstream is sorely needed in this scary and overwhelming time where we gallop towards a very uncertain future d) So, in conclusion, I reckon this renewed focus on RCTs may be useful, but it's not sufficient in terms of the sort of evaluation approaches that we are going to need to support and evaluate transformational change.
领英推荐
Panel members broadened the conversation, and Uncle Doyen brought us back to reality with an example of a man who lived in a tin shack and helped over 250 men coming out of prison to find jobs. This led them to get their own houses and cars and transform lives. He said we need to look to solutions that are already in communities.
Kiri Parata wrapped up the conference with a lovely closing address before the conference handover, and drum roll..... AES24 is coming to Melbourne, and I am delighted to be co-convening with Amy Gullickson . So the conference Owl made it home safely to Melbourne and my bookshelf.
I want to say a big thank you and congratulations to the Australian Evaluation Society , the fantastic presenters and everyone else who contributed to making #aes23BNE such a memorable event. What a hard act to follow!
Making a Difference | Conscious and Visionary Leader | Empowering Others
1 年Great summary, Jess. Thanks for sharing - felt like I was there! Totally agree with your conclusion that RCTs can be useful & valuable while still allowing plenty of space for more systemically-focused forms of evaluation. Hope all is well!
Systems and Program Evaluator at Just Evaluation Services (JESS), LLC
1 年Nice summary! I’m sorry to have missed AES this year. I concur with your RCT insight and would add that Scriven is smiling and that continued focus on RCTs in evaluation is taking us back a few decades in our evolution especially as interventions move to more holistic designs.
Partner, Market Access Research and Evaluation
1 年Great summary Jess. Agree with your comments on the challenges facing us. I am perhaps just a fraction more excited about the potential value of a focus on RCT's, as I believe it has the potential to focus attention on the idea of evidence based practice, something with potential for all system elements.
Social change leadership. Collaborative knowledge creation. Systems and evaluative thinking.
1 年Thanks for sharing these reflections Jess Dart. Your framing around the importance of systems level, upstream change and the strategies and tools required to shift these, in collaboration with community, is so relevant to our current context - the context that Katherine Trebeck painted. We need to step out of these evaluation epistemology wars; where methods are spruiked in a binary of 'always right' or 'always wrong.' We should take the framing used to describe measuring impact - as in, what works for whom, where, how, and in what circumstances and translate that to methodological thinking and methods in context. That is, we should first ask, what methods work for whom, where, how, and in what circumstances; next we should ask how success might be defined and by whom; and then, what methods are therefore appropriate to context and our definition of impact, keeping in mind who might benefit from them in contexts of differential power. Thanks as always for stimulating such great discussion.
Australian Centre for Evaluation, Department of Treasury. PhD candidate at University of Queensland
1 年Great wrap up Jess - thank you! Really looking forward to AES Melbourne 2024 and great to hear that you and Amy will be co-convening.