aes23 Meanjin - Day 3 Reflections
Clear Horizon team members at the Gala Dinner

aes23 Meanjin - Day 3 Reflections

Katherine Trebeck , a political economist and advocate for economic system change, kicked off the final day with a challenge to evaluators to harness our unique role to drive system change. She cautioned us that while GDP grew over the last decade the spoils of this wealth were not shared. She stressed that we need an economy that meets the needs of the people and planet, and called this the wellbeing economy. She talked about the need for "predistribution", things like worker cooperatives and community wealth-building agendas, and more jobs that provide enough money and more dignity and purpose. So where do evaluators come in? Evaluators need to hold the current system to account and share the alternatives. Map the theory of change to a new system!

In the first breakout session I popped in to see Kate McKegg , Julian King , Laurie Porima , Samantha Togni , and Keira Lowther discuss the need to adapt and embrace new evaluative practices and conceptions of rigor that align with the complexities of today. I loved the painting that Sam shared, which was developed over a year through the collaborative act of evaluation. The symbols in the painting tell a rich and complex story that was also captured in Language - now that is rigour!

In the afternoon my colleague Shani Rajendra and I ran an interactive session to present and seek feedback on a set of principles we've been working on to help drive principled MEL practice. Look out for my article on the revised ones in the next few days!

Sharing the principles

The conference concluded with a panel with Andrew Leigh MP who recently established the Australian Centre for Evaluation (ACE) in the Australian Treasury. Prior to the panel, Minister Leigh presented a plenary in which he explained what a Randomized Control Trial (RCT) is and gave us an example of how RCTs can bust myths about medical interventions. Here are my early thoughts a) RCTs certainly have a place in our smorgasbord of evaluation designs b) However, linking in with the main things bubbling up this week [system change and centering Lifeviews] I just can't see how they are going help us get upstream of systemic problems C) Getting upstream is sorely needed in this scary and overwhelming time where we gallop towards a very uncertain future d) So, in conclusion, I reckon this renewed focus on RCTs may be useful, but it's not sufficient in terms of the sort of evaluation approaches that we are going to need to support and evaluate transformational change.

Panel members broadened the conversation, and Uncle Doyen brought us back to reality with an example of a man who lived in a tin shack and helped over 250 men coming out of prison to find jobs. This led them to get their own houses and cars and transform lives. He said we need to look to solutions that are already in communities.

Kiri Parata wrapped up the conference with a lovely closing address before the conference handover, and drum roll..... AES24 is coming to Melbourne, and I am delighted to be co-convening with Amy Gullickson . So the conference Owl made it home safely to Melbourne and my bookshelf.

I want to say a big thank you and congratulations to the Australian Evaluation Society , the fantastic presenters and everyone else who contributed to making #aes23BNE such a memorable event. What a hard act to follow!


Trina Hinkley

Making a Difference | Conscious and Visionary Leader | Empowering Others

1 年

Great summary, Jess. Thanks for sharing - felt like I was there! Totally agree with your conclusion that RCTs can be useful & valuable while still allowing plenty of space for more systemically-focused forms of evaluation. Hope all is well!

Ralph Renger

Systems and Program Evaluator at Just Evaluation Services (JESS), LLC

1 年

Nice summary! I’m sorry to have missed AES this year. I concur with your RCT insight and would add that Scriven is smiling and that continued focus on RCTs in evaluation is taking us back a few decades in our evolution especially as interventions move to more holistic designs.

dean mcdonald

Partner, Market Access Research and Evaluation

1 年

Great summary Jess. Agree with your comments on the challenges facing us. I am perhaps just a fraction more excited about the potential value of a focus on RCT's, as I believe it has the potential to focus attention on the idea of evidence based practice, something with potential for all system elements.

Dr Melinda McPherson

Social change leadership. Collaborative knowledge creation. Systems and evaluative thinking.

1 年

Thanks for sharing these reflections Jess Dart. Your framing around the importance of systems level, upstream change and the strategies and tools required to shift these, in collaboration with community, is so relevant to our current context - the context that Katherine Trebeck painted. We need to step out of these evaluation epistemology wars; where methods are spruiked in a binary of 'always right' or 'always wrong.' We should take the framing used to describe measuring impact - as in, what works for whom, where, how, and in what circumstances and translate that to methodological thinking and methods in context. That is, we should first ask, what methods work for whom, where, how, and in what circumstances; next we should ask how success might be defined and by whom; and then, what methods are therefore appropriate to context and our definition of impact, keeping in mind who might benefit from them in contexts of differential power. Thanks as always for stimulating such great discussion.

Eleanor Williams

Australian Centre for Evaluation, Department of Treasury. PhD candidate at University of Queensland

1 年

Great wrap up Jess - thank you! Really looking forward to AES Melbourne 2024 and great to hear that you and Amy will be co-convening.

要查看或添加评论,请登录

Jess Dart的更多文章

  • Is evaluation stuck? Challenges for Evaluating Place-Based Approaches

    Is evaluation stuck? Challenges for Evaluating Place-Based Approaches

    Evaluation holds the promise of helping us learn about what is/is not working so we can adapt and achieve more impact…

    21 条评论
  • Rethinking Evaluation to Support Systems Transformation

    Rethinking Evaluation to Support Systems Transformation

    In October 2023, Clear Horizon launched the Systems Transformation Unit. The purpose of this unit is to actively…

    50 条评论
  • Principles for MEL [version 3]

    Principles for MEL [version 3]

    Jess Dart, Chief Evaluator, Clear Horizon. Measurement, Evaluation and Learning (MEL) is on the rise.

    12 条评论
  • aes23 Meanjin - Day 2 Reflections

    aes23 Meanjin - Day 2 Reflections

    Continuing the theme of Lifeview that Maggie Walter Kicked off on day 1, Donna Mertens implored us to examine our world…

    2 条评论
  • aes23 in Meanjin - Day 1 Reflections

    aes23 in Meanjin - Day 1 Reflections

    So aes23 has kicked off with a provocative and sparky day. I love these conferences! After being welcomed to Country by…

    4 条评论
  • Principles for Measurement, Evaluation and Learning (MEL): Round 1 reflections

    Principles for Measurement, Evaluation and Learning (MEL): Round 1 reflections

    A few weeks ago, I posted a draft set of principles for Measurement, Evaluation and Learning (MEL). I was staggered by…

    8 条评论
  • AES 22 conference reflections - day 3

    AES 22 conference reflections - day 3

    The final day of #AES22ADL kicked off a provocative session by Amy Gullickson, talking bravely about soul and…

    2 条评论
  • AES 22 conference reflections - day 2

    AES 22 conference reflections - day 2

    The first theme that stood out for me on day two of the conference, was that of government learning. Katina D'Onise…

  • Shared Measurement Versus Evaluation

    Shared Measurement Versus Evaluation

    Is shared measurement really so critical for collective impact or Place-Based approaches? Where does evaluation fit in?…

    1 条评论
  • Come join me for a day of Developmental Evaluation

    Come join me for a day of Developmental Evaluation

    Still Places left at our Masterclass in Melbourne on Developmental Evaluation, Friday 24th. Should be fun, we are going…

社区洞察

其他会员也浏览了