The Role of Research in Professional Development

Dr Deniese Cox partners with educators who want to achieve a better online experience with her powerful research-driven professional development. We recently spent time with Deniese to understand the role of research when it comes to supporting the industry to improve learning outcomes.?A lover of tea and spending time away from technology in the garden, Deniese is a valuable member of the Institute for Learning & Performance community.

Integrating Research in Professional Development?

ILP: Can you share an example of a project where you successfully integrated research findings into a professional development program?

Deniese: I’m lucky enough to have built for myself the best job in the world. I get to spend 50% of my time doing hands-on research, working with educators, designers, and adult learners as we trial, refine, observe, and measure what does and doesn’t work online. I then spend the rest of my time transforming those insights into really practical professional development.?

As much as I love research, professional development is my happy place. It’s an honour to meet so many teams, educators, and designers every year. Importantly, it’s through this process of delivering PD, that I find out what questions are top of mind right now, and this in turn drives what I research next.

ILP: How do you stay updated with the latest research in the field of professional development, and how do you decide which findings to incorporate into your programs?

Deniese: For a finding or insight to make its way into my PD, it has to have two characteristics.?

First, it has to have ‘worked’. That is, it has to have improved the learning experience and learning outcomes at its trial site where it was first refined. We then implement that strategy at two fresh sites with different learners, curriculums, and trainers, and we trial it again – and only then do we say that the strategy is usable.?

The second characteristic is that the strategy needs to have been shown to be adaptable and workload friendly. So, at those second and third trial sites, I pay very close attention to the practicality of the strategy in the real world, with real learners, and real resource (time) constraints, etc. Does the strategy make sense to the trainers and designers? Is it something they can implement within their unique context? And is it workload-friendly? Unless we get a resounding ‘yes’ here, then the strategy (and the 1-2 years research behind it) essentially goes in the bin, we simply don’t roll it out. Instead, we take what we’ve learned to drive the next fresh, alternative investigation or trial.

Research Methodology

ILP: What research methodologies do you find most effective for studying professional development practices, and why?

Deniese: One of my favourite research methodologies is a “reason aloud” protocol that I developed when doing my PhD a few years ago. When asking a trainer, learner, or group to simply chat away while doing something (such as sorting ideas into a matrix, or engaging with materials online), then they almost forget that the researcher (me) is even there. They let their guard down and simply ramble on with what they’re experiencing. This approach elicits responses and commentary that is less constrained than a traditional Q&A type interview. They have more freedom to move back and forth between topics, and they choose what topics are worth spending more or less time deliberating on.

Deniese recommends the following article for further reading: Creating a space for reflection on professional priorities.?

ILP: How do you ensure the reliability and validity of the research you use to inform your professional development strategies?

Deniese: I use a combination of qualitative and quantitative approaches in my work. Those quantitative approaches enable me to incorporate hard numbers that can be analysed in a variety of ways. The numbers tell us ‘what’ is happening. However it’s the qualitative approaches that tell us ‘why’.?

For example, measurable learning outcomes can tell us that this strategy which was applied for one group but not another resulted in improved outcomes for that one group. But it doesn’t tell us why that strategy worked, and that’s why we then move to qualitative approaches to talk with learners to understand how they interacted with that strategy or how they felt it helped them. We also talk with educators and designers about their experience of implementing the strategy.?

In research terms, we call this working with participants to “co-create interpretations” and I think this is such a valuable and important feature of research.

Application of Research

ILP: Can you describe a situation where research challenged your existing beliefs or practices in professional development, and how you adapted to these new insights?

Deniese: I recently spent two years investigating learner digital skills as a barrier to adult education participation. Going into the project, I thought that digital skill barriers would be something that impacted over 30’s or even over 40’s, and I thought that access to digital devices would also be a major factor. It was surprising to meet so many 18-20 year olds who had a backpack full of devices, who created content for social media platforms, but who couldn’t navigate a training providers website to select a course, who couldn’t complete the online enrolment process, who weren’t confident receiving emails.?

All of my research projects until then tended to be with current learners. It was a complete eye opener in that project to realise that we had learners of all ages stuck on the outside looking in because they don’t have the digital skills to find or enrol in courses. Regardless of whether courses are online or on-campus, finding and enrolling in them requires digital skills these days.?

As someone who has a Masters in online Learning & Development, and a PhD in online education, I’m of course a fan of online training. And I’ve personally talked with thousands of learners for whom online has transformed their lives because they now have access to anytime-anywhere education that was previously not an option for them.

But I’ve learned to keep in mind all those adults out there who desperately need digital upskilling because the education they aspire to (and so many other services) is inaccessible because of digital skill barriers. Making a doctor’s appointment, changing your address on your licence, changing electricity provider – digital skills are so important and we can’t gloss over that essential upskilling step that needs to happen before learning, not as part of another course. That’s been a great learning experience for me.

Measuring Impact of Programs?

ILP: What metrics do you use to evaluate the impact of research-driven professional development programs?

Deniese: One of my favourite insights from recent research projects was around learner assessments.?

Reviewing completed (or, more often, not completed) assessments, talking with trainers and learners, we discovered one question type that annoys everyone! It’s the old “list 5 things” or “give 5 examples”. Reviewing more than 100 different assessment tools that incorporated that question type in written assessments, we discovered that instead of listing the required 5 items, learners on average listed less than 3 things. Incomplete answers is such a frustration and workload impactor for trainers and learners alike.?

What I loved about that project is that when we experimented with a variety of different alternatives, we discovered one absolutely brilliant fix. In every single trial we ran with that fix, learners went from giving less than 3 responses, to every single learner giving at least all 5 responses, and 30% of learners offered more than was asked. Best of all they gave robust answers, rather than the one-word throwaway answers that were previously given.

I love this outcome because ever since I put that video into my PD platform, there’s not a day goes by that I don’t get a lovely ‘thank you’ email from a trainer or team who’ve implemented it themselves and seen the same great results.?

Results like that definitely makes the long and often lonely research hours worthwhile!

Research Challenges and Solutions

ILP: What are some common challenges you face when integrating research into professional development, and how do you overcome them?

Deniese: A very common barrier to implementation are the rules or templates that an organisation has in place. Very often those templates go against ‘good’ practice, and make it incredibly difficult for trainers and designers to do their job to the best of their knowledge and ability. This is why I encourage whole teams to come along for training, or to enrol in my video platform – it’s so much easier when the team all have the same ah-ha moment and can make teamwide changes or tweaks.

One of the things my research has consistently found over the years is the importance of both beliefs and knowledge in professional development. Too often professional development focuses on giving knowledge, on telling trainers and designers what to do, what buttons in the LMS to press, what tools to use. However, to trigger action, we need to bring those beliefs along for the ride. Belief that this strategy will help me and my learners, belief that this button is worth pressing, that this tool actually has value for the learning experience or outcomes. Beliefs and knowledge are jointly incredibly powerful drivers of behaviour.

ILP: In your opinion, what are the emerging trends in research that will shape the future of professional development?

Deniese: While Artificial Intelligence (AI) is a hot topic, and definitely something we need to be paying attention to, I’m observing two things around AI.?

First, at the moment, AI makes a great assistant. As educators and designers, we need to be seeing AI as an offsider who can free us up to do what we do best – teach. In short, as a rule, AI doesn’t make a great teacher, but it does make a great assistant so we can teach more/better.?

Second, I keep very detailed records of how many people attend my different PD sessions or watch different videos, and the foundation topics (e.g. slide design for reducing cognitive load and improving assimilation) are always the most well attended. This means we have to remember that trainers and designers still want and need the basics, and we can’t overlook those training needs in favour of new shiny things.

Special thanks to Deniese for spending the time with the Institute for Learning & Performance Asia Pacific to bring you this informative article. Connect with Deniese on LinkedIn and check out her website here.?