Learning Engineering Spotlight (Oct. 1-5): Your Weekly Dive into LE Research & Practice

Using conversational agents (via transformer-based language models) to promote students' higher-order / critical thinking and enhance the learning engineering process

In recent years, the field of learning engineering has experienced a surge of interest—in no small part due to advancements in AI and machine learning technologies. As a case in point, a new study by Mirzababaei and Pammer-Schindler (2024) sheds light on how learning engineers / engineering teams (and educational practitioners more broadly) can imagine and create effective conversational modules using transformer-based language models to support student learning.

This research, titled “Facilitating the Learning Engineering Process for Educational Conversational Modules Using Transformer-Based Language Models”, provides a systematic workflow for building adaptive, AI-powered conversational agents designed to elevate the learning experience.

The Core of the Workflow: A Three-Step Process

The proposed workflow involves three essential steps that guide learning engineers through developing adaptive conversational modules:

1. Defining the Initial Question: This step involves formulating a starting question based on learning materials. The initial question introduces a concept or definition and pairs it with an example, prompting learners to apply their knowledge critically.
2. Defining Expected Phrases: To ensure the agent can identify and classify student responses correctly, the learning engineer specifies expected phrases that reflect claims, warrants, and evidence as defined by Toulmin’s model of argumentation. These phrases enable the AI to recognize whether the learner has provided a well-rounded argument or if additional guidance is needed.
3. Defining the Dialogue Structure: The final step involves structuring the dialogue to support adaptive learning. This entails creating follow-up questions or feedback that can help learners construct more complete arguments, thereby enhancing the depth of their understanding.

Why It Matters: Supporting Learning to Argue and Arguing to Learn

The workflow empowers learning engineers to create conversational modules that provide immediate, context-sensitive feedback to learners. This feedback loop helps learners not only grasp concepts but also articulate their understanding through reasoned argumentation. Research shows that such activities can significantly boost cognitive development, especially in fields requiring high-order thinking, like science and law.

Proven Effectiveness Across Domains

The research team tested the workflow across three diverse learning domains:

1. General Data Protection Regulation (GDPR): By crafting scenarios that highlight GDPR principles, such as data minimization and accuracy, the study demonstrated how learners could apply theoretical knowledge to practical situations.
2. Astronomy: Learners engaged in discussions about planetary definitions, allowing them to apply abstract scientific concepts to specific celestial bodies, enhancing their ability to reason and argue effectively.
3. Artificial Intelligence: Exploring definitions of intelligence, the modules facilitated discussions that not only improved AI literacy but also allowed learners to form arguments based on varied perspectives of intelligence.

Results Speak Volumes

The study reported impressive outcomes: classifier performance achieved F1-macro scores ranging from 0.66 to 0.86 across domains, and the coherency of follow-up questions was consistently between 79% and 84%. These findings illustrate that transformer-based models can effectively support learning engineers in developing sophisticated educational tools without the need for extensive machine-learning expertise.

Looking Ahead

As AI continues to evolve, its role in educational settings will likely expand, enabling more educators to create personalized learning experiences with less technical complexity. This workflow offers a glimpse into how advanced language models can be leveraged to break down the barriers of traditional forms of instruction, making adaptive learning technology accessible to a broader audience.

For those interested in incorporating conversational modules into their educational strategies, this workflow provides a streamlined approach that balances technical and pedagogical needs. With AI’s capacity to enhance both content delivery and learner engagement, the future of education is poised to become more interactive, adaptive, and effective.

Read the Full Paper: Facilitating the Learning Engineering Process for Educational Conversational Modules Using Transformer-Based Language Models

Share Your Thoughts ??

How do you think AI-driven conversational modules can enhance student engagement and learning outcomes?

Hop over to the comments section and let us know! ?

Acknowledgements: Many thanks to the authors Behzad Mirzababaei and Viktoria Pammer-Schindler for conducting this important work to evolve the field of teaching, learning, and technology and for sharing it openly with the world.

*This newsletter was generated with the help of ChatGPT 4o (Sept 30 version).