Voice-Based AI Facilitator

When we think about generative AI, we often gravitate toward on-screen applications. Yet, there's a vast untapped potential in exploring how this technology can extend its support to off-screen activities as well.

Empowering parents to facilitate hands-on activities more effectively

Throughout my career, I've dedicated a large portion of my efforts to developing hands-on activities for learning, particularly in the realms of science and language learning. As the Product Lead at Lelu and a Product Manager at Imagin8ors Pte. Ltd. , I witnessed firsthand the impact of engaging children in hands-on projects in both fostering scientific discovery and enhancing language skills.

At Lelu, I curated three activities each month tailored for young children, each centered around distinct themes. The materials and instructions were sent to families, complemented by additional resources like original games, songs, and online classes.

However, one recurring challenge our company faced was the reliance on parents to oversee and facilitate these activities. We provided detailed instruction sheets and comprehensive tutorial videos, but we obviously could not engage with children ourselves as they embarked on their hands-on journeys. Neither could we anticipate all the questions and challenges children would uncover as they made their way through each activity.

Having conducted over 200 workshops, both physical and virtual, I understand that certain moments can either make or break the learning experience. Specific queries can unlock new dimensions of understanding, while certain frustrating challenges can be overcome with the guidance of a skilled facilitator.

I've long grappled with the idea of wanting parents to be deeply involved in hands-on projects with their children, while simultaneously recognizing that parental involvement sometimes becomes an obstacle to a child's engagement. Indeed, while parental participation fosters meaningful family conversations, strengthens bonds over fun activities, and nurtures a habit of shared learning experiences, it also demands a substantial commitment of time in an already busy schedule. Moreover, parents may not always feel confident in tackling the subject matter or technical aspects of an activity. They must be willing to tolerate a bit of mess and occasionally navigate confrontational moments when their children encounter frustration or veer off the planned path, taking on a “teacher” role that they would prefer to avoid.

What if AI could assume the role of an assistant facilitator?

For this reason, I've been dreaming of an AI facilitator for hands-on activities long before the rise of #ChatGPT. In my vision, this tool would take the form of a voice-based assistant powered by platforms like #Alexa and #GoogleHome. It would be trained on the specific activities presented to children and provide relevant guidance. This assistant would understand what a child is doing by prompting them to describe their actions and inquiring about their next steps, materials used, and more.

Beyond simple step-by-step guidance, this AI facilitator could ask questions providing opportunities for a deeper learning experience:

• "What was your biggest challenge, and how did you overcome it?"
• "What part of this project did you enjoy the most? Why?”
• "Can you think of other applications for what you've built? What else could you create?"
• "If you could share this with someone else, who would it be? Why?"

It is important to highlight that the introduction of an AI facilitator doesn't imply sidelining parents. On the contrary, their involvement would be actively encouraged. In fact, this virtual assistant would alleviate the pressure on them to have all the answers or to take on the daunting role of the "teacher." Instead, it would enable them to focus on the enjoyable aspects of the project and partake in the activities alongside their children. They might even ask questions themselves!

Simultaneously, the AI's adaptability would allow the voice-based facilitator to respond to the family’s potential willingness to explore different avenues and build new connections with the project at hand. These connections form the core of the learning process, greatly enhancing the impact of the activity while offering a more personalized, meaningful, and enjoyable experience for everyone involved.

Furthermore, based on children's inquiries and feedback, the facilitation of the activity could be refined for future participants. If a particular step proves consistently challenging, the AI assistant could anticipate it and offer extra encouragement or appropriate support. Conversely, if some participants find innovative ways to diverge from the project's path, these could serve as inspiration for subsequent learners.

Application of a voice-based AI facilitator for storytelling

Following my thoughts, I began exploring other domains where such AI facilitator could enhance children’s learning experiences. I quickly realized that storytelling was another field this tool could bring significant benefits.

One prime example is its potential to support platforms like Lunii 's Fabulous Storyteller, especially from an educational standpoint. With Lunii’s audio storybook, children pick certain parameters - setting, character, plot, conflict - and listen to a story that incorporates these choices.

About five years ago, I discussed with Lunii's product team and learned that the process of crafting stories was entirely manual. A group of writers was responsible for creating each tale from beginning to end as it was the only way to produce original, engaging, and well-crafted stories. It implies that beyond the initial choices, the stories were fixed.

Lately, Lunii introduced audio escape games, offering children more interactive ways to immerse themselves in their tales. I am unaware of ongoing innovations at Lunii, but I believe that a voice-based AI facilitator could amplify this flexibility and provide an experience where children truly become co-creators, actively steering the story in collaboration with an AI tool trained by skilled writers.

Another application lies in a potential post-story dialogue between the child and the storybook. Children could engage directly with the story's characters. Picture a character you’ve accompanied on thrilling adventures now asking you questions such as "Why do you think I was upset with the king?”, ”What could I have done differently when I lost my place on the first team?" or "How did you feel when I shared the frog's secret with all my friends?" Based on the child's responses, these prompts can spark a meaningful conversation, fostering critical thinking and social-emotional learning.

Limitations and challenges of a voice-based facilitator

Obviously, the primary limitation in an AI-powered facilitator, as is often the case with generative AI, revolves around children's safety. Anything presented to children must be under complete control. Children are naturally inquisitive and will test the boundaries of such a tool. Safeguards must be robust enough to ensure that not a single inappropriate response reaches the child. Nevertheless, Khan Academy is currently proving with their AI-powered tutor Khanmigo that there are ways to overcome this barrier and provide a safe solution.

The second limitation that is key and very specific to this potential tool is the ability to comprehend children's speech. Their pronunciation, expressions, and speech patterns pose more significant challenges for technology compared to those of adults' voices. Children who struggle to make themselves understood will quickly lose interest. Tackling this challenge is a worthy effort to improve interactions between children and technology, and companies like SoapBox Labs have made significant progress in this direction.

Finally, an AI facilitator would necessitate precise adjustments to strike the ideal balance between autonomy and guidance, be it in the facilitation of hands-on activities or the co-creation of stories. Achieving this balance demands a close partnership between domain experts, including educators and writers, and technologists skilled in configuring and prompting an AI tool. This is a whole new area of exploration, and anyone aiming to turn these ideas into reality will need to experiment quickly and invent new roles and processes.

What other applications or limitations do you foresee for a voice-based AI facilitator?

#GenerativeAI #AIInEducation #LearningDesign #GenAI #AIFacilitator #Facilitation #EdTech #Storytelling #HandsOnLearning