AI in Team Coaching (Part 1): Early Steps towards Navigating the Challenge of Complex Systems

Artificial Intelligence has already begun to revolutionize the field of individual coaching, particularly through its application in standardized approaches like the GROW and other models, slowly moving towards more complexity. As we venture into more complex coaching scenarios, the limitations of current AI models become apparent (as of now). High-complexity professional coaching, which often requires nuanced understanding and adaptive interaction, still remains a significant challenge for AI.

The Complexity of Team Coaching

When it comes to team coaching, these challenges multiply. Coaching a team is not just about understanding individual behaviors: it's about navigating the intricate web of interpersonal dynamics, varying motivations, and the collective goals of a group including individual preferences. The complexity of managing multiple people, each embedded in their own unique system, makes it feel like we're still lightyears away from fully integrating AI into team coaching. The sheer variability in team dynamics presents a formidable obstacle for AI, which thrives in more predictable environments.

Early Experiments in AI-Driven Team Coaching 2020/21

At evoach, the startup I co-founded in 2019, we recognized these challenges early on. Starting in early 2020, we explored the potential of chatbots to facilitate individual self-reflection as a preparatory step for team retrospectives. The tools we employed ranged from those commonly used in agile practices (e.g. Stop, Start, Continue) to more traditional coaching methods (e.g. like constellation boards for making team dynamics more transparent) adapted for a digital interface.

These experiments revealed both the promise and the complexity of using AI in team coaching. On the one hand, the chatbots proved effective in helping some team members articulate their thoughts and feelings before entering a group discussion. On the other hand, the diversity in team members' openness to technology created a significant barrier. Some team members embraced the digital tools with enthusiasm, integrating them seamlessly into their reflection processes. Others, however, were less comfortable with the technology, which led to a mixed reception overall.

Challenges of Technology Adoption and Lessons Learned

By 2021, we decided to pause these explorations, not because they were unsuccessful, but because we recognized the need for a more refined approach. Our initial findings highlighted a crucial insight: the success of AI-driven tools in team coaching heavily depends on the psychological safety of the environment. Teams that felt secure in their ability to openly share and discuss their reflections were more likely to benefit from the hybrid approach we were pioneering. This realization led us to shift our focus and rethink how AI could best serve the nuanced needs of team coaching.

A New Opportunity: Research Collaboration in 2023

Fast forward to 2023, when Vanessa Mai approached us with an intriguing proposition: to research the acceptance of rule-based chatbots versus generative AI-based chatbots in team coaching settings. This was an exciting opportunity to build on our earlier work and explore the future potential of AI in this field of teams. The study was designed to compare two distinct types of chatbots—a rule-based system and a generative AI-based model—each supporting team reflection processes during a project week.

The chatbots were to function as virtual process companions, aiding project facilitators in guiding student teams through daily reflection rounds during a project week. This research not only provided a chance to refine our understanding of AI in coaching but also allowed us to gather valuable data on how different chatbot designs might be received by users in real-world settings.

Research Focus

Comparing Rule-Based and Generative AI Chatbots in Team Coaching

The primary focus of the study was to investigate the differences in user acceptance between two distinct types of chatbots: a rule-based chatbot and a generative AI-based chatbot. This comparison was crucial to understanding how these technologies could be integrated into team coaching, a domain that demands a high level of adaptability and sensitivity to the nuances of group dynamics.

Study Design and Methodology

Our study was set in the context of a University-wide Interdisciplinary Project Week, where student teams were tasked with collaborative projects. Each day, these teams participated in reflection sessions designed to help them evaluate their progress, address challenges, and improve team dynamics. To support these sessions, we introduced two types of chatbots:

• Rule-Based Chatbot: Developed using the Landbot platform, this chatbot was designed to follow a predefined conversation flow, utilizing selection options and buttons to guide users through the reflection process. Its structure ensured a consistent and reliable interaction, with less room for deviation from the intended path.
• Generative AI Chatbot: In contrast, the generative AI-based chatbot, built using GPT-4 turbo and the evoach platform, offered a more flexible and natural conversation flow. It was designed to respond dynamically to user input, creating an experience that felt less scripted and more akin to a human conversation. This model aimed to provide a more personalized interaction, adjusting to the specific needs and inputs of each team member.

Role and Task-Based Prompting

To ensure that the generative AI chatbot remained effective throughout the reflection process, we employed a dual-prompting strategy. The chatbot was consistently anchored by a defined role, designed to support and guide the team in their reflection. The tasks assigned to the chatbot varied each day, aligned with the specific phase of the project. This approach was intended to ensure that the AI remained focused on the team’s needs while encouraging deeper self-reflection.

Hybrid Approaches and Third-Party Tools

Interestingly, our study also touched on the potential for hybrid approaches in chatbot design, particularly in team coaching. We found that combining rule-based elements with generative AI capabilities could offer a balanced solution—leveraging the predictability of rule-based systems with the adaptability of AI. However, such hybrid models often require third-party tools or platforms to manage the complexity of conversational design, something that could be a limitation in wider adoption.

Key Questions Explored

Throughout the study, we sought to answer several key questions that would inform the future of AI in team coaching:

1. How effective are these chatbots in supporting team reflection and project processes?
2. What are the differences in user acceptance between rule-based and AI-based chatbots?
3. How do factors like performance expectancy and effort expectancy influence the adoption of these technologies?

--> In Part 2 of this article I will share more details on the research challenges, outcomes, highlights and learnings of this first paper presented at 12th International Conference on Human Interaction and Emerging Technologies (IHIET 2024) on 28 August 2024 in Venice, Italy by Vanessa.

PART 2: https://www.dhirubhai.net/pulse/ai-team-coaching-part-2-simplicity-over-rebecca-rutschmann-6vhxe/

You can find the paper here: AI-based Chatbot Coaching for Interdisciplinary Project Teams: The Acceptance of AI-based in Comparison to Rule-based Chatbot Coaching.