Generative AI in Practice: Evidence on Productivity, Learning, and Job Satisfaction
AI is revolutionizing the modern workplace, and a recent research offers a compelling glimpse into the transformative power of Generative AI in the context of a conversational guidance for customer support agents. A groundbreaking study (ref #1) by leading economists at the National Bureau of Economic Research has revealed the profound effects of #GenerativeAI when implemented on a large scale, specifically within the realm of customer support.
Using data from over 5000 service agents, the study unveils a notable 14% average boost in productivity, with a significant 35% surge for those less experienced in the field. However, it's not just about numbers; the AI assists in imparting best practices, fostering a positive customer sentiment, and potentially cultivating continuous learning among workers. While the advantages are evident, the study also sheds light on the nuanced impacts across different skill levels, emphasizing the vast potential and challenges AI brings to the fore
Interesting? Dive deeper to my sumarisa= uncover more...
Key Takeaways:
Spreading Expertise with AI Assistance
The AI tool monitored real-time text chats between agents and customers. It provided agents with suggested responses based on patterns from prior successful conversations. For example, the AI might propose an empathetic response like “I can definitely assist you with this!” to reassure a frustrated customer.
Though well-intentioned, such AI systems have faced skepticism. Could they operate effectively in complex, real-world environments? Would they provide inaccurate information or be ignored by workers?
This study provides an early “yes” to the first question and a more nuanced take on the second.
The data show that access to AI recommendations increased the number of issues agents could resolve per hour by 13.8%. Much of this came from reductions in handle time, or the time to complete each chat, which decreased by 9%.
Interestingly, these productivity gains accrued primarily to less experienced and lower-skilled workers.
Agents in the bottom quintile of performance improved resolutions per hour by 35% after getting access to the AI assistant.
Agents with 6 months of experience achieved the same productivity as tenured agents with over a year of experience.
AI Levels the Playing Field
Why did novice agents see such outsized benefits? The experts argue that generative AI systems like large language models (LLMs) implicitly identify and mimic patterns that characterize how the most successful agents communicate. LLMs are trained on large datasets of human-generated text, learning to predict the next word in a sequence based on prior context. This ability allows them to absorb nuances like tone and empathy.
In customer service, LLMs can then codify the tacit knowledge that distinguishes top performers—knowledge traditionally difficult to teach—and expose all agents to these best practices. As a result, lower-skilled agents benefit the most from AI assistance.
Indeed, the study found evidence that AI recommendations led agents to converge in their communication patterns. The text that low-skill agents produced became more similar to high-skill agents after adopting the AI tool.
Improved Experience for Agents and Customers
Besides increasing productivity, the introduction of AI-assistance also impacted the experience of work for both service agents and customers.
领英推荐
It was found that customers interacting with agents treated them more positively after AI adoption. Analysis of message sentiment showed customers using fewer antagonistic or abusive language with assisted agents.
Customers were also 25% less likely to request escalation to a supervisor, indicating greater confidence in agents' ability to resolve issues.
These improvements in customers' behavior mattered for agents as well. Access to the AI tool was associated with a 40% decrease in attrition among newer agents. Though wages were unchanged during the study, higher retention suggests assisted agents were more satisfied with their job, even as expectations for productivity increased.
At the same time, agents did not blindly follow AI suggestions. On average, they chose to incorporate recommendations only 35% of the time. The fact that adherence increased over time, however, indicates that agents found the suggestions increasingly useful. Workers who more closely followed recommendations realized the greatest productivity gains from AI access.
Key Challenges Around Data and Incentives
This study provides evidence that generative AI can increase productivity in customer service roles by propagating the skills of top performers.
However, several challenges remain regarding data practices and incentives.
The AI tool studied was trained on customer service dialogues generated by human agents. Yet these agents received no compensation for providing the training data that enabled the AI system to absorb and disseminate their expertise. The fact that top performers saw limited individual benefit from AI assistance compounds this issue. Firms must grapple with how to properly incentivize and compensate workers for their role in developing AI.
More broadly, this study examines the productivity impacts of AI adoption by individual workers at a single firm. It cannot provide insights into how the use of generative AI technologies will impact aggregate labor demand, wages, or inequality.
As firms deploy ever-more capable AI systems, they may seek to automate work currently done by newly productive entry-level workers or shift their skill requirements. Understanding the net impacts of AI adoption on workers, firms, and consumers remains an open challenge.
Despite these limitations, the findings provide an exciting early look into how generative AI can transform frontline work when thoughtfully implemented. The study demonstrates that AI can surface insights from top performers and help disseminate best practices firm-wide. The results show that updates to training practices, incentives, and organizational processes will be critical to realizing the potential of AI while ensuring benefits are shared equitably.
Key Takeaways
The rapid pace of AI progress demands proactive evaluation of its impacts under real-world conditions. This study provides an encouraging step toward understanding the transformative potential of generative AI in the workplace, as well as emerging challenges.
How do you see generative AI impacting managers and professionals in your industry? I would love to hear your perspectives in the comments!
References :
Group Technology VP at Capgemini ?? Ergo a Techguy "a Massive Technology enthusiast, among other enthusiasts that are building towards a better future, making lasting impact" ??
1 年I'm curious about the running cost ??! It is tough to argue that there will be no improvement leveraging LLMs in tasks, the question that raises now in a lot of organization is the operating costs at scale. Some early numbers indicate that I could make the services more expensive because of the usage of AI ?? which will eventually Not fit the CIOs budget unless the business outcomes growth compensate for the higher IT cost. What are your thoughts on that?