UX Experts Misjudge Cost-Benefit from Broad AI Deployment Across the Economy

Permalink for this article: https://www.uxtigers.com/post/misjudge-cost-benefit-ai

The verdict is in from my Quiz on how AI usability influences productivity and the economy: the average score stands at 65%. Despite one participant's claim of the test being a breeze, the results beg to differ.

The most challenging question revolved around the average labor factor productivity growth in the United States, with only 39% answering correctly. It appears the stagnation of productivity growth in recent years has left many in disbelief. This stark contrast with past growth rates fuels my enthusiasm for the research findings that underscore the transformative potential of current AI tools.

And imagine the possibilities if future AI tools are crafted by UX professionals — we could witness an unprecedented surge in productivity growth.

For a comprehensive understanding of the current research and the quiz's context, read my article: ?https://jakobnielsenphd.substack.com/p/ai-vastly-improves-productivity-for

79% of the quiz respondents self-identified as UX professionals. Thus, even though there was a smattering of AI experts, business executives, and other groups among respondents, the data overwhelmingly reflects the UX pros, so that’s how I will discuss the findings.

A common misconception that leads people to overestimate the economy's growth rate is the conflation of their personal economic progression with the nation's overall productivity growth. This misinterpretation stems from the natural human tendency to over-emphasize their personal experience as representative of the general public (which is why we need the UX slogan You ≠ User ). As business professionals mature from their early twenties to their forties, they experience a significant enhancement in their work proficiency, fueled by the accumulation of crystallized intelligence . This personal growth trajectory is often mirrored by a corresponding salary increase. Very little of that extra money comes from general productivity improvements, but as an individual living within the stream of time, you can’t tell the difference between you getting better and all workers getting better.

The question that stumped the second-highest number of participants addressed the productivity surge experienced by programmers utilizing the GitHub Copilot. 42% got this one right. Intriguingly, a majority of respondents significantly undervalued the profound enhancement in programmers' performance when bolstered by AI assistance.

Even more intriguingly, underestimating the gains from AI support for this highly-cognitively-demanding job was shared by the programmers themselves. Even the people in the study couldn’t believe how good the AI had been at helping them develop code, when asked after the experiment to estimate their own productivity gains. (Since this was a between-subjects study, each programmer only developed the code once, with or without AI help, so there was no way for each individual to know how fast he or she would have been in the other experimental condition.)

Respondents made the most mistakes in the two money-related questions (“Coins” by Leonardo.AI).

Combine people’s tendency to *over*estimate the productivity growth without AI and to *under*estimate the measured productivity boost from an AI deployment. This striking dichotomy could explain why many UX professionals remain hesitant to fully integrate AI into their professional repertoire, see https://jakobnielsenphd.substack.com/p/ux-needs-a-sense-of-urgency-about

If you (falsely) believe that business-as-usual is progressing nicely and that the gains from embracing AI are small, then I understand why people are hesitant to make the scary move to a revolutionary new technology. However, the data shows the opposite: the economy has been fairly stagnant in recent decades and AI is an incredibly powerful accelerator that’ll boost our living standards to an extent not seen since the 1950s. (Likely by more! And this time it won’t just be Americans and Europeans who enjoy better living standards: AI will cause a worldwide boom, probably more so outside those old-economy regions.)

[Follow me on LinkedIn for more UX updates like this: https://lnkd.in/gfffgNcr or subscribe to my Substack newsletter to be notified by email: https://jakobnielsenphd.substack.com/ ]

AI UX Article Series

This article is part of a more extensive series I’m writing about the user experience of modern AI tools.?Suggested reading order:

1. AI Is First New UI Paradigm in 60 Years
2. AI Vastly Improves Productivity for Business Users and Reduces Skill Gaps
3. AI vs. Metaverse: Which Is the 5th Generation UI?
4. UX Needs a Sense of Urgency About AI
5. Prompt-Driven AI UX Hurts Usability
6. Generative AI Enhances Old Users’ Intellectual Performance (this article)
7. ChatGPT Does Almost as Well as Human UX Researchers in a Case Study of Thematic Analysis
8. How Much UX Do You Need for AI Projects?
9. “Prompt Engineering” Showcases Poor Usability of Current Generative AI
10. Business professionals with experience using AI are more optimistic about its potential than are colleagues who have not used AI
11. UX Experts Misjudge Cost-Benefit from Broad AI Deployment Across the Economy (this article)
12. (more next week:?follow me on LinkedIn ?to see in your feed)

About the Author

Jakob Nielsen, Ph.D., is a usability pioneer with?40 years of experience in UX . He founded the discount usability movement for fast and cheap iterative design, including heuristic evaluation and the?10 usability heuristics . He formulated the eponymous?Jakob’s Law of the Internet User Experience . Named “the king of usability” by?Internet Magazine, “the guru of Web page usability" by?The New York Times, and “the next best thing to a true time machine” by?USA Today. Before starting NN/g, Dr. Nielsen was a Sun Microsystems Distinguished Engineer and a Member of Research Staff at Bell Communications Research, the branch of Bell Labs owned by the Regional Bell Operating Companies. He is the author of 8 books, including?Designing Web Usability: The Practice of Simplicity,?Usability Engineering, and?Multimedia and Hypertext: The Internet and Beyond. Dr. Nielsen holds 79 United States patents, mainly on making the Internet easier to use. He received the Lifetime Achievement Award for Human–Computer Interaction Practice from ACM SIGCHI.?Follow Jakob on LinkedIn or subscribe to my Substack newsletter to be notified by email: https://jakobnielsenphd.substack.com/

Jakob Nielsen can speak at your company or event. More info: speaker bureau BrightSight Speakers or email [email protected]

Presentation topics:

• 10 Foundational UX Insights
• AI & UX
• UX: What Why How (and Who When Where)
• UX: Past, Present, Future
• UX Fireside Chat with Jakob Nielsen (AKA AMA)