AI and UX Research

Originally published via UXforAI.com : https://www.uxforai.com/p/ai-and-ux-research April 13, 2023

So, as we have discussed previously, AI is radically altering the landscape of the UX profession. What impact will the new technology have on UX Research? Will UX Researchers still have jobs? What research skills are going to be most in demand in the new age of AI? If this is of interest to you, buckle up for a fast ride, for these are the very questions we will attempt to tackle in this installment of our newsletter.?

For clarity, we will split the techniques into three sections: 1. Automated Away, 2. Radically altered, 3. Increasingly Valuable. Finally, we will end with section 4,??“Bullshit” AI applications for UX research that are dead ends and terrible ideas.?Note that this is merely a sample of common UX Research techniques and specializations, not a complete list; thus, the reader will likely require some imagination and extrapolation. However, we aim to provide a sufficiently wide sample to point the way.?

Let’s dig in:

1. UX Techniques that will likely see Full Automation

Increased sophistication of AIs will allow many manual activities to become fully automated. UX Research activities that rely on routinely creating and processing textual information will be some of the first affected, including:

Routine Usability Studies?are going to be mostly automated. This has already been the trend for the past decade, so it should come as little surprise at this point. Everything from writing a usability study script to creating the initial prototype (See our previous newsletter edition, UX is Dead https://www.dhirubhai.net/pulse/ux-dead-greg-nudelman/ ) to collecting user feedback for basic usability studies is routine and based on common, well-established patterns. Routine Usability Studies are also of somewhat limited value compared to RITE studies, as Greg has argued in his 3rd book, The $1 Prototype: A Modern Approach to Mobile UX Design and Rapid Innovation (https://a.co/d/3jUEMou ). We shall cover this topic in greater length in future installments.

Routine NPS Studies and Surveys?are likewise going to require ever-diminishing human intervention. From writing survey questions to analyzing data to creating presentations and making recommendations, AI is now more than up to the task of handling the basics. If this is your bread and butter, read on.

Collecting and organizing the research data?is going to be one of the most radically altered areas of our profession. Whereas before, we required manual tagging and labeling of studies, organizing, breaking up, and managing stored recordings, etc., using tools like Dovetail and dedicated personnel, the new AI capabilities are more than up to the task of collating and reporting data, as well as creating those cool?affinity diagrams?with insights and roll-ups of various kinds. Even more significantly,?executive strategy insights and rollups?will likewise be automated to the point where any Product Manager could use natural language to query a wide-ranging database of deep insights spanning multiple years. There should be few issues importing old data into the new AI tools. While the tools will be expensive at first, their wide popularity and obvious utility will quickly make the pricing competitive. Think of this as Pendo but for qualitative insights, with natural language query capability and auto-complete.?

More than just the isolated quant insight mining, the holy grail of?triangulation of quant and qual insights?and novel insight and product capability generation will become the norm for every new project. The good news is that this should minimize the Cowboy PMs irresponsibly spending millions on pet projects. The bad news is that you may need to retool if that was your specialty.

If routine or text-based workflows are your main trade, we highly recommend re-tooling as a “studies automation supervisor” or up-skilling to some of the more sophisticated UX studies flavors we mention here.

2. UX Techniques that will be Radically Augmented

Although AI-based automation receives the lion’s share of media attention, the biggest gains often come from using AI tools to augment the current processes to increase speed and efficiency. Strategically, AI is best thought of as “Augmented Intelligence” instead of “Artificial Intelligence”. This machine augmentation will show itself in many areas, including those of UX Research and design:

Competitive Analysis?is likewise going to be radically altered by AI that can quickly mine individual screenshots from documentation, video frames, and voiceovers to pull out relevant screens and reverse engineer a guess at the functionality. However, unlike anything text-based, we anticipate that this particular capability will take longer to come online and will remain proprietary for some time. This means that while competitive analysis will not become fully automated tomorrow, it will be radically altered: the researcher will employ more sophisticated research tools and speed up the process of finding and reporting. Again, we anticipate that this kind of study will become routine and required for any serious project due to eventual automation and a decrease in labor and therefore cost and time required to complete. Splitting data gathering among many small independent non-real-time search agents (also called daemons) will speed up intelligence gathering and analysis through parallel threading. Instead of taking weeks to complete, it might take hours or even minutes. However, instead of fully automating a typical NPS study, the competitive analysis will become AI-Augmented, requiring humans to make wider leaps of logic between seemingly unconnected data and make extrapolations beyond the current level of sophistication of our AI.

Identification of Novel Use Cases. Closely related to the core human skills are business skills. Identifying new ways to make money, market opportunities, and unique niche offerings will likely be heavily augmented by AI tools which will be able to point out novel opportunities and market inefficiencies based on competitive analysis, as already mentioned above. This type of AI-augmented business analysis will become the norm for any Business Requirements Document that will help executives make decisions quickly and with increased confidence.

RITE studies?are likewise going to be radically and permanently altered. AIs, in very short order, will be able to interpret customer feedback and suggest alternative design patterns in real time. Tools will become available, allowing researchers to plumb the design space very quickly using a set of existing design patterns mined from the company’s design system or direct competitors. Imagine showing the customer some new functionality, whereupon the customer is not 100% happy or maybe confused about something. AI should be able to detect that confusion and generate an alternative design of the page or flow in real time, offering the researcher multiple design options (similar to how Midjourney's /imagine function works today), whereupon the researcher will be able to choose which version to show to the customer next. Using workflow like this, and due to AI augmentation, the speed and efficiency of the RITE research will reach the next level. Rather than spending weeks discussing and iterating, the researcher will be able to achieve a passable solution within a single day. The key skill will be the intuition of picking the right direction of inquiry and understanding “where the dragons are buried” in other words, the experience of knowing which parts of the new functionality will need to be probed for problems. Over time, AI will get even better at recognizing these patterns and suggesting solutions. It will increasingly help even beginner designers make the right guesses from the start.?

Because of the need for a high level of augmentation, researchers and designers who understand how to work with AI and have experience doing so will most likely take advantage of the new technology. This is the newly emerging specialty class of “AI whisperers” proficient with and comfortable in using the new AI augmentation technology.

3. UX techniques that will become Increasingly Valuable

With AI automating or augmenting routine UX activities, certain skills that AI will have a hard time understanding and simulating will actually increase in value. Among those will be UX skills such as:

Core Skills?(aka Dealing with Humans) — we predict that the old “3 in a box” model where Devs, PMs, and UX folks work in small teams to research, identify, and build new functionality will quickly evolve into “4 in a box” model, additionally including Data Scientists and AI specialists among the folks needed to create the new Product features and functions. This means that the field will be even more reliant on “knowledge leaders,” who can create a plan, achieve consensus and execute delivering the new products to consumers. These core skills of consensus building, negotiation, and making people feel good while working together toward a single goal are not getting replaced by AI any time soon, if ever. In fact, they are going to become even more prominent as various professions will become even more deeply specialized. Understanding the technology and the ability to leverage it for business and humanitarian needs will be key to this cohort of UXers.

Workshop?Facilitation?will likewise not be automated or augmented any time soon. Being able to facilitate brainstorming, come up with novel ideas and drive consensus building upon multiple diverse opinions is going to become a very valuable skill that AI will likely not be able to effectively augment in any appreciable way.

Formative Research, field studies, ethnography, and direct observation?will likewise be very hard for AI to augment or replace. AI has yet to be able to efficiently use robot vision or tie various sensory inputs together to generate novel insights not previously written down or based on complexly integrated visual and textual inputs. For example, user research of tools for hands-on professions such as doctors, plumbers, factory, and agricultural applications, and in short, anything that involves observing people interacting with complex mechanical systems or other humans and drawing complex conclusions will only gain in prominence as routine usability research becomes fully automated.

Vision Prototyping?is a key technique of synthesizing various research inputs, market needs, and horse sense to create a prototype of a novel product or feature. (We will cover this essential technique at length in future installments.) By default, it is creating something new that has not been done before and expressing it using the existing design system components. This skill is difficult to model and harder to automate. Although augmentation might be somewhat helpful to speed up the production of these prototypes, speed is rarely an issue even today — the key is the creative spark that is difficult, if not impossible, for AI to replicate. Although AI can generate a great variety of approaches based on a set of instructions, being able to spot the right direction for a new product or feature is not something that AI can do easily. In fact, today, it is one of the more “Bullshit” AI features (see below.)?

Finally, UX staff involved in augmenting the executive strategy will likewise be fine. Although various research reports may be automated and heavily augmented, finding that “needle in a haystack” of all that data will be harder than ever and will require multi-disciplinary analysis at the intersection of understanding the technology, business use cases, market growth direction, consumer demand, and empathy and human values that UX is uniquely suited for. Any UXer who can leverage their understanding of business and technology and synthesize their understanding into a novel solution will find their skills in great demand.

If your primary skills already fell into this section, rejoice! If not, there is still time to build this up, but we would advise not waiting long, as competition will likely be fierce.

4. AI Bullshit

This section represents but a very small sample of AI applications to UX research that are far-fetched, oversold, and over-complicated, or just fail to grasp the rudimentary principles of UX Design.

AI Strategic Analysis tools?that claim to replace humans in coming up with novel ideas and business use cases. While we mentioned that this UX strategy application will be heavily augmented in the near future, AI is not a replacement for experience, empathy, and understanding of human needs and desires. Adopting AI decisions instead of human decisions is a dangerous and costly assumption, as those will virtually guarantee that you will be building products and features for robots, not humans. Despite the vendor claims to the contrary, AI cannot replace your CPO or UX Director any time soon. Suggesting otherwise is pure folly, similar to selling the AI equivalent of Silicon Valley snake oil.

AI Heuristics Analysis replacing user research and design.?When the robot is dancing, we are not impressed by the style of the dance but by the very fact that the robot is dancing. While impressive, Heuristic Analysis is one of the most basic UX design skills. Claiming that this simple ML function removes the requirement of user testing (or even replaces designers altogether) is pure bullshit. Heuristics are but a?guide?to what questions the researcher might want to ask. It is a “finger pointing at the moon”, not the moon itself (which, in this case, is delivering a functional product that customers actually want to buy, on time and on budget ). Claiming that heuristics alone will solve all of the issues ignores the real-world constraints of what can be built and for whom, which is why 4-in-a-box tiger teams and user research studies with real humans are essentially irreplaceable, at least at this point of AI sophistication.?

Closely related to the idea of using heuristics to replace user research is the idea of AI acting as users.

AI acting as users for the purposes of usability research?is currently being peddled by a few misguided vendors.?This is a terrible idea.?Training AI to pretend to be a user and using that as a replacement for the actual user research is likely one of the most misguided, bullshit ideas to come from the AI field. Let me make this clear: replacing actual user studies with AI models will guarantee that you will build products for AI, not for actual customers.

Research of human needs requires an understanding of technology, knowledge of business needs, and empathy for the customer. Good user research also requires keeping your eyes and ears (and, most importantly, your heart) open for that creative spark that the Gods occasionally see fit to grace us with. If outsourcing your research is like outsourcing your vacation, then replacing that creative process with AI is like outsourcing your vacation… to a bunch of robots.

The same goes for bullshit tools that claim to?Build your Persona using AI. The key part of the Persona-building process is the consensus-building, discussion, and education of the team members. Using AI to get there faster does not buy you any advantage. It’s a bit?like skipping all the tedium of vacation in a rush to see to the photographs. Again you are attempting the shortcut of the most important part, which shows a complete lack of understanding of the value of the UX process.?It’s more of the Silicon Valley snake oil that appeals chiefly to gullible inexperienced business people in a rush to check the “UX box.” And my California peeps should really know better by now than to try and peddle such drivel.

On the other hand, an excellent use case for "AI acting as users" is what our friend and former co-worker Madeleine Le has termed a “Kobayashi Maru” training exercise for researchers, where AI could perhaps pretend to be a cranky human customer who is determined to fail the usability test and tries to unbalance the inexperienced researcher. Now that might make a very intriguing product indeed!

Hungry for more AI use cases? Be sure to head over to UXforAI.com and smash that [Sign Up] button to ensure you don’t miss the next issue of our newsletter, “UX for AI: How to Avoid Outsourcing your Next Vacation to a Bunch of Robots” (ahem, actually, we call it?“UX for AI: UX Leadership in the Age of Singularity”) When you sign up on UXforAI.com , you get to see these in your email inbox every week, plus tons of goodies like a free book, book chapter previews, discounts on workshops, and more… Did I mention a free book?

We are now officially done with our introduction to the subject, and next week we will start on the practical UX for AI techniques, beginning with telling you how to spot your best AI use cases in a simple and reliable way that works (almost) every time.

And also how to spot more AI bullshit.

See you soon,

Greg Nudelman with Daria Kempka