AI-Product Fit: Just Because You Can, Doesn't Mean You Should

We continue to write about what we learned from our AI and machine learning product design and research work. Previously, we wrote posts on AI, people's behaviours, product thinking for teams, and effective user testing for AI products. Today, we cover learnings on a strategic decision-making level: how to think about building the right thing.

So, you’ve been tasked to leverage artificial intelligence in your product or experience? That's brilliant, but before you dive in, let's take a step back and consider the bigger picture.

The complexity of the problem space, the scale of potential unintended consequences, and the emergence of the outcomes require different ways of thinking and approaches to the early stages of design exploration. Engaging in systems thinking, developing cultural agility, learning across disciplines, becoming comfortable with ambiguity, and resisting pressure for simple explanations are vital ingredients in the revised approach to problem definition.

As designers, we have a unique prerequisite for the job: understanding the human context and asking the all-important question, "Why?". This superpower allows us to guide AI-powered projects by defining what not to do with AI, ensuring that we create experiences that are innovative, powerful, robust, ethical, and beneficial to our customers.

Cheryl Platz, in her book Design Beyond Devices: Creating Multimodal, Cross-Device Experiences, suggests a playbook approach that can be distilled into four key steps to follow to ensure your team is heading in the right direction:

1. Start with defining the desired outcomes within your customer's context.

Before integrating AI into your solution, could you take a step back and consider your customer's perspective? How might they react to the use of AI in this situation? Don't assume that AI is inherently desirable or positive. You can conduct user research to find out what your customers want and don't want you to do.

Prompts: How will your proposed system help your customer? Will it augment, automate, or act as their agent?

2. Use an "opti-pessimistic" approach to explore the consequences and risks of success and failure.

When designing AI-powered experiences, it's essential to consider both the best and worst-case scenarios. Design for the best case, plan for the worst case and be ready to adapt.

Prompts:

• What are the worst consequences if the product is successful?
• What are the best ways you can respond to an unforeseen problem?
• How are we defining success? What are the potential unintended side effects of that definition?
• What steps do we take to ensure that underrepresented customers are not excluded or harmed?
• How might the system allow us to mitigate bias in real time?
• What are the human, environmental, and financial costs if the model makes a mistake? How do we mitigate the risks?

3. Carefully examine the data, implementation plan, and your team building the model(s).

Know your data inside and out. Know the decisions that people training your models make regarding biases and transparency.

Prompts:

• Where does the data come from, how was it collected, and how might it reflect societal biases?
• Can your model provide transparency into why it makes specific recommendations?

4. Explore potential manifestations of your AI.

• If your experience includes voice UI, consider how your selections of gender, pitch, speed, and language will impact your customer's perception of the experience.
• Be transparent about your system's digital nature. If not, consider the legal implications of implying your agent is human.
• If you're trying to position your system as somewhat human, with a name and human traits like disfluencies, consider the desired effect of this illusion.
• If your experience includes a visual avatar, could you consider whether a humanistic avatar's additional motivation is appropriate for your customer's situation and needs?

By following these steps, you'll be equipped for an excellent start to thinking about AI-powered experiences that meet your customers' needs and avoid potential pitfalls. Remember, just because you can use AI doesn't mean you should; with careful consideration and a systems thinking approach, there is an intelligent approach that respects the complexity of the task.