Lean Responsible AI Development - from Science Fair to Business Outcome
Valeh Nasser, M.Sc.
Entrepreneur | Connecting Parents & Providers | Mental Health Advocate
Simply assembling a team of data scientists and providing them with some data and a problem to solve might not be enough to achieve the desired business outcomes. By doing so, we might miss on the project’s risks. For instance, if the AI is not transparent and fair, it might alienate customers. Even worse, it can damage the company’s public image. Therefore, we need to identify these kinds of risks early, mitigate them, measure the impact of those mitigations, and intervene early and pivot if mitigation proves impossible.
For this reason, Impact Assessment is one of the first steps in Team Data Science Process published by Microsoft. Impact Assessment is an exercise in requirement analysis and risk management that identifies AI-specific requirements, risks, and mitigation strategies to ensure the system performs well and does not cause harm. Impact Assessment helps to:
In this article, I will discuss how Impact Assessment can be incorporated into a Lean development process to incrementally reduce uncertainties related to responsible AI principles and to enable pivoting the business model early if needed.
Overview of Lean Responsible AI Development Process
There are three iterative phases to the Lean Responsible AI development process (Figure 1):
At the end of each phase, we assess whether a responsible AI seems possible and whether we need to pivot or end the project. Once a responsible AI project is proven to be possible, we continue with the rest of development activities and continue to implement and measure the effectiveness of the mitigations.
The process aims to assess viability and minimize uncertainties related to AI-enabled products from inception. By first estimating the probability of harm after mitigation and then incrementally experimenting and measuring where uncertainty is high, we can reduce risks early and pivot quickly to land on viable solutions that uphold principles of responsible AI. To this end, this process encompasses the Lean methods described by Ash Maurya [1], relies on Impact Assessment best practices published by Microsoft [2], and incorporates principles of “Applied Information Economics” by Douglas Hubbard [3,4].
The rest of this article describes the phases of Lean Responsible AI Development process and is organized as follows:
Phase 1- Start at the inception with the Lean Canvas
At the inception of lean product development, Lean Canvas [1] by Ash Maurya enables quick iteration through alternative business models. It accomplishes that by providing a light-weight tool for clearly communicating value and growth hypotheses and how the system design satisfies them. Assessing the business model for responsible AI enables us to pivot early to focus on more viable business models.
As we define the product in the early stages and pivot through alternative business models, it is important that we keep issues related to responsible AI in sight. To this end, we can make sure to incorporate information that are significant to responsible AI development in the canvas (See Figure 2) and look for potential issues with the business model.
Scrutinize the Lean Canvas to assess if AI the right solution?
Ash Maurya recommends using the sections Problem, Existing Alternatives, Solution, Unique Value Proposition, and High-level Concept of the canvas to evaluate problem-solution fit. To do this, he recommends using the Job-To-Be-Done framework. For a thorough discussion, please refer to the Running Lean book [1].
Here we also need to specifically collect evidence that shows AI is the right approach. We can assess questions such as:
At this stage, we are looking for red flags with potential business models. Further analysis is needed later to answer these questions with more certainty (see next section’s discussion on the Responsible AI Risk Canvas). To be able to assess these concerns, we need to include information about the deployment context, potential datasets, whether data needs to be collected, potential algorithms, and how the system fits into the overall processes in the canvas.
Scrutinize the Lean Canvas to asses if the go-to-market strategy cause any harms?
Going to market can potentially cause harms such as fairness, equity, and inclusiveness. For instance, if we have marginalized stakeholders who are not early adopters, we risk building a system that further marginalizes them (fairness and equity-related harms).
To assess these issues, we need to include geographic areas and languages, demographic information, and whether some demographics have been historically marginalized in these sections. While these might not be known at the beginning, we need to not lose sight as the canvas is refined.
Please note that the traditional Lean Canvas only includes information about paying customers. To be more thorough, here we are also including direct users. At this stage, we are looking for red flags and obvious problems with the potential business models. Further analysis needs to be done later (see next section’s discussion on the Responsible AI Risk Canvas).
Phase 2- Create your Responsible AI Risk Canvas
After you have your Lean Canvas, you can create a Responsible AI Risk Canvas (Figure 3). The Responsible AI Risk Canvas provides a one-stop shop to track potential harms, mitigation approaches, who is responsible for mitigation, and the status of each harm.
This section is heavily based on Microsoft's Responsible AI Impact Assessment guide. For the meaning of specific terms please refer to the guide itself.
What is recorded in the Responsible AI Risk Canvas?
Uses: List the system's uses.?
For each use:
Stakeholders and harms:
For each use identify
Mitigations:
For each harm specify mitigations. These could include:
领英推荐
Probability of harm after mitigation: This field represents the expected effectiveness of the mitigation approach. In other words, it is the estimate of how likely the harm is after mitigation has been implemented.
To begin with, it is set to the subjective estimate of how well mitigation is expected to perform based on expert's subjective opinion, for instance, transparency harms are typically easier to address and pose less risk.
Once the mitigation is in place and tested, this field can be updated to the expected effectiveness of the mitigation based on measurements and tests. For instance, after we design the UI to be transparent about AI interactions, we can conduct usability testing to evaluate whether the AI interactions were transparent. Based on this we can use Laplace's rule of succession to estimate the effectiveness of the design and record it.
This approach is borrowed from Applied Information Economics [2] to help reduce uncertainty in the beginning of the project. By collecting the subjective estimates in the beginning, we can create a project plan that addresses the risks according to their priority. For instance, if the team decides that it is very likely to have an unfair AI, initial development efforts should focus on testing and mitigating the fairness issues. This is not different from how ins any Agile development, spikes are used to address initial risks to guide decisions.
This section is why I am calling this method "Lean" because it relies on hypothesizing, experimenting, measuring, and pivoting to find the right solution early.
Who is responsible for the mitigation? example: Data scientist, Product Manager, QA, Developer, etc.
Harm Status examples: invalid, open, mitigation planed, mitigated, escalated
Notes examples:
How to create the Responsible AI Risk Canvas?
Now that you know what a Responsible AI Risk Canvas is, let's look at how we can create it. Responsible AI Risk Canvas can be created through a brainstorming workshop. Before the workshop the Lean Canvas described in the previous section should be made available to the participants.
Who should attend the workshop: Douglas Hubbard in the book "Failure of Risk Management, Why It Is Broken and How to Fix it" [4] emphasizes on the importance of having a diverse team when brainstorming risks. In this case, you will ideally have representatives from users, system administrators, domain experts, as wells as the development team and project sponsor in the meeting.
Brainstorm to Identify uses, stakeholders, and harms: Microsoft's Responsible AI Impact Assessment Guide provides prompts and categories to aid in brainstorming uses, stakeholders, and harms. You can print these and provide them with the participants ahead of the meeting to aid in brainstorming. The book Game Storming [5] provides guidance on how to have fun and engaging brainstorming sessions.
Use Klein's pre-mortem to identify uses, stakeholders, and harms: Douglas Hubbard in the book "Failure of Risk Management, Why It Is Broken and How to Fix it" [4] notes that brainstorming often falls short for identifying risks. He recommends using Klein's pre-mortem to identify risks. That is, ask the participants "Imagine we are in the future, the system is in place, and something has gone horribly wrong, what has happened?" or simply ask them "What are some reasons that we should not build this system?".
Finally identify mitigation approaches: As discussed, depending on the harm, there can be various mitigation approaches. I will dive into these in a future LinkedIn article.
Estimate probability of harm after mitigation: Some mitigations are more straight-forward and have less uncertainty. Mitigations that are more uncertain pose more risk to the project's outcome. These mitigations should be evaluated and measured earlier in the project lifecycle. To enable this, we ask the participants who are subject matter expert on the mitigation to estimate the probability of harm after the specific mitigation is in place.
Douglas Hubbard [3] provides some guidance on how to collect calibrated estimates that have less random error and bias. One simple approach he recommends is the equivalent bet method. He also recommends a few more tools to combat bias and anchoring such as asking the participant to consider why they are wrong and re-estimate. For further discussion of how to get better estimates please refer to the book [3].
Who is responsible, status, and notes. You can track project management related notes, such as project milestone during which the risks can be mitigated, and other relevant information that is discussed in the meeting in these fields.
Phase 3- Iteratively experiment to measure effectiveness of mitigations where uncertainty is high
Based on the "probability of harm after mitigation" identified in the Responsible AI Risk Canvas, you might decide to do further analyze to assess effectiveness of mitigation approaches for certain risks. For instance, you might want to do an assessment of the data for representativeness or do a survey of the approaches to improve fairness of algorithms should the data be not representative. Based on your investigations you can then update the canvas with new "probability of harm after mitigation".
You can use Applied Information Economics (AIE) to decide how much you want to spend on the analysis and mitigation at this stage [3] (To learn more please see my article on AIE). In an extreme case the sponsor might decide to pivot the business model to remove the use or abandon the project for less risky ones that have better return.
Conclusion and what is next
In this article I proposed that we can start with the Lean Canvas at the inception of the project to look for red flags related to responsible AI development. We can then create a Responsible AI Risk Canvas as a light-weight approach for assessing risks and requirements related to AI projects. We can also estimate "probability of harm after mitigation". Based on this estimate we can assess mitigations that are risky early in the project so that we can pivot early if needed.
The Responsible AI Risk Canvas is heavily based on Microsoft's Responsible AI Impact Assessment Guide [2]. My experience is that documents sit in a corner and collect dust. In addition, we have been telling companies to move to Agile and Lean development approaches for years. Therefore, I think we need a more light-weight approach for tracking these risks and tracing them to actions and measurable outcomes. This is why I recommend using a canvas: A canvas can be printed on a wall or put on a wiki. It can be used as talking point in planning meetings to ensure traceability and continuity throughout the project. The benefit of using the canvas format is that it can be kept alive.
Finally, the message I am hoping to communicate is that responsible AI development is not that different but there are still differences:
a) Responsible AI development is not that different: As I discussed, we can adapt a lot of existing best practices from risk management and product management such as the ones discussed here. The main barrier is that enterprises who are not typically used to green-field product development might not have these capabilities in place to begin with.
b) Responsible AI development is different: It won't happen by accident. We need to specifically pay attention to responsible AI principles and analyze how they apply to our product or feature.
In short, we don't have to start from zero and company's like Microsoft are providing lot's of domain-specific guidance to pave the way.
If you already have an AI-enabled project underway, it is not too late: You can retrospectively create your canvases to better understand and mitigate your risks. If you have a new project coming make sure to do a thorough impact assessment.
What can be done to make this even easier: Ideally companies like Microsoft provide tools to implement the Responsible AI Canvas Risk Canvas in their MLOPs tools. That would allow traceability to test cases, versioning, and auditing. Ideally the model can not be pushed to production until all harms have been marked as mitigated or invalid.
Maybe a Copilot can make it even easier and more efficient: A large language model can be trained to capture potential harms that can be caused by AI through posing questions related to various responsible AI principles. It can then analyze responses, suggest mitigation approaches, and create and assign tasks/issues. But without the tools in place, we rely on practitioners and processes.
Final note: Earlier, I mentioned that some of the barriers to responsible AI development are operational while others are related to corporate buy-in. In this article, I describe a lean methodology that addresses both concerns by shedding light on how we can run successful responsible AI initiatives without compromising either responsible AI or business outcomes.
About me:?
I am a software engineer and computer scientist with a passion in optimizing value. My passion has led me to a diverse experience in product development, product management, and engineering management. I am a tireless learner and thrive in fast-paced and dynamic environments where I can work collaboratively to create value.
References and Further Reading