AI Process Risk in Focus
Sri Krishnamurthy, CFA, CAP
CEO, QuantUniversity | AI Expert | Educator | Author | TedX Speaker |
Last week, in my Big Data and Intelligent systems class at Northeastern University, I taught my students how to build pipelines. A most of you know, AI systems are complex with multiple dependencies. Depending on whether you are designing a training pipeline, testing pipeline or deployment pipeline, the components you choose to build and integrate will dictate how robust your system would be! As the old adage goes:
A chain is only as strong as its weakest link
Risk in ML pipelines need to be understood and a robust AI risk management practice means that emphasis has been put on understanding, assessing and managing AI process risk too!
?? What is AI Process Risk?
As enterprise adoption of AI and ML products have increased, there has been increased focus on scaling and deploying AI systems. Best practices in enterprise software development have influenced AI system developers to build toolchains and products to address the various nuances in AI and ML system development. Since AI and ML systems are predominantly data-driven and the frequency of updates is much higher than traditional software development products, fields like DevOps, MLOps, DevSecOps have evolved to address the idiosyncratic nature of AI/ML systems.
While many companies and open-source efforts have focused on efforts to mature the ML-Lifecycle, the selection, assembly, integration and optimization of the processes (typically pipelines) have been left to the AI system builders. While best practices are still evolving and standard design patterns are being sought, AI system builders are trying their best to understand and address potential risks that are a result of these pipelines. These risks are not just model risks. The upstream and downstream dependencies and the complexity in realizing business requirements in disparate decoupled environments leads to risks that are still being understood.? We collectively call these risks as AI process risks.
?? Why do we need to formally address AI process risks and not just look at model risk?
When you see a simplistic definition of a model, you typically see 3 components as shown in the figure.?
But in the real world, the complexity is much larger and the actual model is just one component in the pipeline. For example, a typical machine workflow would look something like this.
Source: QuantUniversity Course materials
The Google paper by Sculley from 2015 discusses the various components that typically make up an AI/ML system indicating the complexities involved in ML systems.
Source: Google paper by Sculley
Since then the pipelines and products supporting AI/ML pipelines have evolved significantly and a recent MLOps focused paper by Kreuzberger try to illustrate the complexities in an MLOps workflow here.
领英推荐
As the field evolves, the complexities and products supporting AI pipelines are growing and the key drivers and challenges in addressing process risks involved in designing, implementing and evaluating these novel risks needs to be discussed parallel.
Focus on Data cards and Model cards trivializes the gravity of systemwide risk and a comprehensive assessment needs to be incorporated as a part of the AI Risk Management process.
?? What’s coming this week ?
In the next series of newsletters, we will try to address some of these issues and questions.
Particularly, we will discuss:?
1. Drivers for AI Process Risk
2. Scoping AI Process Risk Assessments?
3. How to do AI Process Risk assessments ?
4. Best practices in Process risk assessments
Stay tuned for these topics this week on the AI Risk Newsletter!
??Keep on learning!
?? Want to learn more? Join the AI Risk Management Certificate program developed in partnership with PRMIA ->?https://lnkd.in/eVEhyNSQ
??Many of these topics will be elaborated in the?AI Risk Management?Book published by Wiley. Check updates here ->?https://lnkd.in/gAcUPf_m
??Subscribe to this newsletter/share it with your network ->?https://www.dhirubhai.net/newsletters/ai-risk-management-newsletter-6951868127286636544/
I am constantly learning too :) Please share your feedback and reach out if you have any interesting product news, updates or requests so we can add it to our pipeline.
Sri Krishnamurthy?