Defining the purview of an Algorithmic Assessment - Process
Sri Krishnamurthy, CFA, CAP
CEO, QuantUniversity | Entrepreneur | AI Innovation | Education | Author | Speaker
My mother is big into jig-saw puzzles. She loves the challenge, the intricacies and the possibilities! I remember she waking up some nights and seeing her trying to complete the puzzle. Over time, she has developed processes. She sorts pieces by size and colors, she focused on chunking the areas, she created parts and then assembled the whole.
Not to brag about my mom, but she can now finish any puzzle and we have a few fully made and puzzles hanging in our home! For me the interesting part here is how she follows her process meticulously. For her, every time she gets a puzzle done, it is affirmation that her process works!
Process is the most important thing, from engineering and software development for a product to be successful in production. I would claim, a sub-optimal product that has gone through a thorough design, development and testing process has a much higher chance of success than a superior product which may not have gone through a thorough process. Startups and fintechs are particularly vulnerable here. Many superior and innovative products fail to gain adoption especially in high-critical applications since the risks of failure or non-availability is too high! Thus many sub-par products continue to be used since?we stick to the old adage:
“Better?the?devil?you know,?than?the?devil?you don't”
Algorithmic decision systems fall under this category. Many times, a cumbersome, sub-par, sub-optimal workflow is being replaced by an algorithmic decision making system and the bar for the replacement is very high! Any failures or behaviors would raise red-flags and companies typically gravitate towards status-quo.
How do we address this? Well, like any product, as the product matures and is ready for production, the focus should be on the process! We elaborated aspects of process risk in prior newsletters. Please review them for context. One thing I would like to add here is the last mile problem.
?? The last mile is where things go wrong!
Everything works perfect in development environments, test beds, sandboxes till it deployed in production. The last mile creates so many uncertainties that special attention should be paid to address potential issues. This again is a process issue!.?
You cannot address all things that could go wrong in the design, development and testing stages. You will never be able to have a product that is 100% defect free.
But you can define processes upfront on how to handle uncertainties and things which pop up unanticipated. That’s where What-if analysis, Stress testing and Scenario analysis are required parts of testing of an algorithmic decision system.
领英推荐
?? The case study
In the case study of XYZ bank, we will have to address the design, development, testing and deployment aspects of the system. We will have to evaluate the processes involved in the entire workflow in the lifecycle of the system. Notice that we are not talking just about the model, or data or the environment in which the models operate. We are talking about an entire system and the processes that support it. Also, the process assessment is not about just reviewing the socio-technical aspects of the system. The business process risks for the task at hand should be the focus rather than just the technical aspects. Technology and the algorithms are just enablers of the business and the right stakeholders should be involved to address the associated risks!
Now that we have discussed the four elements, data, model, environment and process, let’s discuss the formal scoping of the algorithmic assessment factoring the four elements!
?? In the news
A test car with autonomous steering capability veered into oncoming traffic in Germany, killing one person and seriously injuring nine others. Testing autonomous systems is hard! The risks are real! https://apnews.com/article/technology-germany-8a2333532b91635d5b214485043ed94a?s=09&?
??Keep on learning!
?? Want to learn more? QuantUniversity is offering a LIVE Algorithmic Auditing course on August 22nd 9.30 AM to 4.00pm in partnership with PRMIA. If you are interested, check out details here:?https://prmia.org/Shared_Content/Events/PRMIA_Event_Display.aspx?EventKey=8906
??Many of these topics will be elaborated in the?AI Risk Management?Book published by Wiley. Check updates here ->?https://lnkd.in/gAcUPf_m
??Subscribe to this newsletter/share it with your network ->?https://www.dhirubhai.net/newsletters/ai-risk-management-newsletter-6951868127286636544/
I am constantly learning too :) Please share your feedback and reach out if you have any interesting product news, updates or requests so we can add it to our pipeline.
Sri Krishnamurthy?