Who should your next hire be? The emergence of the AI Validator Role

The way AI models have changed

As the AI-as-a-service model is becoming the norm of the day with OpenAI, Google, Amazon etc. all serving various models through APIs, it is imperative that organizations consider the skills needed when incorporating these new products intro their workflows.

Gone are the days when recruits could show their GitHub repos with a bunch of projects forked from Kaggle and every project, more or less, started with loading the data, split to training/testing, build a model, evaluate and display the metrics and call it done!

Today, you don't have to build start from scratch to build many models. You should be conversant with using APIs and be able to call these APIs with specific inputs and parameters and voila, you get results!! You may have to fine-tune the model, retrain with additional data or choose different flavors of models. But very few companies, especially the ones trying to use Large Language Models are building models from scratch. Now the big questions is:

How do you test if the outputs are valid, borderline-accurate or absurd?

Enter the AI Validator!

1. An AI Validator is an emerging new role in the AI-driven enterprise! An AI validator may not be fully conversant with how the models were built. But armed with excellent investigation skills, a knack to elucidate responses from models through effective interrogation, and at times, crafting sneaky questions to make the models confess or spill out secrets, the AI validator knows how to test models!
2. An AI Validator also knows the business context well. He/She (Sorry, I am not including automated testing platforms for testing yet!) understands the users well, understands where the models are going to be used, understands the technology infrastructure and the role of the model in accomplishing the business use cases.
3. An AI Validator also understands the impact of a malfunction/misbehavior of the model. He/She can contextualize the material and other risks associated with the improper use or unexpected behaviors and tests what aspects should be trusted to the model and what checks and balances should happen before the model's decisions are used for business decisions.
4. An AI Validator is also an effective challenger and sometimes has few friends in the enterprise. He/She is the gatekeeper before the models are rolled out to production and is under pressure to dumb down requirements at times!
5. An AI Validator has to work with limited budgets. While most budgets are dedicated to build models, an AI validator should plan processes and workflows that are comprehensive and yet cost effective.
6. An AI Validator is in the eye of the storm when something bad happens. An effective AI Validator has done the due diligence and has planned mitigation and control measures to address issues. An AI Validator also ensures there is sufficient monitoring in place to catch issues faster to mitigate damage.
7. Lastly, an AI Validator is humble and is constantly learning. As the models improve in sophistication and has feature changes, an AI validator, keeps track of these innovations or changes and ensures the enterprise adopt or plan for these changes.

As you can see, none of these are taught in undergraduate/graduate programs in Data science and Machine learning today! Many students and professionals pick these skills on the job as they are bombarded with processes, mandates, rules and regulations and innovations and products brought by the industry to the enterprise.

It is high time we train the AI Validator well. We at QuantUniversity have a whole 6-month program partnering with PRMIA - Professional Risk Managers' International Association to address AI Risk Management. (https://quantuniversity.com/course-details/mlrisk.html). As the industry is changing, we are also making new changes to address the novel innovations and challenges in AI Risk Management.

So the next time, you plan to hire someone new to validate AI models, think like an AI Validator and ask questions! Again, someone who has built a lot of models but has tested only a few and doesn't have the appetite to test models may not be the AI Validator you have plan to hire!!!

Sri Krishnamurthy, CFA, CAP

QuantUniversity