Does your Algorithmic auditor know how to code ?

I started my career 25 years ago as an ISO 14001 auditor. At that time, the primary tools I had was Microsoft Excel and Powerpoint. I had the domain knowledge and I would do audits based on the expectations and requirements.

Fast forward 25 years, I just finished teaching a graduate course entitled "Big Data and Intelligent Analytics" at Northeastern University. This is an intense boot-camp styled course, where over 15 weeks, students build full stack AI-applications. Along the way, they learn the problems in working with messy datasets, scaling and deployment and the pain in using multiple components and technologies.

As a summary, here are some of the projects my students worked on!

1. Data Engineering with PDF Summary Tool: Create a Streamlit app to summarize PDFs, comparing nougat and PyPDF libraries, and integrate architectural diagrams.
2. Large Language Models for SEC Document Summarization: Develop a tool for summarizing PDF documents, evaluating different libraries, and creating Jupyter notebooks and APIs for Streamlit integration.
3. Document Summarization with LLMs and RAG: Focus on automating embedding creation, data processing, and developing a client-facing application with secure login and search functionalities.
4. Data Engineering with Snowpark Python: Reproduce data pipeline steps, analyze datasets, design architectural diagrams, and integrate Streamlit with OpenAI for SQL query generation using natural language.
5. Project Redesign and Re-architecture: Review existing architecture and redesign using open-source components and enterprise alternatives, focusing on flexible, scalable, and cost-effective solutions.

More interestingly, the technologies used to build these projects involved:

1. Data Processing and Visualization:

- Streamlit

- Nougat or PyPDF libraries

- pandas-profiling

- greatexpectations

- Diagrams tool for architecture visualization

2. APIs and Integration Tools:

- FAST API

- OpenAI APIs

- Docker for containerization

3. Database and Cloud Technologies:

- Airflow

- Pinecone

- JWT (JSON Web Token)

- SQL Database

4. Machine Learning and Language Models:

- Huggingface

- LLAMA from Meta

5. Enterprise Solutions:

- Snowpark Python

- Snowflake Marketplace

- Google Bard

- Anthropic

- Cohere

Each student team spent close to 600 hours collectively over 16 weeks to build these applications. Many enterprises spend comparable time and comparable tools to build AI-based applications! With all the hoopla and the emerging profession of AI auditing,

What does this mean if you are an Algorithmic auditor invited to audit such applications ?

Well, first, Algorithmic auditing is not an arm-chair exercise, where you sit back in a conference room and critique the application. Understanding the domain and reviewing the complete architecture holistically is crucial in AI-based applications to ensure the solution is contextually relevant, scalable, and integrates well with existing systems.

Second, Algorithmic auditors often focus narrowly on the algorithm due to its direct impact on decision-making and its complexity. However, considering all components of AI-based applications is crucial for comprehensive risk evaluation. This includes data sources, infrastructure, user interfaces, and integration points. Understanding the domain ensures the solutions are contextually appropriate, addressing specific needs and challenges of the field.

Moreover, a deep understanding of the tech stack is essential for auditors to identify potential risks and vulnerabilities in AI systems, which often arise from interactions between different components rather than solely from the algorithm.

Algorithmic auditors need to know how to code and must have done extensive domain-specific work and application development themselves! So the next time, you are hiring an algorithmic auditor, show them your system architecture of your AI-based system and ask them to explain what they understand before giving them the contract!!

Sri Krishnamurthy

Promotional note: Upcoming events from QuantUniversity :

If you are interested in learning how to build LLM-based applications, QuantUniversity & CFA Society Boston is hosting a 1-day workshop next week on December 14th. The course can be attended online or in-person in Boston! Check https://www.quantuniversity.com/course-details/GENAI.html for details!