How can IT auditors communicate and report on AI and ML audit findings and recommendations?

As professionals in the realm of auditing, it's imperative that we delve into the intricacies of emerging technologies like AI and ML. When it comes to communicating and reporting on audit findings and recommendations related to these domains, IT auditors need to adopt a nuanced approach. This involves not only understanding the technical aspects but also translating them into comprehensible insights for stakeholders. By employing clear and concise language, backed by thorough analysis, IT auditors can effectively convey the implications of AI and ML implementations, thereby facilitating informed decision-making within organizations.

Auditing in this area gives your audit function a great opportunity to show its value in your business. Most, if not all of your senior leaders will not understand the subject matter. This is a shame, as the fundamentals of machine learning require only a conceptual understanding of high school maths. For example a linear regression analysis is simply a line of best fit. If you can take the time as part of your audit to help them understand how machine learning works, then your audit function will have succeeded in helping them better manage the day to day risks.

If you're in a regulated financial services institution, alert and collaborate with audit colleagues who cover model risk, or at least non-model tools risk. Auditing "AI" and "ML" can be fun but don't lose sight of audit fundamentals like good governance and controls, especially your own.

Having just conducted one, it's important to understand the business and process objectives of the AI with a level of detail. More often than not, the AI systems are more explicitly measurable, but the business outcomes are not. Or requires the link between model performance and business performance

Carefully consider your customers point of view. Would they be happy if they found out you were using their data for a given purpose?

The use of AI and automated tooling is paramount in a domain that not only is changing incredibly fast, but will scale to a degree not manually manually in the traditional way. Further, with deep embedding of AI into processes and decisions, the risks manifest in the detail and are often hidden from desktop reviews and interviews

In the rare event you have access to a sandbox, you could consider using software testing approaches to see how the model behaves in stressed conditions.

Auditing AI and ML systems is quite complex, which is why new certification training programs are now appearing in cyberspace. The point to realise is AI or ML systems are purely Algorithms. When you understand the psychology of Algorithms and the processing of Algorithms, you are almost half way there. Ability to read Algorithms and understanding their use-cases is a good starting point. Liaising with the floor-shop development and security teams, understanding both their business requirements and technical specifications will go a long way in gathering information for the Audit Report.

This subject is complicated. Auditors should ensure any technical findings are explained simply, and of course focus on the potential impact.

Machine learning is pervasive across organisations, although most leaders are unaware. This is because it is bundled into software and sometimes overlooked as analytical activity. Helping your leaders understand the footprint of machine learning in your organisation will better equip them to manage the risk.

Let’s embark on a cosmic voyage through the binary stars of AI and ML audit findings. ?? 1. Data Constellations: Illuminate the Cosmos ?? The Quantum Communication Relay Astro-Clarity: Speak the language of quarks, not black holes. Cosmic Context: Set the galactic stage. Describe the AI comet’s trajectory—training data, algorithms, and warp-speed predictions. Risk Nebulae: Highlight anomalies—like rogue asteroids in your data. 2. Reporting Quasars: Illuminate the Cosmos Visual Star Maps: Create dashboards that rival Orion’s Belt. Animated Orbits: GIFs reveal cosmic truths. Watch AI models orbit—like electrons in a quantum cloud. Narrative Supernovae: Spin tales of cosmic battles—AI vs. bias, ML vs.