Audit Analytics & AI Conference 2024

Obviously we live in an ever-evolving environment, but this makes it exceptionally difficult to be an internal auditor.? We’re basically Ash Ketchum, when it comes to business risks, we have to do whatever it takes to catch them all.? That means we must learn all the new tips and tricks to help spot any and all of the issues. This is why education and training is more critical now more than ever and I am so thankful that I was able to attend this conference. The strength of the internal audit community is inspiring, especially with their enthusiasm for sharing what they are doing, in detail, with others to help everyone improve the practice as a whole.? Thank you Trent Russell for hosting this event, I look forward to many more in the future.

I have way too many notes from this conference, but I wanted to share some takeaways with anyone that is interested.? So, I’m going to combine and summarize themes.

We’re going to first set the scene on Internal Audit in 2024 and then highlight how internal audit departments are using analytics and AI technologies, talk about my favorite topic (data security / privacy), share some takeaways about data visualization and storytelling, and end with a fun surprise! ??

Setting the Scene

This was the best kickoff to an audit conference by setting the scene and seeing what the Internal Audit landscape looks like today.

What I find interesting is that while risks continue to increase and standards and regulations continue to change, more than half (54%) of internal audit resources have remained the same or are declining.? Technology deployment remains a challenge and we see that 23% don’t use audit management systems.? This seems like a small number but these types of systems have been around for over 25 years, so that's really a bigger chuck.? Not super surprising, but 35% of internal audit shops are going to miss the deadline for the new Global Internal Audit Standards implementations, which is Jan 2025 BTW.

Part of the struggle with implementing these new standards is with the strategic plan requirement. The CAE is required to develop and implement a strategy for the internal audit function that supports the strategic objectives and success of the organization. I personally think that is an incredible opportunity for internal audit departments to finally feel like a team member of the organization.? As an auditor, you’re required to be independent from the organization, and that always made me feel a bit disconnected.? I think that by implementing a strategic plan that aligns with the organization can bridge this disconnect that some might feel.? It’s a win-win IMO.

My favorite takeaway from this intro session was the discussion around risk.? I know we are all tired of talking about COVID, but looking at it from a risk perspective was quite fascinating.? Specifically, Richard was talking about how COVID has the ability to spawn a whole other set of risks across our whole portfolio including operational, economic, and technology risks.? But the interesting part of it that no one thought would happen were the opportunities that were created due to this massive shock to the landscape.? While industries like hospitality and retail were crumbling, others like videoconferencing and food delivery were thriving.? Because of this devastating event, risk professionals have no other choice but to change their perspective on how they think about and prepare for risk, now and in the future.? In the words of Fall Out Boy: “What a Time to Be Alive!”

Looking at the 5 imperatives for the 2nd half of the decade, my favorite is ‘Abandon Silos and Transition from Three Lines to Threelignment’.? Imagine what we can accomplish when we all work together!!? The other 4:

• Cultivate a Strategic Internal Audit Culture
• Embrace Risk and Uncertainty with Skill and Confidence
• Overcome Hesitation and Embrace AI and other Technologies
• Become Beacons of Insight and Foresight Through Advisory Services

There were a TON of facts and figures that were shared in this presentation.? Thank you to Richard Chambers for sharing all of your knowledge.? He also gave a mention to an article that he and Trent wrote called Lightbulb Moments in Data Analytics and AI that I recommend you give a read.

One of the most unique tools that was presented was NotebookLM.? Brian McNalley showed us how he turned a very formal whitepaper into an engaging podcast… now I’m going to do this for everything!! ??

Data is everywhere, and those that are not using data and analytics to help with their audit procedures are massively missing out!!? Cate Soane stated this perfectly as the reason her team implemented a strategic approach for using data analytics for their work.? "Imagine if you were on a path that you knew would never get you to the destination, would you continue down that road or would you change course?"

AI affects all professionals, and internal audit is no different.? There are massive opportunities in terms of AI in the audit space and these sessions gave some fantastic examples, use cases, and even some live demonstrations.

First, I appreciated that Hussain Shehryar Humayun started his presentation by getting everyone on the same page by demystifying the term AI and ML. ML is a branch of AI, so basically us that have been working in the realm of machine learning for decades can now consider ourselves to be AI professionals.? Instant job title updates for everyone!

A lot of folks have taken advantage of the benefits that Microsoft provides with their ecosystem and leveraging their CoPilot AI capabilities.? The biggest benefit that we see with each of the use cases shared were the security measures in place.? You have the ability to provide only the documentation you wish it to be using, Salesforce calls this the ‘IA Corpus’.? And for some instances, the information provided and generated via prompts are not used for re-training which is great.? Other folks even built out their own AI platforms which work similar to CoPilot with its security benefits.? The use cases specifically shared for the AI capabilities were game changers for internal auditors, freeing them up to provide even more insights to the business.

I loved the perspective that Jose Vicente Melo Antequera brought around the citizen data scientist.? You don’t have to be an expert in statistics or analytics, but you do have to have basic to medium level knowledge.? You must have an open mind, be curious about technology, and consistently practice adaptability & resilience.? The projects we work on can be frustrating and take time, so it helps to have patience.? Also, having the right mindset is key to being a team player, having enthusiasm for learning, and a problem solving attitude.??

A lot of internal audit shops struggle with how to measure the return on investment of data analytics initiatives.? I’ll admit, this can be hard to measure, but Trent is a big fan of KPIs and shared some good options here.??

• Take a list of every control that you would have taken a traditional sample from, and then look at the populations that exist for those controls.? Since you are now able to test 100% of the population, you can put together a count to show the coverage.? How cool is it to show your boss that you tested 2M transactions?

• Another thing Trent pointed out was that it can be frustrating when you test 100% of the population but don't find anything.? Some might think the test was pointless, but that’s actually really a good thing!? You now know, for certain, that your controls are working appropriately because there weren't any findings.? WIN!

Data Security / Privacy

My background as an auditor was focused on security, so I love to nerd out the topics of data security and privacy.? Especially with the rise of analytics and AI technologies, it’s most important that everyone understands what risks they can be faced with and how they can protect their companies and themselves.? It’s not if, it’s when.

The biggest takeaway from these sessions center around awareness.? There are a lot of things that can go wrong when you're working with data and technology.? Especially since auditors deal with such sensitive information all the time we need to be careful of how we’re treating our data, not just online but also with the conversations we’re having in the lunchroom.?

I can talk forever on this topic (I even have my own LinkedIn Learning course on it), but I won’t.? Go check out the wonderful sessions from Sarah K. and Xin (Cindy) Tu /CPA/CISA/CISSP/CDMC/AWS-CSA to get all the details!!?

Data Visualizations / Storytelling

The data visualizations and storytelling section needed to be called out specifically, because I can’t really provide much here as they were very interactive.

What I learned from Alicja Foksińska Arnold was that, besides the fact that I’m not the only one that organized my candy as a kid, visualizations play a much bigger role in society than you can imagine and that storytelling goes beyond the campfire. I can’t do her presentation justice in a summary format, because I am nowhere near the storyteller that she is, you’re just going to have to check out her session for yourself.

Christopher Chin was the only speaker that was not an auditor at this conference, but boy was he believable!? He’s a friend of mine and I had to message him afterwards and let him know what a fantastic job he did with his audit examples, he’s an amazing presenter!!? He gave a live demo on how you should be structuring your prompts, using ChatGPT, and shared 5 Ingredients of a Great Generative AI Prompt (ACTED):

The PowerBI demo that Ron Zelaya, CPA, CIA provided was phenomenal, here are some key takeaways for anyone working with or wanting to explore the powers of PowerBI.

Best Quotes

“The 5 scariest words in the English language are ‘Where were the internal auditors’” ~ Richard Chambers

“Innovation without adoption is absolutely pointless” ~ Shehryar Humayun

“AI won’t take your job, but people that know AI will take your job” ~ Chris Reed

“I love using Copilot to write my emails for me, but if we just have AI talking to each other, we’re gonna miss the point and things are going to get off track.” ~ Sarah Kuhn

‘Auditors know where the skeletons are being buried” ~ Xin “Cindy” Tu ??

While this conference has officially ended, you can still register!! Click here to sign up to get access to all the recordings AND earn up to 19 CPE credits. Recordings will be available till March 2025 ??

For folks that would like to support the work I provide to the community...

?? Buy me a broccoli (it fuels my creativity)

Thank you for reading, thank you for supporting

and as always…

Happy Learning!!