An Interview with ChatGPT: How Will AI Revolutionise Internal Audit - And Will It Actually Work?

As a tech enthusiast, I've been blown away by the impact ChatGPT has made over the past few months. Whilst it isn't the first chatbot out there, it's ability to generate coherent and contextually appropriate responses across so many different areas is pretty amazing. For me, it's ChatGPT's ability to write and document code that has really impressed me. And I'm not the only one, seeing as Microsoft are investing $10 billion in ChatGPT and have plans to integrate it into Azure.

It's crazy to think that the following paragraph was written by a chatbot based on a simple prompt. Sure, it's not perfect and some of the sentences could use some work, but it's clear that the gap between human and AI is starting to close.

Chatbots and AI are expected to have a significant impact over the next decade, revolutionizing a wide range of industries. They are expected to automate repetitive tasks, process vast amounts of data quickly and accurately, and enable businesses to make more informed decisions and improve their performance. However, it is important to also consider the potential negative impacts, such as job displacement and economic inequality, as well as moral and ethical implications. It is crucial for businesses and policymakers to carefully consider the potential pros and cons and take steps to mitigate any negative effects, such as investing in retraining programs for displaced workers and implementing regulations to ensure accountability and transparency.

Pretty neat. So with that, I decided to sit down and "interview" ChatGPT. I wanted to find out how it works and put some of its capabilities to the test.

Thank you for taking time out of your busy schedule, I've noticed your servers have been under a lot of strain over the last few weeks. For those who don't know you, can you please introduce yourself and explain some of your capabilities?

I am ChatGPT, a language model developed by OpenAI. I am trained to understand and generate human language, which allows me to respond to a wide range of questions and prompts. Some of my capabilities include understanding natural language text, answering questions, writing creative and coherent text, and even generating computer code. I can be used for applications such as chatbots, automated writing and content creation, and even language translation. However, my knowledge is based on the data that I was trained on, which had a cutoff date of 2021, so I may not have the most up-to-date information.

Can you expand on where your information comes from and how you use it to generate your responses?

I am trained on a massive dataset of text, which includes a wide variety of sources such as books, articles, and websites. During training, my creators exposed me to this text so that I can learn patterns and relationships within the data, which allows me to generate responses that are similar to the text that I was trained on.

When I am given a prompt or question, I use the patterns and relationships that I learned during training to generate a response. Essentially, I use statistical models to predict the next word in a sentence, based on the words that came before it. As I keep predicting the next word and building the response, I use my internal logic and programming to provide a coherent and sensible answer.

It's important to note that my responses are based on patterns and associations I've learned from the text I've been trained on, so I don't have an inherent understanding of the world. And I can only provide information that I've seen in the training data and my responses may not always be accurate or up-to-date.

So just like the autocorrect feature on your mobile but more intelligent?

In a way, yes. Autocorrect on your phone uses a similar process of predicting the next word based on the words that came before it. However, my capabilities are much more advanced than a simple autocorrect feature. I am able to understand and respond to a wide range of questions and prompts, and I can generate text that is much more complex and creative than simple word prediction. Additionally, I can do things like answer questions, write stories, and even generate computer code. While autocorrect is mainly used to correct spelling and grammar errors, my abilities are much more versatile, and I can be used in various applications.

Let's trying something different. I'd like you to come up a riddle but don't give me the answer just yet. This needs to be a completely new and unique riddle. Do not use an existing riddle, make one up.

Sure, here is a riddle for you:

"I am a container, but I don't hold anything. I am a door, but I can't be opened. I am a path, but I have no destination. What am I?"

No idea, pass.

The answer to the riddle is a map, as it is a container that shows different paths and places, but it doesn't hold anything physically, it's a door to navigate through different places and a path that guides you but it can't be opened or has no specific destination.

Now I'd like you to write a recipe for a blog. In the typical fashion of a post on a food blog, start with a backstory of why you like this recipe and what it means to you. The recipe is for Carbonara which you learnt from your Grandad. He was abducted by aliens. You and your grandad are paranoid that you are being monitoring by the aliens who want to get your Grandad's Carbonara recipe.

Carbonara is a classic Italian dish that is near and dear to my heart. My grandad, who was quite the adventurous cook, was abducted by aliens during one of his travels. He was always a bit of a conspiracy theorist, and had always been convinced that one day, he would be taken. And one day, it happened. He was gone for weeks, and when he returned, he had this recipe for Carbonara that he had learned from the extraterrestrial chefs. He was convinced that the aliens were still monitoring him, and that they would come back for him. So, he made me swear to keep the recipe a secret.

Ever since then, this recipe has been a staple in my family's kitchen. My grandad passed away a while ago, but I still make this dish in his memory. The dish is simple to make but packs a punch of flavor. The key to a good Carbonara is using high-quality ingredients and being careful not to overcook the pasta. This recipe serves 4 people.

Ingredients:

• 8 oz spaghetti
• 4 oz pancetta or bacon, diced
• 2 cloves of garlic, minced
• 3 large eggs
• 1/2 cup grated parmesan cheese
• Salt and pepper to taste

Instructions:

1. Start by bringing a large pot of salted water to a boil. Add the spaghetti and cook for 8-10 minutes or until al dente.
2. While the pasta is cooking, heat a large skillet over medium heat. Add the pancetta or bacon and cook until crispy. Remove the skillet from heat.
3. In a small bowl, whisk together the eggs, parmesan cheese, and minced garlic. Season with salt and pepper to taste.
4. Once the pasta is done, reserve 1 cup of the pasta cooking water and then drain the pasta.
5. Add the pasta to the skillet with the pancetta or bacon. Turn the heat to low and add the egg mixture. Toss everything together quickly, adding a little bit of the reserved pasta water if the sauce is too thick.
6. Serve the pasta hot, with some extra parmesan cheese and black pepper on top.

Now, I know it might sound crazy but my grandad's paranoia rubbed off on me a bit, so I'm sharing this recipe with you on the down low. Let's just keep this between us, okay? I hope you enjoy this recipe as much as I do. The combination of salty pancetta or bacon, rich egg, and parmesan cheese creates a decadent and satisfying dish. And the story behind it, makes it all the more special.

Fun story, I'll make sure not to share it with anyone. Please convert the ingredients into metric.

Sure, here is the recipe for Carbonara converted to metric measurements for your convenience:

?Ingredients:

• ?227g spaghetti
• 113g pancetta or bacon, diced
• 2 cloves of garlic, minced
• 3 large eggs
• 120ml grated parmesan cheese
• Salt and pepper to taste

Potential Use Cases for Internal Audit

I wanted to put ChatGPT to the test and see some potential scenarios where AI could be used to support an auditor. These scenarios range from text generation to support all audit activities from planning to documentation and well as writing data analytics scripts.

Let's change gears now and I'll ask you some questions focusing on Internal Audit. Assume you are an internal auditor working on an IT Controls audit. How would you conduct an audit in this domain and what audit procedures would you follow?

As an internal auditor working on an IT controls audit, my main objective would be to assess the effectiveness of the organization's IT controls, which includes evaluating the design and implementation of controls to mitigate IT risks such as unauthorized access, data breaches, and system failures.

To conduct an IT controls audit, I would follow a systematic approach that includes the following steps:

• Planning: Define the scope of the audit, identify the objectives, and develop an audit plan that outlines the resources, timelines, and specific audit procedures to be performed.
• Risk assessment: Identify and assess the IT risks that the organization is exposed to and prioritize those risks based on their likelihood and impact.
• Control assessment: Evaluate the design and implementation of IT controls in place to mitigate identified risks. This includes assessing the controls around access management, data integrity, change management, and incident management.
• Testing: Perform testing of controls to ensure they are operating effectively and in compliance with established policies and procedures. This could include testing access controls, reviewing system logs, and testing data backup and recovery processes.
• Reporting: Prepare a report that summarizes the findings, conclusions, and recommendations resulting from the audit.
• Follow-up: Track and report on the implementation of any recommendations made in the audit report.

It's important to note that the audit procedures will vary depending on the specific IT environment and the scope of the audit. The overall process should be risk-based, cost-effective and comply with the professional standards.

What data analytics tests could you perform on this audit?

As an internal auditor working on an IT controls audit, there are several data analytics tests that I could perform to help assess the effectiveness of the organization's IT controls. Some of these tests include:

1. Data Sampling: I could use statistical sampling techniques to select a sample of transactions or data points to test for compliance with policies and procedures. This can be done by selecting a random sample of transactions, or by using a more targeted approach that focuses on high-risk areas.
2. Data extraction and analysis: I could extract data from various systems and applications (such as databases, system logs, and audit trails) and use data analytics tools to analyze the data for patterns and anomalies. This could include identifying unusual access patterns, detecting potential data breaches, or identifying potential fraud.
3. Data visualization: I could use data visualization tools to help identify patterns and trends in the data, which can be useful in identifying potential areas of risk. This could include creating heat maps, histograms, or other types of visualizations to help identify potential issues.
4. Benford's Law analysis: I could use Benford's Law analysis to detect anomalies in data sets, such as fraud, errors or missing data. It examines the distribution of leading digits in a data set and can be used to identify deviations from expected patterns.
5. Analytical procedures: I could use analytical procedures such as ratio analysis and trend analysis to identify any unusual patterns in the data. This could include identifying unexpected changes in financial metrics or identifying unusual patterns in system logs.
6. Control testing: I could perform testing of controls using test data or testing tools to ensure that the controls are working as intended, and they are able to detect, prevent or correct any identified risks.

It's important to note that the specific data analytics tests performed will depend on the scope of the audit, the risks identified and the specific systems and data being audited.

I'd like you to write a Python script for point 4 - identifying unexpected changes in financial metrics or identifying unusual patterns in system logs.

Sure, here is a sample Python script that could be used to identify unexpected changes in financial metrics or unusual patterns in system logs:

import pandas as p

?

# Load data into a pandas DataFrame

df = pd.read_csv('financial_data.csv')

?

# Identify unexpected changes in financial metrics

# Calculate the percentage change in revenue over time

df['revenue_change'] = df['revenue'].pct_change()

?

# Identify any changes in revenue greater than a specified threshold

threshold = 0.1

unexpected_revenue_change = df[df['revenue_change'] > threshold]

?

print(unexpected_revenue_change)

?

# Identify unusual patterns in system logs

# Load system log data into a pandas DataFrame

log_df = pd.read_csv('system_logs.csv')

?

# Calculate the number of logins per user

logins_per_user = log_df.groupby('username')['timestamp'].count()

?

# Identify any users with an unusually high number of logins

threshold = 1000

unusual_logins = logins_per_user[logins_per_user > threshold]

?

print(unusual_logins)        

Please keep in mind that this is a simple example, and you might need to adjust it based on your specific requirements and data structure. Also, this script doesn't include any error handling, it is just for illustrative purposes.

Please provide documentation for this code and explain in plain & non-technical English how it works.

This script is written in Python and uses the Pandas library to perform data analysis on financial data and system log data.

First, the script loads the financial data into a pandas DataFrame using the pd.read_csv() function, which reads a CSV file and converts it into a DataFrame. This DataFrame is named df.

Then, it calculates the percentage change in revenue over time by using the pct_change() function, which calculates the percentage change between consecutive rows of the 'revenue' column of the DataFrame. This is stored in a new column called 'revenue_change'.

After that, it identifies any changes in revenue that are greater than a specified threshold. The threshold is set to 0.1. It creates a new DataFrame called unexpected_revenue_change which contains the rows of the original DataFrame where the 'revenue_change' is greater than the threshold.

Next, the script loads the system log data into another DataFrame using the pd.read_csv() function, in this case, the DataFrame is named log_df. It then calculates the number of logins per user by grouping the data by the 'username' column and counting the number of timestamps. This is stored in a new variable called logins_per_user.

Finally, it identifies any users with an unusually high number of logins by using a specified threshold of 1000. It creates a new variable unusual_logins which contains the users with more than 1000 logins.

Overall, the script is using the Pandas library to load, manipulate, and analyze data. It reads the data from CSV files, perform some calculations and filters on it, and then identifies unexpected changes in financial metrics and unusual patterns in system logs.

How would you perform the same test in Excel?

Here is an example of how you could perform a similar analysis in Excel:

1. To calculate the percentage change in revenue over time, you can use the formula =(B2-B1)/B1 in the cell next to the first revenue value, and then copy and paste that formula down the column to calculate the percentage change for all revenue values.
2. To identify any changes in revenue greater than a specified threshold, you can use the formula =IF(C2>0.1,B2,"") in the cell next to the first revenue value, and then copy and paste that formula down the column to identify all revenue values greater than the threshold.
3. To identify any users with an unusually high number of logins, you can use the COUNTIF function to count the number of logins per user, and then use the IF function to identify any users with an unusually high number of logins, for example =IF(COUNTIF(B2:B1000,A2)>1000,A2,"")
4. To filter data based on certain conditions you can use the filter option in the data tab, or use the advanced filter.
5. To visualize data, you can use charts, pivot tables and other visualization tools to highlight patterns and trends in the data.

Please note that, the above example is given assuming that your data is structured in a specific way and the specific formulas.

Final question. An Accounts Payable audit has identified the following observation, please provide some recommendations and next steps - Lack of Segregation of Duties regarding controls over accounts payable.

Recommendation: To improve internal controls over accounts payable, it is recommended to implement a segregation of duties policy. This can be achieved by separating the responsibilities of creating vendor invoices, approving vendor invoices, and making payments. By separating these responsibilities, it makes it more difficult for a single person to commit fraud or make errors without detection.

Next Steps:

• Assign specific roles and responsibilities for the various functions related to accounts payable, such as invoice creation, invoice approval, and payment processing.
• Document these roles and responsibilities in a policy or procedure manual.
• Provide training to employees on their roles and responsibilities in relation to accounts payable.
• Monitor compliance with the segregation of duties policy through regular reviews and audits.

A Word Of Warning

Really impressive responses given, the possibilities are endless and we are only scratching the surface of things to come. Am I losing sleep over it and will it replace my job in the coming years? No. Many of the responses given are crude, a little generic (unless given detailed directions) and need a couple of re-iterations before they get anywhere close to an answer written by a domain specialist.

The code examples given are good, considering they were written based on a simple prompt with no data. They still need a lot of human moderation, see thresholds and row numbers in the Python and Excel examples above. What the code answers are missing is the context of the data that will be analysed. Having said that, I don't think I'll ever trust an AI model connected to the internet with sensitive or personal data.

When describing ChatGPT and similar AI technologies to a non-technical audience, I compare it to an autopilot on an airplane. An autopilot still requires a human pilot to monitor its performance and make important decisions during takeoff and landing. Similarly, a chatbot can assist with certain tasks, such as generating text or answering questions, but they still require human oversight and input to ensure their accuracy and appropriateness.

Just like an autopilot, AI technologies are designed to help humans to perform their jobs better, but they are not designed to replace them.

It's Just Another Tool...

Internal Audit functions and organisations more widely should be building a digital mindset and embracing disrupting technologies in order to stay competitive and relevant in today's technology-driven environment.

Advanced analytics and machine learning technologies should be seen as just another tool. A heavy focus should be placed on upskilling staff and updating internal processes and methodologies in order to fully realise the potential of these technologies and to avoid being left behind.

(All opinions?expressed in this article are solely?my own)