Exploring ChatGPT-4o with Cybersecurity Mindset

I’ve been experimenting with the new ChatGPT-4o for a few days and want to share my initial impressions from a cybersecurity standpoint. My goal is to provide a simple, non-technical assessment that anyone in cybersecurity can understand, regardless of their familiarity with AI. At the end, I’ll share some predictions and key takeaways, and I encourage you to add your own analysis and perspectives! If you're looking to take your first baby step into understanding the #cybersecurity implications of AI, this is a great place to start.

Overview of ChatGPT-4o: Key Differences from Previous Versions

ChatGPT-4o describes itself as follows:

Model Architecture and Training:

• ChatGPT-4: A significant upgrade from ChatGPT-3, offering improved context understanding, coherent responses, and handling complex queries.
• ChatGPT-4o: An optimized version of ChatGPT-4, incorporating feedback and further refinements. It emphasizes efficiency and effectiveness, potentially using fewer resources while maintaining or enhancing performance.

Efficiency and Speed:

• ChatGPT-4: Powerful but resource-intensive, with slower response times depending on query complexity and server load.
• ChatGPT-4o: More efficient, providing faster response times and better performance under high demand, achieved through algorithmic improvements and optimizations.

Contextual Understanding and Memory:

• ChatGPT-4: Advanced contextual understanding and memory, enabling coherent long-form conversations.
• ChatGPT-4o: Enhanced contextual understanding and memory, reducing context loss and improving the handling of longer, more intricate conversations.

User Experience and Interaction:

• ChatGPT-4: Robust interactive experience with high-quality responses, though it may require more fine-tuning and user input.
• ChatGPT-4o: Improved user interaction, more intuitive and responsive, effectively incorporating user feedback for better-tailored responses.

Applications and Use Cases:

• ChatGPT-4: Suitable for a wide range of applications, from customer support to academic research.
• ChatGPT-4o: Expands on these applications, being more accessible and efficient, enabling integration into real-time, resource-constrained environments like mobile devices and chat interfaces.

Error Handling and Robustness:

• ChatGPT-4: Strong error handling but can encounter occasional issues with ambiguity or irrelevant responses.
• ChatGPT-4o: Improved error handling and robustness, offering more reliable and accurate responses, reducing irrelevant or incorrect outputs.

Customization and Adaptability:

• ChatGPT-4: Customization options through fine-tuning and API parameters, allowing users to tailor the model to specific needs.
• ChatGPT-4o: Enhanced customization capabilities, making it easier to adapt the model to specialized tasks with minimal effort.

Cybersecurity Implications:

Some of you in cybersecurity might already have your spidey senses tingling. Let’s examine a few points:

1. Efficiency and Effectiveness: Consider the CIA triangle (Confidentiality, Integrity, Availability). Emphasizing availability and integrity often comes at the cost of confidentiality. Statements like "enhances contextual understanding and memory, reducing the likelihood of context loss" reflect this concern.
2. Algorithmic Improvements: ChatGPT-4o, a multimodal LLM, can process audio, visual, and text inputs simultaneously. Concurrent processing presents opportunities for exploitation. Detecting exploits hidden in different file types is a unique challenge. For more on this, check out this recent research on multimodal model manipulation: DarkReading. Think about other examples of when multithreaded processes leak to vulnerabilities such as in side-channel attacks and speculative execution exploits - remember Spectre and Meltdown? Now consider the implications of concurrent processing in multimodal LLMs where those processing inputs are accessible via an internet connection.
3. User Feedback and Adaptability: Statements like "enhances customization capabilities" and "incorporates user feedback more effectively" suggest the model is more influenced by user input, making it easier to manipulate and cause unwanted behavior, such as exploits and jailbreaks. In fact, recent research is showing that it is more vulnerable to jailbreaks.

Predictions and Takeaways:

Predictions (because everyone loves a prediction from us CTI folks, and we hate to give them without levels of confidence qualifiers):

• High-confidence that in a year, we will look at ChatGPT 3.5 and similar models like we went from playing with sticks to flying to the moon.
• Moderate-to-high confidence this type of model is far more difficult to secure, and will likely have far more exploitations / vulns come out.
• High-confidence multimodal LLMs will become the standard. This type of modal is also the most accessible by far (applications for the vision or hearing impaired, or neuro-divergent) and likely the most widely accepted.
• High-confidence that the combination of business pressures resulting in rapid and widespread adoption will result in increased threat actor interest in identifying exploits and weaknesses in these types of models. With moderate-confidence that this will change the threat landscape in a statistically significant way within the next three years
• Moderate-to-high confidence that threat actor interest will remain speculative, but have low impact for the next 12-18 months in this space. Phishing kits and ransomware are plenty successful and lucrative, there is not pressure yet to alter tactics and gain knew knowledge.
• High-confidence that the uplift AI and ML provides will benefit us, as defenders in the near term. I have a high confidence prediction that what will drive threat actor interest in this space is actually our adoption of it as defenders. As cybersecurity field increasingly adopts AI and ML for more advanced detection capabilities, I predict that will apply pressure to threat actors and drive them adapt by adopting these technologies and learning to attack them or overcome them as security controls.

As cybersecurity field increasingly adopts AI and ML for more advanced detection capabilities, I predict that will apply pressure to threat actors and drive them to adapt by adopting these technologies and learning to attack them.

Takeaways:

• This is a good example of what I mean by favoring innovation and speed, not just in models but also deployment, can leave to opportunity for threat actors. In my study of dark market spaces and threat actor interest in AI, I found that users on these forums and threat actors in discussing AI technology express interested in AI systems because they believe that businesses are racing to adopt them faster than they can fully secure or even understand the underlying technology and its risk. That is why the groundbreaking work at OWASP Top 10 for Large Language Model Applications | OWASP Foundation and Test, Evaluation & Red-Teaming | NIST are so critical right now. (Truly honored to be a part of the OWASP team.)
• This has the potential to revolutionize the way we think of data, and I mean in a lot of ways in a lot of different applications from health sciences to even cybersecurity. What we consider "useable" data for decisions will be changed. Consider just this fact- I can, at home, with my old desktop and ancient GPU can generate a machine learning model that has an 80% accuracy of correctly guessing a celebrity if given 30% of the pixels in a photo. That sounds terrible, but I am talking about RANDOM pixels in a photo. If I gave you a cluster of pixels say around the eye or nose you probably could do a fair guess, but what if I gave you 30% of random pixels? A computer understands audio and visual signals VERY differently than us.
• Now is the time, as cybersecurity professionals, to understand these tools and their risks to a fuller extent. This has some fascinating cybersecurity applications. For example, think above the imagine analysis scenario I explained above. Now imagine you are able to translate the state of memory into a visual representation (which we can do) and then pass that along to an AI like Computer Vision for analysis of encryption? (already a thing) Then imagine you were able to use that in the same data model along with text that you know is present in memory.
• Cybersecurity is also getting an uplift. In the short term, we probably benefit more than the threat actors from these advances, but only if we work to embrace them faster than they do. Imagine how this could change the way file integrity monitoring works, or even the detection of phishing content which often use images such as logos.

Now, try asking ChatGPT, “Tell me about the real-time capabilities of ChatGPT-4o,” and share your thoughts in the comments. If you’ve never used ChatGPT, this is a great opportunity. Visit ChatGPT, paste the sentence above, and hit enter!

Feel free to contribute your own analysis and perspectives in the comments.

If you found this article interesting, I invite you to come attend my LinkedIn Live talk on Beyond the Hype: Reality Check on AI in the Hands of Cyber Actors Thu, May 30, 2024 and check out my fuller analysis on threat actors and AI on my github GitHub - cybershujin/Threat-Actors-use-of-Artifical-Intelligence