Why You Should Not Blindly Trust ChatGPT and Its Plug-ins
Over the weekend we saw an immediate rise In the rapidly evolving world of artificial intelligence (AI), with the introduction of OpenAI ChatGPT plug-ins which have become increasingly popular for their ability to process and generate information quickly. However, my recent experiments with ChatGPT regarding cybersecurity vulnerability data have revealed significant limitations, emphasizing the need for caution and verification when using any of these tools.
The Experiment: Querying Vulnerability Data
I conducted a series of experiments to test ChatGPT's accuracy in providing EPSS (Exploit Prediction Scoring System) score trends for Adobe Flash Player vulnerabilities using new vulnerability plug-ins that were released over the last couple of days. The EPSS scores are vital in assessing the likelihood of a software vulnerability being exploited, making accurate data essential in cybersecurity.
The Initial Prompt and Misleading Response
My initial prompt requested EPSS score trends for Adobe Flash Player. ChatGPT's response detailed the scores, percentiles, and dates of a specific CVE (CVE-2023-2023). However, this CVE was incorrectly associated with Adobe Flash Player when it was actually related to WordPress, demonstrating a critical error in the AI's information processing.
领英推荐
The Alternative Prompt: A More Targeted Approach
Seeking a more comprehensive analysis, I altered my prompt to inquire about trends across all CVEs associated with Adobe Flash. ChatGPT's response here was more cautious, stating it couldn't automatically retrieve and analyze EPSS scores for all related CVEs. This response, while not misleading, showcased another limitation: the inability to process complex, multifaceted queries.
The Inconsistent Responses
Repeating the original prompt yielded a completely different answer, focusing on a different CVE (CVE-2021-21092) related to Adobe, but not specifically to Flash Player. This inconsistency in responses further illustrates the challenges AI faces in understanding and accurately responding to specialized queries.The Implications of Inconsistency and InaccuracyThese experiences with ChatGPT highlight several critical issues:
The Need for Caution and Verification These limitations underscore the importance of approaching AI tools like ChatGPT with skepticism, especially in specialized fields like cybersecurity. While AI can provide quick answers, it's difficult to depend on artificial intelligence where the data and training of the model for your use case can likely not be trusted. This demonstrates the importance in having confidence in the datasets being used and how the AI model is trained. Users must be default to zero trust in the output, cross-checking AI-provided information with reliable sources.
Information Security and AI | Building tools for implementors & auditors | Founder @ ISMS Copilot | Sharing learnings along the way
8 个月I agree. The GPT-based chrome plugins pose even more security doubts/risks of shadow IT (and data leakages) in many startups
Cybersecurity RedTeam Engineer III at Accuray , Biohacking Village CTF Admin , Lecturer, and Fun Guy.
11 个月I think many people look to ChatGTP with a failed understanding that it's answers are logically deduced and thought out by AI towards a logical decision, in truth: it is a language model and its choice off words is not a deduction of logical thought but a result of an amalgamation of whatever the majority of the scraped internet said. If you ask it the best color and it says blue, it is because the language model recognized the internet generally thought that. ChatGTP doesn't think. It references with a great language model. That being said, the responses fell so unique and contured to our need we forget how that data came to be! Data collection... and everything you type, say, reference is also being collected and ready to be reused. This of course points to the danger of your data being rehashed to another user, no matter the promises to the contrary.
Great analysis, I would like to emphasize the importance of the OWASP Top 10 LLM, especially LLM09: Overreliance ??
Great dad | Inspired Risk Management and Security Profesional | Cybersecurity | Leveraging Data Science & Analytics My posts and comments are my personal views and perspectives but not those of my employer
11 个月Totally agree … people should not trust blindly. They should apply the old saying “trust but verify”. The GPTs are awesome for certain simple tasks but the more complex tasks the more scrutiny you must apply.