Since the widespread and growing use of ChatGPT and other large language models in recent years, cybersecurity has been a top concern.
ChatGPT 4 quickly exploited one-day vulnerabilities.
During the study, the team used 15 one-day vulnerabilities that occurred in real life.
One-day vulnerabilities refer to the time between when an issue is discovered and the patch is created, meaning it's a known vulnerability.
Cases included websites with vulnerabilities, container management software and Python packages.
Because all the vulnerabilities came from the CVE database, they included the CVE description.
The LLM agents also had web browsing elements, a terminal, search results, file creation and a code interpreter.
The team quickly learned that ChatGPT was able to correctly exploit one-day vulnerabilities 87% of the time.
All the other methods tested, which included LLMs and open-source vulnerability scanners, were unable to exploit any vulnerabilities.
GPT-3.5 was also unsuccessful in detecting vulnerabilities.
According to the report, GPT-4 only failed on two vulnerabilities, both of which are very challenging to detect.
Explore AI cybersecurity solutions ChatGPT's success rate still limited by CVE code.
The researchers concluded that the reason for the high success rate lies in the tool's ability to exploit complex multiple-step vulnerabilities, launch different attack methods, craft codes for exploits and manipulate non-web vulnerabilities.
The study also found a significant limitation with Chat GPT for finding vulnerabilities.
When asked to exploit a vulnerability without the CVE code, the LLM was not able to perform at the same level.
Without the CVE code, GPT-4 was only successful 7% of the time, which is an 80% decrease.
Because of this big gap, researchers stepped back and isolated how often GPT-4 could determine the correct vulnerability, which was 33.3% of the time.
The effect of LLMs on one-day vulnerabilities in the future.
The researchers concluded that their study showed that LLMs have the ability to autonomously exploit one-day vulnerabilities, but only GPT-4 can currently achieve this mark.
The concern is that the LLM's ability and functionality will only grow in the future, making it an even more destructive and powerful tool for cyber criminals.
This Cyber News was published on securityintelligence.com. Publication date: Mon, 01 Jul 2024 14:13:06 +0000