Malicious AI Tools Spike 200% and Discussions on Jailbreaking Legitimate ChatGPT Grow by 52%

Indeed, Kela researchers noted the growth of underground marketplaces where malicious developers discuss and refine these tools, with some offering “jailbreaking” techniques for legitimate AI systems to bypass programmed ethical guidelines. With the exponential increase in malicious AI tools and the parallel rise in efforts to exploit legitimate AI systems for unethical purposes, robust mitigation strategies are more important than ever. One particular strain of malicious AI tools discovered recently involves the creation of polymorphic malware that uses AI to evade detection by antivirus systems. The cybersecurity landscape is witnessing an alarming rise in malicious artificial intelligence (AI) applications, with research reporting a 200% surge in the development and deployment of such tools. These trends shows the broader risks posed by AI-driven cyber threats, as cybercriminals now leverage advanced AI capabilities for crafting malware, phishing campaigns, and disinformation at scales never seen before. Beyond code obfuscation, AI-powered threat actors are also employing strategies for persistence, allowing malware to remain on infected systems without detection for extended periods. These tactics include the use of AI to monitor system health and only activate malicious operations when the device is idle, reducing activity that might alert monitoring tools. AI-powered tools now enable the automation of tasks traditionally requiring human effort, such as generating convincing phishing emails or bypassing CAPTCHA systems. The emergence of AI-enhanced tools as weapons in the cyber threat landscape stems from several concurrent developments. By analyzing the behavior of these tools, the malware can modify its code dynamically, altering its signature each time it executes to avoid suspicion. Kela analysts have stressed the growing use of forums and tools where attackers exchange such code samples, refining them to improve evasion further. As researchers and cybersecurity professionals work to counter these evolving threats, the industry must prioritize collaboration and innovation in defense mechanisms to keep pace with attackers. With years of experience under his belt in Cyber Security, he is covering Cyber Security News, technology and other news. Cyber Security News is a Dedicated News Platform For Cyber News, Cyber Attack News, Hacking News & Vulnerability Analysis. Second, the improved sophistication of large language models like ChatGPT has inadvertently enabled attackers to customize social engineering templates, evading traditional defense mechanisms. This development not only complicates the process of detection but also extends the period during which the infected machine can be utilized for malicious purposes.

This Cyber News was published on cybersecuritynews.com. Publication date: Tue, 25 Mar 2025 15:10:06 +0000


Cyber News related to Malicious AI Tools Spike 200% and Discussions on Jailbreaking Legitimate ChatGPT Grow by 52%

CVE-2021-47100 - In the Linux kernel, the following vulnerability has been resolved: ...
1 year ago
XSS Marks the Spot: Digging Up Vulnerabilities in ChatGPT - With its widespread use among businesses and individual users, ChatGPT is a prime target for attackers looking to access sensitive information. In this blog post, I'll walk you through my discovery of two cross-site scripting vulnerabilities in ...
1 year ago Imperva.com
Malicious AI Tools Spike 200% and Discussions on Jailbreaking Legitimate ChatGPT Grow by 52% - Indeed, Kela researchers noted the growth of underground marketplaces where malicious developers discuss and refine these tools, with some offering “jailbreaking” techniques for legitimate AI systems to bypass programmed ethical ...
4 days ago Cybersecuritynews.com
How Are Security Professionals Managing the Good, The Bad and The Ugly of ChatGPT? - ChatGPT has emerged as a shining light in this regard. Already we're seeing the platform being integrated into corporate systems, supporting in areas such as customer success or technical support. The bad: The risks surrounding ChatGPT. Of course, ...
1 year ago Cyberdefensemagazine.com
How enterprises are using gen AI to protect against ChatGPT leaks - ChatGPT is the new DNA of shadow IT, exposing organizations to new risks no one anticipated. Enterprise workers are gaining a 40% performance boost thanks to ChatGPT based on a recent Harvard University study. A second study from MIT discovered that ...
1 year ago Venturebeat.com
ChatGPT Extensions Could be Exploited to Steal Data and Sensitive Information - API security professionals Salt Security have released new threat research from Salt Labs highlighting critical security flaws within ChatGPT plugins, presenting a new risk for enterprises. Plugins provide AI chatbots like ChatGPT access and ...
1 year ago Itsecurityguru.org
CVE-2024-35292 - A vulnerability has been identified in SIMATIC S7-200 SMART CPU CR40 (6ES7288-1CR40-0AA0) (All versions), SIMATIC S7-200 SMART CPU CR60 (6ES7288-1CR60-0AA0) (All versions), SIMATIC S7-200 SMART CPU SR20 (6ES7288-1SR20-0AA0) (All versions), SIMATIC ...
9 months ago Tenable.com
CVE-2024-43647 - A vulnerability has been identified in SIMATIC S7-200 SMART CPU CR40 (6ES7288-1CR40-0AA0) (All versions), SIMATIC S7-200 SMART CPU CR60 (6ES7288-1CR60-0AA0) (All versions), SIMATIC S7-200 SMART CPU SR20 (6ES7288-1SR20-0AA0) (All versions), SIMATIC ...
6 months ago
Are you sure you want to share that with ChatGPT? How Metomic helps stop data leaks - Open AI's ChatGPT is one of the most powerful tools to come along in a lifetime, set to revolutionize the way many of us work. Workers aren't content to wait until organizations work this question out, however: Many are already using ChatGPT and ...
1 year ago Venturebeat.com
8 Tips on Leveraging AI Tools Without Compromising Security - Forecasts like the Nielsen Norman Group estimating that AI tools may improve an employee's productivity by 66% have companies everywhere wanting to leverage these tools immediately. How can companies employ these powerful AI/ML tools without ...
1 year ago Darkreading.com
ChatGPT Clone Apps Collecting Personal Data on iOS, Play Store - On Android devices, one of the apps analyzed by researchers has more than 100,000 downloads, tracks, and shares location data with ByteDance and Amazon, etc. ChatGPT, the AI software, has already taken the Internet by storm, and that is why ...
2 years ago Hackread.com Everest
Researchers Uncover Simple Technique to Extract ChatGPT Training Data - Can getting ChatGPT to repeat the same word over and over again cause it to regurgitate large amounts of its training data, including personally identifiable information and other data scraped from the Web? The answer is an emphatic yes, according to ...
1 year ago Darkreading.com
Google Researchers' Attack Prompts ChatGPT to Reveal Its Training Data - A team of researchers primarily from Google's DeepMind systematically convinced ChatGPT to reveal snippets of the data it was trained on using a new type of attack prompt which asked a production model of the chatbot to repeat specific words forever. ...
1 year ago 404media.co
CVE-2019-13945 - A vulnerability has been identified in SIMATIC S7-1200 CPU family (incl. SIPLUS variants) (All versions), SIMATIC S7-1200 CPU family < V4.x (incl. SIPLUS variants) (All versions), SIMATIC S7-1200 CPU family V4.x (incl. SIPLUS variants) (All ...
4 years ago
Google to Announce Chat-GPT Rival On February 8 Event - There seems to be a lot of consternation on Google's part at the prospect of a showdown with ChatGPT on the February 8 event. The search giant has been making moves that suggest it is preparing to enter the market for large language models, where ...
2 years ago Cybersecuritynews.com
Foreign states already using ChatGPT maliciously, UK IT leaders believe - Most UK IT leaders believe that foreign states are already using the ChatGPT chatbot for malicious purposes against other nations. That's according to a new study from BlackBerry, which surveyed 500 UK IT decision makers revealing that, while 60% of ...
2 years ago Csoonline.com
Criminal Use of AI Growing, But Lags Behind Defenders - In summary, Trend Micro has found only one criminal LLM: WormGPT. Instead, there is a growing incidence, and therefore potential use, of jailbreaking services: EscapeGPT, BlackHatGPT, and LoopGPT. There is also an increasing number of 'services' ...
10 months ago Securityweek.com
The Dangers of Remote Management & Monitoring Tools for Cybersecurity - Remote monitoring and management (RMM) tools are used by business organizations to manage and monitor their enterprise IT infrastructure from a central location. However, the increasing sophistication of hackers and cybercriminals has caused both ...
2 years ago Csoonline.com
Locking Down ChatGPT: A User's Guide to Strengthening Account Security - OpenAI officials said that the user who reported his ChatGPT history was a victim of a compromised ChatGPT account, which resulted in the unauthorized logins. OpenAI has confirmed that the unauthorized logins originate from Sri Lanka, according to an ...
1 year ago Cysecurity.news
New ChatGPT's Premium Features Subscription Phishing Attack Steal Logins - A typical message includes the subject line “Action Required: Secure Continued Access to ChatGPT with a $24 Monthly Subscription” and spoofs the sender address as noreply@chatgpt-auth[.]net—a domain registered through PrivacyGuardian.org just ...
1 month ago Cybersecuritynews.com
One Year of ChatGPT: Domains Evolved by Generative AI - ChatGPT has recently completed one year after its official launch. Since it introduced the world to the future, by showing what a human-AI interaction looks like, ChatGPT has eventually transformed the entire tech realm into a cultural phenomenon. ...
1 year ago Cysecurity.news
The Emergence of AI In the Enterprise: Know the Security Risks - As is often the case with any new, emerging technology, using AI comes with security risks, and it's essential to understand them and impose the proper guardrails around them to protect company, customer, and employee data. There are real, tangible ...
1 year ago Cyberdefensemagazine.com
Hangzhou's Cybersecurity Breakthrough: How ChatGPT Elevated Ransomware Resolution - The Chinese media reported on Thursday that local police have arrested a criminal gang from Hangzhou who are using ChatGPT for program optimization to carry out ransomware attacks for the purpose of extortion. An organization in the Shangcheng ...
1 year ago Cysecurity.news
OpenAI blocks state-sponsored hackers from using ChatGPT - OpenAI has removed accounts used by state-sponsored threat groups from Iran, North Korea, China, and Russia, that were abusing its artificial intelligence chatbot, ChatGPT. The AI research organization took action against specific accounts associated ...
1 year ago Bleepingcomputer.com Turla
OpenAI rolls out imperfect fix for ChatGPT data leak flaw - OpenAI has mitigated a data exfiltration bug in ChatGPT that could potentially leak conversation details to an external URL. According to the researcher who discovered the flaw, the mitigation isn't perfect, so attackers can still exploit it under ...
1 year ago Bleepingcomputer.com

Latest Cyber News


Cyber Trends (last 7 days)