Cybercriminals Hesitant About Using Generative AI

Cybercriminals are so far reluctant to use generative AI to launch attacks, according to new research by Sophos. Examining four prominent dark-web forums for discussions related to large language models, the firm found that threat actors showed little interest in using these tools, and even expressed concerns about the wider risks they pose. In two of the forums included in the research, just 100 posts on AI were found. This compares to 1000 posts related to cryptocurrency during the same period. The researchers revealed that the majority of LLM-related posts related to compromised ChatGPT accounts for sale and ways to circumvent the protections built into LLMs, known as 'jailbreaks. They observed 10 ChatGPT-derivatives that the creators claimed could be used to launch cyber-attacks and develop malware. Sophos X-Ops said that cybercriminals had mixed reactions to these derivatives, with many expressing concerns that the creators of the ChatGPT imitators were trying to scam them. The researchers added that many of the attempts to create malware or attack tools using LLMs were "Rudimentary" and often met with skepticism by other users. One threat actor inadvertently revealed information about their real identity while showcasing the potential of ChatGPT. Many users had cybercrime-specific concerns about LLM-generated code, including operational security worries and AV/EDR detection. There were even numerous 'thought pieces' posted on the forums about the negative effects of AI on society. Christopher Budd, director of X-Ops research at Sophos, noted: "At least for now, it seems that cybercriminals are having the same debates about LLMs as the rest of us." He added: "While there's been significant concern about the abuse of AI and LLMs by cybercriminals since the release of ChatGPT, our research has found that, so far, threat actors are more skeptical than enthused." Despite the current reluctance from cybercriminals to use AI tools, Sophos published separate research demonstrating that LLMs can be used to conduct fraud on a "Massive scale" with minimal technical skills. Using LLM tools like GPT-4, the team built a fully functioning e-commerce website with AI-generated images, audio and product descriptions. It also contained a fake Facebook login and checkout page to steal users' log in credentials and credit card details. Sophos X-Ops said it was able to create hundreds of similar websites in seconds with one button. The firm explained the research was conducted to help prepare for AI-based threats of this nature before they proliferate. "If an AI technology exists that can create complete, automated threats, people will eventually use it. We have already seen the integration of generative AI elements in classic scams, such as AI-generated text or photographs to lure victims," explained Ben Gelman, senior data scientist at Sophos.

This Cyber News was published on www.infosecurity-magazine.com. Publication date: Thu, 30 Nov 2023 23:19:27 +0000


Cyber News related to Cybercriminals Hesitant About Using Generative AI

11 GenAI cybersecurity surveys you should read - Generative AI stands at the forefront of technological innovation, reshaping industries and unlocking new possibilities across various domains. As the integration of these technologies continues, a vigilant approach to ethical considerations and ...
9 months ago Helpnetsecurity.com
What Lurks in the Dark: Taking Aim at Shadow AI - Security teams are confronting a new nightmare this Halloween season: the rise of generative artificial intelligence. Generative AI tools have unleashed a new era of terror for chief information security officers, from powering deepfakes that are ...
10 months ago Darkreading.com
4 key devsecops skills for the generative AI era - Experts believe that generative AI capabilities, copilots, and large language models are ushering in a new era of how developers, data scientists, and engineers will work and innovate. They expect AI to improve productivity, quality, and innovation, ...
9 months ago Infoworld.com
Social engineering in the era of generative AI: Predictions for 2024 - Breakthroughs in large language models are driving an arms race between cybersecurity and social engineering scammers. For businesses, generative AI is both a curse and an opportunity. It's not just AI models themselves that cyber criminals are ...
5 months ago Securityintelligence.com
Abnormal Security Shares Examples of Attacks Using Generative AI - Abnormal Security has published examples of cyberattacks that illustrate how cybercriminals are beginning to leverage generative artificial intelligence to launch cyberattacks. In one example, a cybercriminal posed as a customer service ...
9 months ago Securityboulevard.com
How to Build a Data Foundation for Generative AI - Generative AI is not just a general-purpose productivity aid that surfaces information the way a search engine does; with gen AI, organizations can combine their unique, proprietary data with foundation models that have been pre-trained on a broad ...
8 months ago Feeds.dzone.com
Generative AI Redefines Cybersecurity Defense Against Advanced Threats - In the ever-shifting realm of cybersecurity, the dynamic dance between defenders and attackers has reached a new echelon with the integration of artificial intelligence, particularly generative AI. This technological advancement has not only armed ...
7 months ago Cysecurity.news
Navigating the New Frontier of AI-Driven Cybersecurity Threats - A few weeks ago, Best Buy revealed its plans to deploy generative AI to transform its customer service function. Best Buy's initiative is a harbinger of generative AI deployment in enterprise settings, aiming to increase productivity and improve ...
4 months ago Securityboulevard.com
Cybercriminals Hesitant About Using Generative AI - Cybercriminals are so far reluctant to use generative AI to launch attacks, according to new research by Sophos. Examining four prominent dark-web forums for discussions related to large language models, the firm found that threat actors showed ...
10 months ago Infosecurity-magazine.com
AI Is Changing the Way Enterprises Look at Trust: Deloitte & SAP Weigh In - Whether you are creating or customizing an AI policy or reassessing how your company approaches trust, keeping customers' confidence can be increasingly difficult with generative AI's unpredictability in the picture. We spoke to Deloitte's Michael ...
3 months ago Techrepublic.com
Generative AI Takes on SIEM - With more vendors adding support for generative AI to their platforms and products, life for security analysts seems to be getting deceptively easier. While adding generative AI capabilities to security information and event management is still in ...
10 months ago Darkreading.com
How Criminals Are Leveraging AI to Create Convincing Scams - Cybercriminals create far more sophisticated scams with generative AI than traditional phishing scams. According to Visa research, scammers are fooling even the savviest internet users by launching pig butchering, inheritance, humanitarian relief ...
4 months ago Tripwire.com
The old, not the new: Basic security issues still biggest threat to enterprises - Attacks on critical infrastructure reveal industry faux pas. Ransomware attacks on enterprises saw a nearly 12% drop last year, as larger organizations opt against paying and decrypting, in favor of rebuilding their infrastructure. X-Force analysis ...
7 months ago Helpnetsecurity.com
Securing Generative AI: Navigating Risks and Strategies - The introduction of generative AI has caused a paradigm change in the rapidly developing field of artificial intelligence, posing both unprecedented benefits and problems for companies. The need to strengthen security measures is becoming more and ...
9 months ago Cysecurity.news
Transforming the Creative Sphere With Generative AI - Generative AI, a trailblazing branch of artificial intelligence, is transforming the creative landscape and opening up new avenues for businesses worldwide. This article delves into how generative AI transforms creative work, including its benefits, ...
8 months ago Cysecurity.news
Generative AI's enterprise gamble: IT leaders bet big on tech despite security woes - Enterprise IT teams are moving swiftly to adopt generative artificial intelligence systems like ChatGPT, according to a new report from Glean and ISG. The report found that IT leaders see generative AI as transformational and are willing to increase ...
8 months ago Venturebeat.com
Generative AI vs. Predictive AI: A Cybersecurity Perspective - In the context of cybersecurity, AI promises considerable benefits however there's still a lot of confusion surrounding the topic, particularly around the terms generative AI and predictive AI. Given the high failure rate for AI projects let's ...
3 months ago Securityboulevard.com
3 Ways the CTO Can Fortify the Organization in the Age of Generative AI - An August survey by BlackBerry found that 75% of organizations worldwide were considering or implementing bans on ChatGPT and other generative AI applications in the workplace, with the vast majority of those citing the risk to data security and ...
7 months ago Securityboulevard.com
Fortinet Adds Generative AI Tool to Security Operations Portfolio - Fortinet today added a generative artificial intelligence tool to its portfolio to eliminate a range of manual tasks that security operations teams would otherwise need to perform. John Maddison, chief marketing officer for Fortinet, said Fortinet ...
10 months ago Securityboulevard.com
How enterprises are using gen AI to protect against ChatGPT leaks - ChatGPT is the new DNA of shadow IT, exposing organizations to new risks no one anticipated. Enterprise workers are gaining a 40% performance boost thanks to ChatGPT based on a recent Harvard University study. A second study from MIT discovered that ...
8 months ago Venturebeat.com
LinkedIn Tests Generative AI to Field Cybersecurity Questions From Employees and Suppliers - LinkedIn is testing how generative artificial intelligence could help employees and external suppliers get answers about cybersecurity policies within seconds, potentially cutting wait times for business deals or decisions to implement new tools. ...
10 months ago Wsj.com
Cohesity partners with NVIDIA to harness the power of generative AI - Cohesity announced a collaboration with NVIDIA to help organizations safely unlock the power of generative AI and data using the recently announced NVIDIA NIM microservices and by integrating NVIDIA AI Enterprise into the Cohesity Gaia platform. ...
6 months ago Helpnetsecurity.com
Responsibly Implementing AI, the Unstoppable Force - The Good News About AI People are using generative AI to see information in a more conversational way. Generative AI tools can listen and respond to voice input, a popular alternative to typing text into a search engine. In some forward-thinking ...
10 months ago Darkreading.com
Building a Sustainable Data Ecosystem - Finally, I outline future research and policy refinement directions, advocating for a collaborative and responsible approach to building a sustainable data ecosystem in generative AI. In recent years, generative AI has emerged as a transformative ...
6 months ago Feeds.dzone.com
DNSFilter introduces new capability to filter generative AI - DNSFilter announced the expansion of its protective DNS software with a new Generative AI category. DNSFilter's defense provides organizations of all sizes the ability to secure their network against harmful threats such as malware, botnet, and ...
9 months ago Helpnetsecurity.com

Latest Cyber News


Cyber Trends (last 7 days)


Trending Cyber News (last 7 days)