The Challenges of Building Generative AI Applications in Cybersecurity

Armorblox was acquired by Cisco to further their AI-first Security Cloud by bringing generative AI experiences to Cisco's security solutions.
Quickly a new mission came my way: Build generative AI Assistants that will allow cybersecurity administrators to find the answers they seek quickly, and therefore make their lives easier.
The AI Assistant can help with troubleshooting, such as locating policies, giving summarization of existing configurations, providing documentation, and more.
The first and most obvious challenge has been evaluation of the model.
There are several ways a model's responses can be evaluated.
An innovative method that was proposed early on by the community was using LLMs to evaluate LLMs. This works wonders for generalized use cases, but can fall short when assessing models tailored for niche tasks.
In order for niche use cases to perform well, they require access to unique or proprietary data that is inaccessible to standard models like GPT-4.
As we expand the size of real user data that can be used for validation, the importance of automated metrics will grow.
With real user questions, we can more appropriately benchmark against real use cases and automated metrics become a stronger signal for good models.
The first set of use cases for our AI assistant are aimed at allowing a user to become more efficient by either compiling and presenting data coherently or making information more accessible.
Once the AI assistant gives summarizes their rule configuration, they want to know how to alter it.
The AI assistant will give them guided steps to configured the policy as desired.
This has already given me insight into some hallucinations and poor assumptions that the AI assistant is making.
Engaging domain experts as a proxy for real customers at pre-launch to test the AI assistant has proven invaluable.
Instituting a regular team ritual to review and act on this feedback ensures continued alignment with expectations for the model responses.
Prioritizing the feedback we get is extremely important, focusing on the impact of the user experience and the loss of trust in the AI assistant are the core criteria for prioritization along with the frequency of the issue.
The pathways for addressing evaluation gaps are varied - be it through prompt engineering, different models, or trying various augmented model strategies like knowledge graphs.
As the solution evolves into a tangible, demoable product, latency, the amount of time it takes for a response to be returned to a user, becomes increasingly important.
It's been an exciting start to the journey of building products with LLMs and I can't wait to learn more as we continue building and shipping awesome AI products.
Recently, Open AI released their Assistants API, which will enable developers to more easily access the potential of LLMs to operate as agents with multiple tools and larger contexts.


This Cyber News was published on feedpress.me. Publication date: Tue, 12 Dec 2023 13:13:05 +0000


Cyber News related to The Challenges of Building Generative AI Applications in Cybersecurity

11 GenAI cybersecurity surveys you should read - Generative AI stands at the forefront of technological innovation, reshaping industries and unlocking new possibilities across various domains. As the integration of these technologies continues, a vigilant approach to ethical considerations and ...
10 months ago Helpnetsecurity.com
Fortinet Contributes to World Economic Forum's Strategic Cybersecurity Talent Framework - Shining a light on the cybersecurity workforce challenge, the World Economic Forum recently published its Strategic Cybersecurity Talent Framework, which is intended to serve as a reference for public and private decision-makers concerned by the ...
5 months ago Feeds.fortinet.com
Digital Learning Tools for Cybersecurity Education - In the field of cybersecurity education, digital learning tools have become indispensable. This article explores various digital learning tools tailored specifically to cybersecurity education. These digital learning tools play a crucial role in ...
10 months ago Securityzap.com
4 key devsecops skills for the generative AI era - Experts believe that generative AI capabilities, copilots, and large language models are ushering in a new era of how developers, data scientists, and engineers will work and innovate. They expect AI to improve productivity, quality, and innovation, ...
10 months ago Infoworld.com
Social engineering in the era of generative AI: Predictions for 2024 - Breakthroughs in large language models are driving an arms race between cybersecurity and social engineering scammers. For businesses, generative AI is both a curse and an opportunity. It's not just AI models themselves that cyber criminals are ...
5 months ago Securityintelligence.com
Growing threats outpace cybersecurity workforce - The cybersecurity skills shortage threatens the well-being and even survival of numerous businesses as cybersecurity threats grow more numerous, sophisticated, and dangerous to the point that cybersecurity groups have vowed not to pay ransom demands. ...
9 months ago Legal.thomsonreuters.com
Student Cybersecurity Clubs: Fostering Online Safety - Student cybersecurity clubs are playing a crucial role in promoting online safety among students. Student cybersecurity clubs play a vital role in this regard, as they provide a platform for students to learn about the latest threats, share best ...
10 months ago Securityzap.com
OT Cybersecurity: Safeguarding Building Operations in a Digitized World - These concerns are brought into the spotlight by high-profile ransomware attacks, which can now penetrate even the most intricate systems. Recently, in September, the gravity of this threat was illustrated when two Las Vegas casinos fell victim to an ...
9 months ago Cyberdefensemagazine.com
How to become a cybersecurity architect - Cybersecurity architects implement and maintain a comprehensive cybersecurity framework to protect their company's digital assets. The cybersecurity architect position is a fundamental role that all organizations need, said Lester Nichols, director ...
4 months ago Techtarget.com
What Lurks in the Dark: Taking Aim at Shadow AI - Security teams are confronting a new nightmare this Halloween season: the rise of generative artificial intelligence. Generative AI tools have unleashed a new era of terror for chief information security officers, from powering deepfakes that are ...
11 months ago Darkreading.com
Building a Sustainable Data Ecosystem - Finally, I outline future research and policy refinement directions, advocating for a collaborative and responsible approach to building a sustainable data ecosystem in generative AI. In recent years, generative AI has emerged as a transformative ...
7 months ago Feeds.dzone.com
Simbian Unveils Generative AI Platform to Automate Cybersecurity Tasks - Simbian today launched a cybersecurity platform that leverages generative artificial intelligence to automate tasks that can increase in complexity as the tool learns more about the IT environment. Fresh off raising $10 million in seed funding, ...
6 months ago Securityboulevard.com
Navigating the New Frontier of AI-Driven Cybersecurity Threats - A few weeks ago, Best Buy revealed its plans to deploy generative AI to transform its customer service function. Best Buy's initiative is a harbinger of generative AI deployment in enterprise settings, aiming to increase productivity and improve ...
5 months ago Securityboulevard.com
Generative AI vs. Predictive AI: A Cybersecurity Perspective - In the context of cybersecurity, AI promises considerable benefits however there's still a lot of confusion surrounding the topic, particularly around the terms generative AI and predictive AI. Given the high failure rate for AI projects let's ...
4 months ago Securityboulevard.com
How to Build a Data Foundation for Generative AI - Generative AI is not just a general-purpose productivity aid that surfaces information the way a search engine does; with gen AI, organizations can combine their unique, proprietary data with foundation models that have been pre-trained on a broad ...
9 months ago Feeds.dzone.com
Gamification in Cybersecurity Education - Gamification has become increasingly prevalent in numerous domains, including cybersecurity education. Gamification presents a promising approach to meet this challenge, making cybersecurity education both effective and enjoyable. One way to ...
10 months ago Securityzap.com
2024 Cybersecurity Predictions - Entering a new year, the cybersecurity landscape is poised for significant shifts, driven by the dynamic interplay between technological advancements and persistent threats. In this blog, industry experts share their insights and predictions, ...
9 months ago Securityboulevard.com
Generative AI Redefines Cybersecurity Defense Against Advanced Threats - In the ever-shifting realm of cybersecurity, the dynamic dance between defenders and attackers has reached a new echelon with the integration of artificial intelligence, particularly generative AI. This technological advancement has not only armed ...
8 months ago Cysecurity.news
What the cybersecurity workforce can expect in 2024 - For cybersecurity professionals, 2023 was a mixed bag of opportunities and concerns. The good news is that the number of people in cybersecurity jobs has reached its highest number ever: 5.5 million, according to the 2023 ISC2 Global Workforce Study. ...
10 months ago Securityintelligence.com
Cybersecurity Training for Business Leaders - This article explores the significance of cybersecurity training for business leaders and its crucial role in establishing a secure and resilient business environment. By examining the key components of effective training programs and the ...
9 months ago Securityzap.com
Business Cybersecurity Culture: Building a Secure Workforce - To protect their operations and sensitive data, organizations must prioritize the development of a strong cybersecurity culture within their workforce. In this discussion, we will explore the crucial elements of a robust cybersecurity culture and ...
9 months ago Securityzap.com
Cybersecurity Curriculum Development Tips for Schools - With the constant threat of cyber attacks, schools must prioritize the development of a robust cybersecurity curriculum to equip students with the necessary skills and knowledge. This article provides valuable insights and tips for schools aiming to ...
10 months ago Securityzap.com
The Importance of Cybersecurity Education in Schools - Cybersecurity education equips students with the knowledge and skills needed to protect themselves and others from cyber threats. Cybersecurity education can teach students about the impact of cyberbullying, how to prevent it, and how to respond ...
10 months ago Securityzap.com
Cybersecurity Workshops for Students - Cybersecurity workshops for students serve as an effective means to educate and empower the younger generation in protecting their digital assets. With proper planning and organization, cybersecurity workshops enable students to navigate the digital ...
10 months ago Securityzap.com
How to Avoid Falling Below the Cybersecurity Poverty Line - The security poverty line broadly defines a divide between the organizations that have the means and resources to achieve and maintain mature security postures to protect data, and those that do not. It was first coined by cybersecurity expert Wendy ...
1 year ago Csoonline.com

Latest Cyber News


Cyber Trends (last 7 days)


Trending Cyber News (last 7 days)