How machine learning helps us hunt threats | Securelist

In this post, we will share our experience hunting for new threats by processing Kaspersky Security Network (KSN) global threat data with ML tools to identify subtle new Indicators of Compromise (IoCs). The model can process and learn from millions of data points in real time, pointing out subtle indicators that may signal the presence of a new or advanced threat. To maintain model maturity, incremental learning is often needed, which means an ongoing process of updating and refining the machine learning model by incorporating new data over time. By allowing models to learn from decentralized data without sharing the actual data, federated learning could facilitate the creation of more robust and generalizable threat detection models. The Kaspersky Security Network (KSN) infrastructure is designed to receive and process complex global cyberthreat data, transforming it into actionable threat intelligence that powers our products. Random Forest is especially effective in handling non-linear data, reducing the risk of overfitting, and providing insights into the importance of various features in the dataset. Preprocessing is a crucial step in a machine learning pipeline where raw data is transformed into a format suitable for training an ML model. These outcomes provide valuable feedback that helps data scientists and ML engineers make informed decisions to improve the model’s performance, guide adjustments, and ensure the final model is robust, generalizes well, and meets the desired criteria. When dealing with text data, a common approach is to first transform the raw text into numerical features using techniques like TF–IDF and then apply an ML algorithm such as Random Forest to classify or analyze the data. Additionally, federated learning presents a significant opportunity for collaborative threat detection across organizations while preserving data privacy. This approach is particularly important in dynamic fields where data distributions and patterns can shift, leading to the need for models that can keep up with these changes, which is exactly the case with the cybersecurity threat landscape. Machine learning enables systems to learn from data and improve their performance over time without being explicitly programmed. A machine learning dataset is a collection of data used to train, validate, and test ML models. In this post, we have evaluated the utilization of ML models on our KSN global threat data, which has led us to reveal thousands of new advanced threats. The ability of ML-powered technology to analyze vast amounts of data in real time ensures that potential threats are identified and addressed more quickly, minimizing the window of vulnerability. Through the analysis of vast and complex logs, ML models can identify subtle patterns and IoCs, providing organizations with a powerful tool to enhance their security posture. Random Forest models can become demanding when dealing with high-dimensional data and large datasets. We already use deep learning in some of our products, and applying it to threat hunting could potentially further improve detection accuracy and uncover even more sophisticated threats. TF–IDF transforms raw text data into a set of machine-readable numerical features, which can be then fed to an ML model. The choice of dataset, its quality, and how it is prepared and split into training, validation, and test sets can significantly impact the model’s ability to learn and generalize new data. The combination of TF–IDF with Random Forest allows handling high-dimensional data, while also providing robustness and scalability, very much needed to handle data with millions of entries daily. The journey of refining ML models through meticulous dataset preparation, preprocessing, and model implementation has highlighted the importance of leveraging these technologies to build robust, adaptable, and scalable solutions. By analyzing this data, organizations can detect anomalies, pinpoint malicious activity, and mitigate potential cyberattacks before they cause significant damage. A machine learning model reaches maturity when it performs consistently well on the kind of tasks it was designed for, meeting the performance criteria set during its development. The outcomes of machine learning during model training play a crucial role in guiding the development, refinement, and optimization of the model. As we started this study, we kept in mind that the usage of ML in log analysis enables the discovery of previously unknown cyberthreats by analyzing vast amounts of data and uncovering patterns. One promising area is the integration of deep learning techniques, which can automatically extract and learn complex patterns from raw data. ML, a subset of artificial intelligence (AI), with its ability to process and analyze large datasets, offers a powerful solution to enhance threat detection capabilities. As we continue to explore and enhance these capabilities, the potential for machine learning to reshape cybersecurity and protect against increasingly sophisticated threats becomes ever more apparent. TF–IDF is known to be efficient and versatile, while Random Forest is known for accuracy, reduced overfitting, and an ability to capture complex, non-linear relationships between features. The model is then trained and tested, before being deployed to examine larger amounts of data. Continuous learning allows ML models to detect subtle and novel cyberthreats, providing more robust defense. When a model is ready, it can be integrated into a production environment where it can start making predictions on new data. We utilize a variety of ML models and methods that are key to automating threat detection, anomaly recognition, and enhancing the accuracy of malware identification. ML then acts to “reconstruct the cyber-reality” by transforming raw telemetry data into actionable insights that reflect the true state of a network or system’s security. This process involves cleaning the data, handling missing values, transforming variables into a scaled and normalized numerical representation, and ensuring that the data is in a consistent and standardized format. In the ever-evolving landscape of cybersecurity, logs, that is information collected from various sources like network devices, endpoints, and applications, plays a crucial role in identifying and responding to threats. These technologies will not only improve detection accuracy but also enable more proactive and collaborative defense strategies, allowing organizations to stay ahead of the ever-evolving cyberthreat landscape. Ultimately, the ability of ML to partially reconstruct the cyber-reality from logs helps organizations stay ahead of cyberthreats by offering a clearer, more precise view of their security posture, enabling faster and better informed decision-making. Another area of exploration is reinforcement learning, where models can continuously adapt and improve by interacting with dynamic cybersecurity environments. Random Forest is highly effective at identifying patterns, but this strength can lead to challenges in interpretability, particularly with larger models. We will also discuss challenges in implementing machine learning and interpreting threat hunting results. A dataset consist of various examples, each containing features (input variables) and, in supervised learning tasks, corresponding labels (output variables or targets).

This Cyber News was published on securelist.com. Publication date: Wed, 02 Oct 2024 10:43:07 +0000


Cyber News related to How machine learning helps us hunt threats | Securelist

The Role of Machine Learning in Cybersecurity - Machine learning plays a crucial role in cybersecurity by enhancing defense mechanisms and protecting sensitive information. The key advantage of using machine learning in cybersecurity is its ability to constantly adapt and learn from new threats. ...
9 months ago Securityzap.com
How machine learning helps us hunt threats | Securelist - In this post, we will share our experience hunting for new threats by processing Kaspersky Security Network (KSN) global threat data with ML tools to identify subtle new Indicators of Compromise (IoCs). The model can process and learn from millions ...
2 months ago Securelist.com
The Role of AI in Personalized Learning - Artificial Intelligence is playing an increasingly significant role in the field of education, particularly in personalized learning. In this article, we will explore the role of AI in personalized learning, with a focus on AI-driven adaptive ...
10 months ago Securityzap.com
The Role of IoT in Modern Education - From smart classrooms equipped with IoT devices to personalized learning platforms, IoT has paved the way for a more immersive and tailored educational experience. Overall, the integration of IoT in education holds great promise in transforming the ...
11 months ago Securityzap.com
Online Learning Security Best Practices - The rapid increase in remote learning has raised security concerns surrounding online learning platforms. The security of online learning platforms involves implementing robust measures to protect against unauthorized access and data breaches. By ...
11 months ago Securityzap.com
Cybersecurity Challenges in Remote Learning - The increasing prevalence of remote learning in the education sector has brought about new cybersecurity challenges that must be addressed. This article aims to delve into the various cyber threats faced in remote learning and provide practical ...
11 months ago Securityzap.com
Digital Learning Tools for Cybersecurity Education - In the field of cybersecurity education, digital learning tools have become indispensable. This article explores various digital learning tools tailored specifically to cybersecurity education. These digital learning tools play a crucial role in ...
11 months ago Securityzap.com
For the Love of Learning: We're Here for You at Cisco Live 2024 Las Vegas! - Cisco Live is all about learning, as are Cisco Learning & Certifications and Cisco U. We're here to provide the opportunities you need to learn everything you can and apply your newfound knowledge as soon as possible in the tech career you want. ...
6 months ago Feedpress.me
JFrog, AWS team up for machine learning in the cloud - Software supply chain provider JFrog is integrating with the Amazon SageMaker cloud-based machine learning platform to incorporate machine learning models into the software development lifecycle. The JFrog platform integration with Amazon SageMaker, ...
10 months ago Infoworld.com
Cybersecurity Trends: Shaping the Future Landscape - Embark on a journey through the ever-evolving landscape of cybersecurity, where hidden threats and silent breaches shape the digital realm. AI is transforming the cybersecurity landscape by enhancing threat detection and mitigation, ushering in a ...
8 months ago Securityzap.com
9 Best DDoS Protection Service Providers for 2024 - eSecurity Planet content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More. One of the most powerful defenses an organization can employ against distributed ...
11 months ago Esecurityplanet.com
Cracking the Code: The Role of AI and UBA in Mitigating Insider Threats to Businesses - Automating mundane tasks and driving data-driven decisions, big data enables businesses to make better decisions and drive transformation. The use of AI has been shown as an effective way of streamlining operations and enhancing security measures, ...
11 months ago Cysecurity.news
Paraguay warns of Black Hunt ransomware attacks after Tigo Business breach - The Paraguay military is warning of Black Hunt ransomware attacks after Tigo Business suffered a cyberattack last week impacting cloud and hosting services in the company's business division. Tigo is the largest mobile carrier in Paraguay, with its ...
10 months ago Bleepingcomputer.com
The Evolution of Cyber Threats: Past, Present, and Future - Cyber threats have evolved significantly over time, posing increasing risks to individuals, organizations, and governments in our interconnected world. Let's explore the past, present, and future of cyber threats to better understand how to protect ...
10 months ago Securityzap.com
Exploring Technology in Classroom Learning - This article aims to explore the effective utilization of technology to enhance classroom learning experiences. Technology plays a crucial role in facilitating effective and engaging learning experiences in the classroom. With the advancement of ...
11 months ago Securityzap.com
5 Tips for Pi Day Savings at the Cisco Learning Network Store - Save 25% on select training products from the Cisco Learning Network Store for 24 hours only. Two new multicloud training courses are now available in the Cisco Learning Network Store-and they're included in the Pi Day Sale. If you are an active ...
8 months ago Feedpress.me
Advanced ransomware campaigns expose need for AI-powered cyber defense - In this Help Net Security interview, Carl Froggett, CIO at Deep Instinct, discusses emerging trends in ransomware attacks, emphasizing the need for businesses to use advanced AI technologies, such as deep learning, for prevention rather than just ...
1 year ago Helpnetsecurity.com
Privacy-Preserving AI: Protocols to Practice - At the same time, it increases the possibility of personal information misuse, reaching unprecedented levels of power and speed in analyzing and spreading individuals' data. Machine learning employs algorithms to analyze data, improve performance, ...
9 months ago Feeds.dzone.com
AI trends: A closer look at machine learning's role - The hottest technology right now is AI - more specifically, generative AI. The trend is so popular that every conference and webinar speaker feels obligated to mention some form of AI, no matter their field. The heavy focus on this technology ...
10 months ago Securityintelligence.com
How to Identify & Monitor Insider Threat Indicators [A Guide] - Most security protocols look outward when looking for cybersecurity threats. Our recent study found that 42% of exposed credentials came from an insider threat-former employees whose credentials were still active, employee error, or a malicious ...
8 months ago Securityboulevard.com
Navigating the Labyrinth of Digital Cyberthreats Using AI-Powered Cryptographic Agility - Welcome to the new world of cybersecurity - an ever-changing, intricate maze where new threats lurk around every corner. On the other hand, they can be used to create brand-new threats, introducing vulnerabilities previously unimagined, leaving our ...
8 months ago Cybersecurity-insiders.com
Navigating the Labyrinth of Digital Cyberthreats Using AI-Powered Cryptographic Agility - Welcome to the new world of cybersecurity - an ever-changing, intricate maze where new threats lurk around every corner. On the other hand, they can be used to create brand-new threats, introducing vulnerabilities previously unimagined, leaving our ...
8 months ago Cybersecurity-insiders.com
Navigating the Labyrinth of Digital Cyberthreats Using AI-Powered Cryptographic Agility - Welcome to the new world of cybersecurity - an ever-changing, intricate maze where new threats lurk around every corner. On the other hand, they can be used to create brand-new threats, introducing vulnerabilities previously unimagined, leaving our ...
8 months ago Cybersecurity-insiders.com
Navigating the Labyrinth of Digital Cyberthreats Using AI-Powered Cryptographic Agility - Welcome to the new world of cybersecurity - an ever-changing, intricate maze where new threats lurk around every corner. On the other hand, they can be used to create brand-new threats, introducing vulnerabilities previously unimagined, leaving our ...
8 months ago Cybersecurity-insiders.com
Navigating the Labyrinth of Digital Cyberthreats Using AI-Powered Cryptographic Agility - Welcome to the new world of cybersecurity - an ever-changing, intricate maze where new threats lurk around every corner. On the other hand, they can be used to create brand-new threats, introducing vulnerabilities previously unimagined, leaving our ...
8 months ago Cybersecurity-insiders.com

Latest Cyber News


Cyber Trends (last 7 days)


Trending Cyber News (last 7 days)