Carbon Filter: Real-time Alert Triage Using Large Scale Clustering and Fast Search

• URL: http://arxiv.org/abs/2405.04691v1
• Date: Tue, 7 May 2024 22:06:24 GMT
• Title: Carbon Filter: Real-time Alert Triage Using Large Scale Clustering and Fast Search
• Authors: Jonathan Oliver, Raghav Batta, Adam Bates, Muhammad Adil Inam, Shelly Mehta, Shugao Xia,
• Abstract summary: "Alert fatigue" is one of the biggest challenges faced by the Security Operations Center (SOC) today. We present Carbon Filter, a statistical learning based system that dramatically reduces the number of alerts analysts need to manually review.
• Score: 6.830322979559498
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: "Alert fatigue" is one of the biggest challenges faced by the Security Operations Center (SOC) today, with analysts spending more than half of their time reviewing false alerts. Endpoint detection products raise alerts by pattern matching on event telemetry against behavioral rules that describe potentially malicious behavior, but can suffer from high false positives that distract from actual attacks. While alert triage techniques based on data provenance may show promise, these techniques can take over a minute to inspect a single alert, while EDR customers may face tens of millions of alerts per day; the current reality is that these approaches aren't nearly scalable enough for production environments. We present Carbon Filter, a statistical learning based system that dramatically reduces the number of alerts analysts need to manually review. Our approach is based on the observation that false alert triggers can be efficiently identified and separated from suspicious behaviors by examining the process initiation context (e.g., the command line) that launched the responsible process. Through the use of fast-search algorithms for training and inference, our approach scales to millions of alerts per day. Through batching queries to the model, we observe a theoretical maximum throughput of 20 million alerts per hour. Based on the analysis of tens of million alerts from customer deployments, our solution resulted in a 6-fold improvement in the Signal-to-Noise ratio without compromising on alert triage performance.

Related papers

• Forecasting Attacker Actions using Alert-driven Attack Graphs [1.3812010983144802]
  This paper builds an action forecasting capability on top of the alert-driven AG framework for predicting the next likely attacker action. We also modify the framework to build AGs in real time, as new alerts are triggered. This way, we convert alert-driven AGs into an early warning system that enables analysts circumvent ongoing attacks and break the cyber killchain.
  arXiv  Detail & Related papers  (2024-08-19T11:04:47Z)
• FineWAVE: Fine-Grained Warning Verification of Bugs for Automated Static Analysis Tools [18.927121513404924]
  Automated Static Analysis Tools (ASATs) have evolved over time to assist in detecting bugs. Previous research efforts have explored learning-based methods to validate the reported warnings. We propose FineWAVE, a learning-based approach that verifies bug-sensitive warnings at a fine-grained granularity.
  arXiv  Detail & Related papers  (2024-03-24T06:21:35Z)
• A Hierarchical Security Events Correlation Model for Real-time Cyber Threat Detection and Response [0.0]
  We develop a novel hierarchical event correlation model that promises to reduce the number of alerts issued by an Intrusion Detection System. The proposed model takes the best of features from similarity and graph-based correlation techniques to deliver an ensemble capability not possible by either approach separately. The model is implemented as a proof of concept with experiments run on the DARPA 99 Intrusion detection set.
  arXiv  Detail & Related papers  (2023-12-02T20:07:40Z)
• Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information [67.78183175605761]
  Large Language Models are susceptible to adversarial prompt attacks. This vulnerability underscores a significant concern regarding the robustness and reliability of LLMs. We introduce a novel approach to detecting adversarial prompts at a token level.
  arXiv  Detail & Related papers  (2023-11-20T03:17:21Z)
• That Escalated Quickly: An ML Framework for Alert Prioritization [2.5845893156827158]
  We present That Escalated Quickly (TEQ), a machine learning framework that reduces alert fatigue with minimal changes to SOC. On real-world data, the system is able to reduce the time it takes to respond to actionable incidents by $22.9%$, suppress $54%$ of false positives with a $95.1%$ detection rate, and reduce the number of alerts an analyst needs to investigate within singular incidents by $14%$.
  arXiv  Detail & Related papers  (2023-02-13T19:20:52Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• Towards Adversarial Patch Analysis and Certified Defense against Crowd Counting [61.99564267735242]
  Crowd counting has drawn much attention due to its importance in safety-critical surveillance systems. Recent studies have demonstrated that deep neural network (DNN) methods are vulnerable to adversarial attacks. We propose a robust attack strategy called Adversarial Patch Attack with Momentum to evaluate the robustness of crowd counting models.
  arXiv  Detail & Related papers  (2021-04-22T05:10:55Z)
• Detecting Invisible People [58.49425715635312]
  We re-purpose tracking benchmarks and propose new metrics for the task of detecting invisible objects. We demonstrate that current detection and tracking systems perform dramatically worse on this task. Second, we build dynamic models that explicitly reason in 3D, making use of observations produced by state-of-the-art monocular depth estimation networks.
  arXiv  Detail & Related papers  (2020-12-15T16:54:45Z)
• Robust Spammer Detection by Nash Reinforcement Learning [64.80986064630025]
  We develop a minimax game where the spammers and spam detectors compete with each other on their practical goals. We show that an optimization algorithm can reliably find an equilibrial detector that can robustly prevent spammers with any mixed spamming strategies from attaining their practical goal.
  arXiv  Detail & Related papers  (2020-06-10T21:18:07Z)
• Moving Metric Detection and Alerting System at eBay [4.778341933013294]
  At eBay, there are thousands of product health metrics for different domain teams to monitor. We built a two-phase alerting system to notify users with actionable alerts based on anomaly detection and alert retrieval.
  arXiv  Detail & Related papers  (2020-04-06T00:28:39Z)
• Detection in Crowded Scenes: One Proposal, Multiple Predictions [79.28850977968833]
  We propose a proposal-based object detector, aiming at detecting highly-overlapped instances in crowded scenes. The key of our approach is to let each proposal predict a set of correlated instances rather than a single one in previous proposal-based frameworks. Our detector can obtain 4.9% AP gains on challenging CrowdHuman dataset and 1.0% $textMR-2$ improvements on CityPersons dataset.
  arXiv  Detail & Related papers  (2020-03-20T09:48:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.