Detection Latencies of Anomaly Detectors: An Overlooked Perspective ?

• URL: http://arxiv.org/abs/2402.09082v1
• Date: Wed, 14 Feb 2024 10:52:39 GMT
• Title: Detection Latencies of Anomaly Detectors: An Overlooked Perspective ?
• Authors: Tommaso Puccetti, Andrea Ceccarelli
• Abstract summary: In this paper, we argue the relevance of measuring the temporal latency of attacks and errors. We propose an evaluation approach for detectors to ensure a pragmatic trade-off between correct and in-time detection.
• Score: 1.8492669447784602
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ever-evolving landscape of attacks, coupled with the growing complexity of ICT systems, makes crafting anomaly-based intrusion detectors (ID) and error detectors (ED) a difficult task: they must accurately detect attacks, and they should promptly perform detections. Although improving and comparing the detection capability is the focus of most research works, the timeliness of the detection is less considered and often insufficiently evaluated or discussed. In this paper, we argue the relevance of measuring the temporal latency of attacks and errors, and we propose an evaluation approach for detectors to ensure a pragmatic trade-off between correct and in-time detection. Briefly, the approach relates the false positive rate with the temporal latency of attacks and errors, and this ultimately leads to guidelines for configuring a detector. We apply our approach by evaluating different ED and ID solutions in two industrial cases: i) an embedded railway on-board system that optimizes public mobility, and ii) an edge device for the Industrial Internet of Things. Our results show that considering latency in addition to traditional metrics like the false positive rate, precision, and coverage gives an additional fundamental perspective on the actual performance of the detector and should be considered when assessing and configuring anomaly detectors.

Related papers

• A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [52.228708947607636]
  This paper introduces a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods. The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics. We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
  arXiv  Detail & Related papers  (2024-06-05T13:40:07Z)
• Incorporating Gradients to Rules: Towards Lightweight, Adaptive Provenance-based Intrusion Detection [11.14938737864796]
  We propose CAPTAIN, a rule-based PIDS capable of automatically adapting to diverse environments. We build a differentiable tag propagation framework and utilize the gradient descent algorithm to optimize these adaptive parameters. The evaluation results demonstrate that CAPTAIN offers better detection accuracy, less detection latency, lower runtime overhead, and more interpretable detection alarms and knowledge.
  arXiv  Detail & Related papers  (2024-04-23T03:50:57Z)
• LTRDetector: Exploring Long-Term Relationship for Advanced Persistent Threats Detection [20.360010908574303]
  Advanced Persistent Threat (APT) is challenging to detect due to prolonged duration, infrequent occurrence, and adept concealment techniques. Existing approaches primarily concentrate on the observable traits of attack behaviors, neglecting the intricate relationships formed throughout the persistent attack lifecycle. We present an innovative APT detection framework named LTRDetector, implementing an end-to-end holistic operation.
  arXiv  Detail & Related papers  (2024-04-04T02:30:51Z)
• On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection [55.73320979733527]
  We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs. Experimental results show that our method achieves competitive detection performance on various text classification tasks.
  arXiv  Detail & Related papers  (2023-06-27T02:54:07Z)
• TBDetector:Transformer-Based Detector for Advanced Persistent Threats with Provenance Graph [17.518551273453888]
  We propose TBDetector, a transformer-based advanced persistent threat detection method for APT attack detection. Provenance graphs provide rich historical information and have the powerful attacks historic correlation ability. To evaluate the effectiveness of the proposed method, we have conducted experiments on five public datasets.
  arXiv  Detail & Related papers  (2023-04-06T03:08:09Z)
• SoftED: Metrics for Soft Evaluation of Time Series Event Detection [4.263111781491367]
  Time series event detection methods are evaluated mainly by standard classification metrics that focus solely on detection accuracy. Inaccuracy in detecting an event can often result from its preceding or delayed effects reflected in neighboring detections. This paper introduces SoftED metrics, a new set of metrics designed for soft evaluating event detection methods.
  arXiv  Detail & Related papers  (2023-04-02T03:27:31Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• A Robust and Explainable Data-Driven Anomaly Detection Approach For Power Electronics [56.86150790999639]
  We present two anomaly detection and classification approaches, namely the Matrix Profile algorithm and anomaly transformer. The Matrix Profile algorithm is shown to be well suited as a generalizable approach for detecting real-time anomalies in streaming time-series data. A series of custom filters is created and added to the detector to tune its sensitivity, recall, and detection accuracy.
  arXiv  Detail & Related papers  (2022-09-23T06:09:35Z)
• Bandit Quickest Changepoint Detection [55.855465482260165]
  Continuous monitoring of every sensor can be expensive due to resource constraints. We derive an information-theoretic lower bound on the detection delay for a general class of finitely parameterized probability distributions. We propose a computationally efficient online sensing scheme, which seamlessly balances the need for exploration of different sensing options with exploitation of querying informative actions.
  arXiv  Detail & Related papers  (2021-07-22T07:25:35Z)
• No Need to Know Physics: Resilience of Process-based Model-free Anomaly Detection for Industrial Control Systems [95.54151664013011]
  We present a novel framework to generate adversarial spoofing signals that violate physical properties of the system. We analyze four anomaly detectors published at top security conferences.
  arXiv  Detail & Related papers  (2020-12-07T11:02:44Z)
• ReRe: A Lightweight Real-time Ready-to-Go Anomaly Detection Approach for Time Series [0.27528170226206433]
  This paper introduces ReRe, a Real-time Ready-to-go proactive Anomaly Detection algorithm for streaming time series. ReRe employs two lightweight Long Short-Term Memory (LSTM) models to predict and jointly determine whether or not an upcoming data point is anomalous. Experiments based on real-world time-series datasets demonstrate the good performance of ReRe in real-time anomaly detection.
  arXiv  Detail & Related papers  (2020-04-05T21:26:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.