A Revealing Large-Scale Evaluation of Unsupervised Anomaly Detection Algorithms

• URL: http://arxiv.org/abs/2204.09825v1
• Date: Thu, 21 Apr 2022 00:17:12 GMT
• Title: A Revealing Large-Scale Evaluation of Unsupervised Anomaly Detection Algorithms
• Authors: Maxime Alvarez, Jean-Charles Verdier, D'Jeff K. Nkashama, Marc Frappier, Pierre-Martin Tardif, Froduald Kabanza
• Abstract summary: Anomaly detection has many applications ranging from bank-fraud detection and cyber-threat detection to equipment maintenance and health monitoring. We extensively reviewed twelve of the most popular unsupervised anomaly detection methods.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Anomaly detection has many applications ranging from bank-fraud detection and cyber-threat detection to equipment maintenance and health monitoring. However, choosing a suitable algorithm for a given application remains a challenging design decision, often informed by the literature on anomaly detection algorithms. We extensively reviewed twelve of the most popular unsupervised anomaly detection methods. We observed that, so far, they have been compared using inconsistent protocols - the choice of the class of interest or the positive class, the split of training and test data, and the choice of hyperparameters - leading to ambiguous evaluations. This observation led us to define a coherent evaluation protocol which we then used to produce an updated and more precise picture of the relative performance of the twelve methods on five widely used tabular datasets. While our evaluation cannot pinpoint a method that outperforms all the others on all datasets, it identifies those that stand out and revise misconceived knowledge about their relative performances.

Related papers

• Unifying Unsupervised Graph-Level Anomaly Detection and Out-of-Distribution Detection: A Benchmark [73.58840254552656]
  Unsupervised graph-level anomaly detection (GLAD) and unsupervised graph-level out-of-distribution (OOD) detection have received significant attention in recent years. We present a Unified Benchmark for unsupervised Graph-level OOD and anomaly Detection (our method) Our benchmark encompasses 35 datasets spanning four practical anomaly and OOD detection scenarios. We conduct multi-dimensional analyses to explore the effectiveness, generalizability, robustness, and efficiency of existing methods.
  arXiv  Detail & Related papers  (2024-06-21T04:07:43Z)
• A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [52.228708947607636]
  This paper introduces a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods. The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics. We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
  arXiv  Detail & Related papers  (2024-06-05T13:40:07Z)
• Benchmark Evaluation of Anomaly-Based Intrusion Detection Systems in the Context of Smart Grids [2.479074862022315]
  Anomaly detection has emerged as a key technology for cybersecurity in smart grids. We present an evaluation environment for anomaly detection methods in smart grids.
  arXiv  Detail & Related papers  (2023-12-21T10:17:36Z)
• On Pixel-level Performance Assessment in Anomaly Detection [87.7131059062292]
  Anomaly detection methods have demonstrated remarkable success across various applications. However, assessing their performance, particularly at the pixel-level, presents a complex challenge. In this paper, we dissect the intricacies of this challenge, underscored by visual evidence and statistical analysis.
  arXiv  Detail & Related papers  (2023-10-25T08:02:27Z)
• On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection [55.73320979733527]
  We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs. Experimental results show that our method achieves competitive detection performance on various text classification tasks.
  arXiv  Detail & Related papers  (2023-06-27T02:54:07Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• A Comparative Study on Unsupervised Anomaly Detection for Time Series: Experiments and Analysis [28.79393419730138]
  Time series anomaly detection is often essential to enable reliability and safety. Many recent studies target anomaly detection for time series data. We introduce for data, methods, and evaluation strategies. We systematically evaluate and compare state-of-the-art traditional as well as deep learning techniques.
  arXiv  Detail & Related papers  (2022-09-10T10:44:25Z)
• Towards Benchmarking and Evaluating Deepfake Detection [18.758101631430726]
  Deepfake detection automatically recognizes the manipulated medias through the analysis of the difference between manipulated and non-altered videos. It is difficult to conduct a sound benchmarking comparison of existing detection approaches because evaluation conditions are inconsistent across studies. Our objective is to establish a comprehensive and consistent benchmark, to develop a repeatable evaluation procedure, and to measure the performance of a range of detection approaches.
  arXiv  Detail & Related papers  (2022-03-04T03:12:15Z)
• Data-Uncertainty Guided Multi-Phase Learning for Semi-Supervised Object Detection [66.10057490293981]
  We propose a data-uncertainty guided multi-phase learning method for semi-supervised object detection. Our method behaves extraordinarily compared to baseline approaches and outperforms them by a large margin.
  arXiv  Detail & Related papers  (2021-03-29T09:27:23Z)
• Unsupervised Anomaly Detectors to Detect Intrusions in the Current Threat Landscape [0.11470070927586014]
  We show that Isolation Forests, One-Class Support Vector Machines and Self-Organizing Maps are more effective than their counterparts for intrusion detection. We detail how attacks with unstable, distributed or non-repeatable behavior as Fuzzing, Worms and Botnets are more difficult to detect.
  arXiv  Detail & Related papers  (2020-12-21T14:06:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.