On Pixel-level Performance Assessment in Anomaly Detection
- URL: http://arxiv.org/abs/2310.16435v1
- Date: Wed, 25 Oct 2023 08:02:27 GMT
- Title: On Pixel-level Performance Assessment in Anomaly Detection
- Authors: Mehdi Rafiei, Toby P. Breckon, Alexandros Iosifidis
- Abstract summary: Anomaly detection methods have demonstrated remarkable success across various applications.
However, assessing their performance, particularly at the pixel-level, presents a complex challenge.
In this paper, we dissect the intricacies of this challenge, underscored by visual evidence and statistical analysis.
- Score: 87.7131059062292
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Anomaly detection methods have demonstrated remarkable success across various
applications. However, assessing their performance, particularly at the
pixel-level, presents a complex challenge due to the severe imbalance that is
most commonly present between normal and abnormal samples. Commonly adopted
evaluation metrics designed for pixel-level detection may not effectively
capture the nuanced performance variations arising from this class imbalance.
In this paper, we dissect the intricacies of this challenge, underscored by
visual evidence and statistical analysis, leading to delve into the need for
evaluation metrics that account for the imbalance. We offer insights into more
accurate metrics, using eleven leading contemporary anomaly detection methods
on twenty-one anomaly detection problems. Overall, from this extensive
experimental evaluation, we can conclude that Precision-Recall-based metrics
can better capture relative method performance, making them more suitable for
the task.
Related papers
- Adaptive Deviation Learning for Visual Anomaly Detection with Data Contamination [20.4008901760593]
We introduce a systematic adaptive method that employs deviation learning to compute anomaly scores end-to-end.
Our proposed method surpasses competing techniques and exhibits both stability and robustness in the presence of data contamination.
arXiv Detail & Related papers (2024-11-14T16:10:15Z) - A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [52.228708947607636]
This paper introduces a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods.
The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics.
We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
arXiv Detail & Related papers (2024-06-05T13:40:07Z) - Few-Shot Anomaly Detection with Adversarial Loss for Robust Feature
Representations [8.915958745269442]
Anomaly detection is a critical and challenging task that aims to identify data points deviating from normal patterns and distributions within a dataset.
Various methods have been proposed using a one-class-one-model approach, but these techniques often face practical problems such as memory inefficiency and the requirement of sufficient data for training.
We propose a few-shot anomaly detection method that integrates adversarial training loss to obtain more robust and generalized feature representations.
arXiv Detail & Related papers (2023-12-04T09:45:02Z) - Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy.
We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples.
Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
arXiv Detail & Related papers (2023-02-22T14:50:24Z) - Unsupervised Anomaly Detection in Time-series: An Extensive Evaluation and Analysis of State-of-the-art Methods [10.618572317896515]
Unsupervised anomaly detection in time-series has been extensively investigated in the literature.
This paper proposes an in-depth evaluation study of recent unsupervised anomaly detection techniques in time-series.
arXiv Detail & Related papers (2022-12-06T15:05:54Z) - Abnormal-aware Multi-person Evaluation System with Improved Fuzzy
Weighting [0.0]
We choose the two-stage screening method, which consists of rough screening and score-weighted Kendall-$tau$ Distance.
We use Fuzzy Synthetic Evaluation Method(FSE) to determine the significance of scores given by reviewers as well as their reliability.
arXiv Detail & Related papers (2022-05-01T03:42:43Z) - A Revealing Large-Scale Evaluation of Unsupervised Anomaly Detection
Algorithms [0.0]
Anomaly detection has many applications ranging from bank-fraud detection and cyber-threat detection to equipment maintenance and health monitoring.
We extensively reviewed twelve of the most popular unsupervised anomaly detection methods.
arXiv Detail & Related papers (2022-04-21T00:17:12Z) - Information-Theoretic Bias Reduction via Causal View of Spurious
Correlation [71.9123886505321]
We propose an information-theoretic bias measurement technique through a causal interpretation of spurious correlation.
We present a novel debiasing framework against the algorithmic bias, which incorporates a bias regularization loss.
The proposed bias measurement and debiasing approaches are validated in diverse realistic scenarios.
arXiv Detail & Related papers (2022-01-10T01:19:31Z) - Data-Uncertainty Guided Multi-Phase Learning for Semi-Supervised Object
Detection [66.10057490293981]
We propose a data-uncertainty guided multi-phase learning method for semi-supervised object detection.
Our method behaves extraordinarily compared to baseline approaches and outperforms them by a large margin.
arXiv Detail & Related papers (2021-03-29T09:27:23Z) - Self-trained Deep Ordinal Regression for End-to-End Video Anomaly
Detection [114.9714355807607]
We show that applying self-trained deep ordinal regression to video anomaly detection overcomes two key limitations of existing methods.
We devise an end-to-end trainable video anomaly detection approach that enables joint representation learning and anomaly scoring without manually labeled normal/abnormal data.
arXiv Detail & Related papers (2020-03-15T08:44:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.