Testing the Test: Score-Direction Instability in Class-Split Anomaly Detection

• URL: http://arxiv.org/abs/2606.02601v1
• Date: Sat, 23 May 2026 06:27:42 GMT
• Title: Testing the Test: Score-Direction Instability in Class-Split Anomaly Detection
• Abstract summary: Within-dataset class-split evaluation is widely used as a proxy for fully unconditional out-of-distribution anomaly detection.<n>We show that this protocol can become ill-posed when the held-out anomaly class overlaps the normal mixture in representation space.<n>We introduce a simple training-free diagnostic, neighborhood class leakage, and show that it predicts score-direction instability across Fashion-MNIST, CIFAR-10, and Imagenette.
• Score: 56.362776482614976
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Within-dataset class-split evaluation is widely used as a proxy for fully unconditional out-of-distribution anomaly detection. We show that this protocol can become ill-posed when the held-out anomaly class overlaps the normal mixture in representation space. In this regime, anomaly scores may collapse toward chance or even invert, and the preferred score direction can depend on the unknown anomaly class. We introduce a simple training-free diagnostic, neighborhood class leakage, and show that it predicts score-direction instability across Fashion-MNIST, CIFAR-10, and Imagenette, in both pixel and VAE latent spaces. Our results suggest that class-split AD benchmarks should be treated as geometry-dependent stress tests rather than unconditional evidence of anomaly-detection ability.

Related papers

• Unsupervised Anomaly Detection Using Flow Matching on Tabular Data [18.400208887365732]
  We study flow-matching-based anomaly detection under contaminated training data by comparing Time-Conditioned Contraction Matching (TCCM) with Forest-Flow.<n>Our results show that the choice of anomaly score is critical.<n>These findings highlight the importance of anomaly scoring for flow-matching methods in financial anomaly detection under severe class imbalance.
  arXiv  Detail & Related papers  (2026-08-20T08:56:46Z)
• Universal Transformation of One-Class Classifiers for Unsupervised Anomaly Detection [51.73001988341294]
  Anomaly detection is typically formulated as a one-class classification problem.<n>We present a dataset folding method that transforms an arbitrary one-class classifier-based anomaly detector into a fully unsupervised method.
  arXiv  Detail & Related papers  (2026-02-13T16:54:12Z)
• Multi-Cue Anomaly Detection and Localization under Data Contamination [0.6703429330486276]
  We propose a robust anomaly detection framework that integrates limited anomaly supervision into the adaptive deviation learning paradigm.<n>Our framework achieves strong detection and localization performance, interpretability, and robustness under various levels of data contamination.
  arXiv  Detail & Related papers  (2026-01-30T12:34:13Z)
• FUN-AD: Fully Unsupervised Learning for Anomaly Detection with Noisy Training Data [1.0650780147044159]
  We propose a novel learning-based approach for fully unsupervised anomaly detection with unlabeled and potentially contaminated training data. Our method is motivated by two observations, that i) the pairwise feature distances between the normal samples are on average likely to be smaller than those between the anomaly samples or heterogeneous samples and ii) pairs of features mutually closest to each other are likely to be homogeneous pairs. Building on the first observation that nearest-neighbor distances can distinguish between confident normal samples and anomalies, we propose a pseudo-labeling strategy using an iteratively reconstructed memory bank.
  arXiv  Detail & Related papers  (2024-11-25T05:51:38Z)
• Revisiting Non-separable Binary Classification and its Applications in Anomaly Detection [10.031370250511207]
  We show that linear classification of XOR is possible. We propose equality separation, that adapts the SVM objective to distinguish data within or outside the margin. Our classifier can then be integrated into neural network pipelines with a smooth approximation.
  arXiv  Detail & Related papers  (2023-12-03T23:59:03Z)
• MSFlow: Multi-Scale Flow-based Framework for Unsupervised Anomaly Detection [124.52227588930543]
  Unsupervised anomaly detection (UAD) attracts a lot of research interest and drives widespread applications. An inconspicuous yet powerful statistics model, the normalizing flows, is appropriate for anomaly detection and localization in an unsupervised fashion. We propose a novel Multi-Scale Flow-based framework dubbed MSFlow composed of asymmetrical parallel flows followed by a fusion flow. Our MSFlow achieves a new state-of-the-art with a detection AUORC score of up to 99.7%, localization AUCROC score of 98.8%, and PRO score of 97.1%.
  arXiv  Detail & Related papers  (2023-08-29T13:38:35Z)
• That's BAD: Blind Anomaly Detection by Implicit Local Feature Clustering [28.296651124677556]
  Setting blind anomaly detection (BAD) can be converted into a local outlier detection problem. We propose a novel method named PatchCluster that can accurately detect image- and pixel-level anomalies. Experimental results show that PatchCluster shows a promising performance without the knowledge of normal data.
  arXiv  Detail & Related papers  (2023-07-06T18:17:43Z)
• Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection [81.07346419422605]
  Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. We propose a novel hierarchical semi-supervised contrastive learning framework, for contamination-resistant anomaly detection.
  arXiv  Detail & Related papers  (2022-07-24T18:49:26Z)
• Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
  We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold. We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples. We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
  arXiv  Detail & Related papers  (2022-06-23T14:16:30Z)
• SLA$^2$P: Self-supervised Anomaly Detection with Adversarial Perturbation [77.71161225100927]
  Anomaly detection is a fundamental yet challenging problem in machine learning. We propose a novel and powerful framework, dubbed as SLA$2$P, for unsupervised anomaly detection.
  arXiv  Detail & Related papers  (2021-11-25T03:53:43Z)
• Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
  We introduce a novel weakly-supervised anomaly detection framework to train detection models. The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability. Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
  arXiv  Detail & Related papers  (2021-08-01T14:33:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.