Rethinking Weak Supervision in Anomaly Detection: A Comprehensive Benchmark

• URL: http://arxiv.org/abs/2605.26068v3
• Date: Sat, 30 May 2026 00:15:58 GMT
• Title: Rethinking Weak Supervision in Anomaly Detection: A Comprehensive Benchmark
• Abstract summary: This paper introduces WSADBench, the first benchmark that unifies evaluation across distinct weakly supervised scenarios.<n>Based on over 700K experiments, WSADBench reveals four critical insights.
• Score: 41.98173822607898
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Weakly supervised anomaly detection (WSAD) has developed in three primary directions: incomplete, inexact, and inaccurate supervision. However, these directions remain isolated, lacking a unified framework to assess whether they address unique challenges or share fundamental mechanisms. This paper introduces WSADBench, the first benchmark that unifies evaluation across distinct weakly supervised scenarios, benchmarking diverse approaches from specialized WSAD methods to advanced tabular foundation models. WSADBench establishes standardized protocols to evaluate 36 algorithms across 4 modalities by systematically varying label quantity, granularity, and quality, revealing the performance boundaries of various methods. Based on over 700K experiments, WSADBench reveals four critical insights: (i) Strong intrinsic correlations exist between these weak supervision scenarios, challenging the isolation of current research directions. (ii) Specialized WSAD algorithms excel only in extreme label-scarcity regimes but are quickly dominated by tabular foundation models and general classification methods as supervision increases or in OOD scenarios. (iii) Unlabeled data shows inconsistent utility across settings, with marginal gains compared to label refinement. (iv) Models exhibit asymmetric sensitivity to different types of label noise. We release WSADBench as an open-source benchmark with code and datasets to facilitate future WSAD research: https://github.com/SUFE-AILAB/WSADBench.

Related papers

• Mind the Gap: Robustness Risks in PII Detection Systems [6.167318681955322]
  We study how different PII detection paradigms fail under noisy, unstructured, and informal inputs.<n>No single architecture is uniformly reliable across PII categories.<n>Motivated by these findings, we propose a hybrid detection pipeline with a QA-driven feedback loop for iterative risk mitigation.
  arXiv  Detail & Related papers  (2026-09-03T07:23:28Z)
• Evaluating Agentic AI in the Wild: Failure Modes, Drift Patterns, and a Production Evaluation Framework [0.0]
  We present a taxonomy of seven failure modes unique to production agentic systems.<n>Standard metrics fail to detect four of the seven failure modes entirely.<n>We propose PAEF, a five-dimension evaluation framework with an open-source reference implementation.
  arXiv  Detail & Related papers  (2026-05-02T21:02:08Z)
• EXHIB: A Benchmark for Realistic and Diverse Evaluation of Function Similarity in the Wild [9.69274678999646]
  We introduce EXHIB, a benchmark comprising five realistic datasets collected from the wild.<n>We evaluate 9 representative models spanning multiple BFSD paradigms on EXHIB.<n>Our results show that robustness to low- and mid-level binary variations does not generalize to high-level semantic differences.
  arXiv  Detail & Related papers  (2026-04-02T02:58:55Z)
• DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning [64.33887406863899]
  Multimodal deception detection aims to identify deceptive behavior by analyzing audiovisual cues for forensics and security.<n>Existing benchmarks provide only binary labels without intermediate reasoning cues.<n>We construct reasoning datasets with structured cue-level descriptions and reasoning chains.<n>With 1695 samples, it is the largest non-laboratory deception detection dataset.
  arXiv  Detail & Related papers  (2026-03-25T04:06:36Z)
• Refining Decision Boundaries In Anomaly Detection Using Similarity Search Within the Feature Space [3.3202103799131795]
  We introduce SDA2E, a Sparse Dual Adversarial Attention-based AutoEncoder designed to learn compact and discriminative latent representations from imbalanced, high-dimensional data.<n>We propose a similarity-guided active learning framework that integrates three novel strategies to refine decision boundaries efficiently.<n>We evaluate SDA2E extensively across 52 imbalanced datasets, including multiple DARPA Transparent Computing scenarios, and benchmark it against 15 state-of-the-art anomaly detection methods.
  arXiv  Detail & Related papers  (2026-02-02T23:55:08Z)
• Retrieving Semantically Similar Decisions under Noisy Institutional Labels: Robust Comparison of Embedding Methods [0.0]
  General-purpose embedder (OpenAI) outperforms domain-specific BERT-trained from scratch on 30,000 decisions.<n>Our framework is robust enough to be used for evaluation under a noisy gold dataset.
  arXiv  Detail & Related papers  (2025-12-05T12:54:26Z)
• Labels Matter More Than Models: Quantifying the Benefit of Supervised Time Series Anomaly Detection [56.302586730134806]
  Time series anomaly detection (TSAD) is a critical data mining task often constrained by label scarcity.<n>Current research predominantly focuses on Unsupervised Time-series Anomaly Detection.<n>This paper challenges the premise that architectural complexity is the optimal path for TSAD.
  arXiv  Detail & Related papers  (2025-11-20T08:32:49Z)
• Unifying Unsupervised Graph-Level Anomaly Detection and Out-of-Distribution Detection: A Benchmark [73.58840254552656]
  Unsupervised graph-level anomaly detection (GLAD) and unsupervised graph-level out-of-distribution (OOD) detection have received significant attention in recent years.<n>We present a underlinetextbfUnified underlinetextbfBenchmark for unsupervised underlinetextbfGraph-level underlinetextbfOOD and anomaunderlinetextbfLy underlinetextbfDetection (ourmethod)<n>Our benchmark encompasses 35 datasets
  arXiv  Detail & Related papers  (2024-06-21T04:07:43Z)
• DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection [55.70982767084996]
  A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark. We present the first comprehensive benchmark for deepfake detection, called DeepfakeBench, which offers three key contributions. DeepfakeBench contains 15 state-of-the-art detection methods, 9CL datasets, a series of deepfake detection evaluation protocols and analysis tools, as well as comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-07-04T01:34:41Z)
• WRENCH: A Comprehensive Benchmark for Weak Supervision [66.82046201714766]
  benchmark consists of 22 varied real-world datasets for classification and sequence tagging. We use benchmark to conduct extensive comparisons over more than 100 method variants to demonstrate its efficacy as a benchmark platform.
  arXiv  Detail & Related papers  (2021-09-23T13:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.