Keep the Needle, Prune the Haystack: Defect-Preserving Token Pruning for Efficient Zero-Shot Anomaly Detection

• URL: http://arxiv.org/abs/2608.03681v3
• Date: Sun, 09 Aug 2026 09:55:13 GMT
• Title: Keep the Needle, Prune the Haystack: Defect-Preserving Token Pruning for Efficient Zero-Shot Anomaly Detection
• Abstract summary: Token pruning offers a promising solution, but introduces an asymmetric pruning risk in anomaly detection.<n>We propose KeepAD, a defect-preserving token pruning framework that formulates token selection as high-recall, anomaly-aware routing.<n> Experiments show that KeepAD reduces the token retention ratio to below $20%$, while limiting the average degradation in image-level and pixel-level AUROC to within $2.7$ percentage points.
• Score: 23.595452794635847
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-shot visual anomaly detection has achieved remarkable progress, with recent vision-only approaches further improving performance while simplifying the inference pipeline. However, existing methods typically perform dense computation over all images and spatial tokens, despite the fact that normal samples dominate real-world scenarios and anomalies usually occupy only small regions. Token pruning offers a promising solution, but introduces an asymmetric pruning risk in anomaly detection: retaining normal tokens mainly incurs redundant computation, whereas removing anomalous tokens may eliminate the only evidence for detection and localization. This risk is particularly severe in early layers, where pruning provides the greatest computational benefit but anomaly semantics remain unreliable. We propose KeepAD, a defect-preserving token pruning framework that formulates token selection as high-recall, anomaly-aware routing. In shallow layers, KeepAD combines coverage-preserving selection over local $2\times2$ patch neighborhoods with deterministic anomaly rescue to reduce the risk of discarding subtle defects. In deeper layers, frozen normal and abnormal prototypes guide pruning under an image-adaptive token budget, aggressively removing low-risk normal tokens while preserving local anomaly evidence. Dense-to-sparse self-distillation further supervises early token routing without introducing additional inference overhead. Experiments on six industrial and seven medical zero-shot anomaly detection benchmarks show that KeepAD reduces the token retention ratio to below $20\%$, while limiting the average degradation in image-level and pixel-level AUROC to within $2.7$ percentage points. At the most aggressive operating point, KeepAD achieves a $7.9\times$ speedup over the strongest CLIP-based baseline.

Related papers

• Bayesian Best-Arm Identification with Abstention: A Polynomial-to-Exponential Phase Transition [51.253617466579335]
  We study the Bayesian fixed-budget best-arm identification problem in which a learner can abstain from making a terminal recommendation.<n>We analyze the probability of undetected error--the risk of recommending a suboptimal arm without abstaining.
  arXiv  Detail & Related papers  (2026-06-28T05:07:27Z)
• DCP-Prune: Ultra-Low Token Pruning with Distribution Consistency Preservation [55.38884985761757]
  Recent vision token pruning methods effectively preserve model performance under moderate token budgets but become unstable under ultra-low token budget.<n>We propose a two-stage pruning framework consisting of Anchor-Context Graph Recovery (ACGR) and Text-Aware Token Cluster Selection (TATCS)<n>Experiments demonstrate that our method achieves superior and more stable performance under ultra-low token budget.
  arXiv  Detail & Related papers  (2026-06-15T12:24:07Z)
• Localization then Neutralization: Gradient-guided Token Suppression against Visual Prompt Injection Attack [78.02110947708535]
  Adversarial images pose a severe security threat to multimodal large language models through prompt injection.<n>We show that successful adversarial attacks do not rely on the entire image uniformly but instead depend on a small subset of critical image tokens.<n>We propose Gradient Token Masking (GTM), which localizes these tokens via gradient analysis and neutralizes them through masking.
  arXiv  Detail & Related papers  (2026-05-24T17:51:34Z)
• Mitigating the ID-OOD Tradeoff in Open-Set Test-Time Adaptation [52.620992698137236]
  Open-set test-time adaptation (OSTTA) addresses the challenge of adapting models to new environments.<n>In this paper, we first analyze the limitations of entropy in OSTTA and then introduce an angular loss to regulate feature norm magnitudes.<n>Our method achieves strong OOD detection while maintaining high ID classification performance.
  arXiv  Detail & Related papers  (2026-04-02T04:00:57Z)
• IMPACT: Influence Modeling for Open-Set Time Series Anomaly Detection [41.04237014380757]
  Open-set anomaly detection (OSAD) is an emerging paradigm designed to utilize limited labeled data from anomaly classes seen in training to identify both seen and unseen anomalies during testing.<n>Current approaches rely on simple augmentation methods to generate pseudo anomalies that replicate unseen anomalies.<n>This work introduces $textbfi$nfluence $underlinetextbfm$odeling for o$underlinetextbfp$en-set time series.
  arXiv  Detail & Related papers  (2026-03-31T02:49:46Z)
• DRL-Guided Neural Batch Sampling for Semi-Supervised Pixel-Level Anomaly Detection [0.0]
  Anomaly detection in industrial visual inspection is challenging due to the scarcity of defective samples.<n>We propose a semi-supervised deep reinforcement learning framework that integrates a neural batch sampler, an autoencoder, and a predictor.
  arXiv  Detail & Related papers  (2025-11-25T12:53:53Z)
• Towards Multiple Missing Values-resistant Unsupervised Graph Anomaly Detection [10.061847825452677]
  We propose M$2$V-UGAD, a multiple missing values-resistant unsupervised GAD framework on incomplete graphs.<n>M$2$V-UGAD consistently outperforms existing unsupervised GAD methods across varying missing rates.
  arXiv  Detail & Related papers  (2025-11-13T03:30:43Z)
• Industrial Anomaly Detection and Localization Using Weakly-Supervised Residual Transformers [44.344548601242444]
  We introduce a novel framework, Weakly-supervised RESidual Transformer (WeakREST), to achieve high anomaly detection accuracy.<n>We reformulate the pixel-wise anomaly localization task into a block-wise classification problem.<n>We develop a novel ResMixMatch algorithm, capable of handling the interplay between weak labels and residual-based representations.
  arXiv  Detail & Related papers  (2023-06-06T08:19:30Z)
• Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
  We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold. We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples. We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
  arXiv  Detail & Related papers  (2022-06-23T14:16:30Z)
• SLA$^2$P: Self-supervised Anomaly Detection with Adversarial Perturbation [77.71161225100927]
  Anomaly detection is a fundamental yet challenging problem in machine learning. We propose a novel and powerful framework, dubbed as SLA$2$P, for unsupervised anomaly detection.
  arXiv  Detail & Related papers  (2021-11-25T03:53:43Z)
• Mean-Shifted Contrastive Loss for Anomaly Detection [34.97652735163338]
  We propose a new loss function which can overcome failure modes of both center-loss and contrastive-loss methods. Our improvements yield a new anomaly detection approach, based on $textitMean-Shifted Contrastive Loss$. Our method achieves state-of-the-art anomaly detection performance on multiple benchmarks including $97.5%$ ROC-AUC.
  arXiv  Detail & Related papers  (2021-06-07T17:58:03Z)
• Patch-wise++ Perturbation for Adversarial Targeted Attacks [132.58673733817838]
  We propose a patch-wise iterative method (PIM) aimed at crafting adversarial examples with high transferability. Specifically, we introduce an amplification factor to the step size in each iteration, and one pixel's overall gradient overflowing the $epsilon$-constraint is properly assigned to its surrounding regions. Compared with the current state-of-the-art attack methods, we significantly improve the success rate by 35.9% for defense models and 32.7% for normally trained models.
  arXiv  Detail & Related papers  (2020-12-31T08:40:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.