Dual Distillation for Few-Shot Anomaly Detection

• URL: http://arxiv.org/abs/2603.01713v1
• Date: Mon, 02 Mar 2026 10:38:19 GMT
• Title: Dual Distillation for Few-Shot Anomaly Detection
• Authors: Le Dong, Qinzhong Tan, Chunlei Li, Jingliang Hu, Yilei Shi, Weisheng Dong, Xiao Xiang Zhu, Lichao Mou,
• Abstract summary: Anomaly detection is a critical task in computer vision with profound implications for medical imaging.<n>We introduce D$2$4FAD, a novel dual distillation framework for few-shot anomaly detection.<n>Our method identifies anomalies in previously unseen tasks using only a small number of normal reference images.
• Score: 41.127862518102425
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Anomaly detection is a critical task in computer vision with profound implications for medical imaging, where identifying pathologies early can directly impact patient outcomes. While recent unsupervised anomaly detection approaches show promise, they require substantial normal training data and struggle to generalize across anatomical contexts. We introduce D$^2$4FAD, a novel dual distillation framework for few-shot anomaly detection that identifies anomalies in previously unseen tasks using only a small number of normal reference images. Our approach leverages a pre-trained encoder as a teacher network to extract multi-scale features from both support and query images, while a student decoder learns to distill knowledge from the teacher on query images and self-distill on support images. We further propose a learn-to-weight mechanism that dynamically assesses the reference value of each support image conditioned on the query, optimizing anomaly detection performance. To evaluate our method, we curate a comprehensive benchmark dataset comprising 13,084 images across four organs, four imaging modalities, and five disease categories. Extensive experiments demonstrate that D$^2$4FAD significantly outperforms existing approaches, establishing a new state-of-the-art in few-shot medical anomaly detection. Code is available at https://github.com/ttttqz/D24FAD.

Related papers

• Foundation Visual Encoders Are Secretly Few-Shot Anomaly Detectors [58.75916798814376]
  We develop a few-shot anomaly detector termed FoundAD.<n>We observe that the anomaly amount in an image directly correlates with the difference in the learnt embeddings.<n>The simple operator acts as an effective tool for anomaly detection to characterize and identify out-of-distribution regions in an image.
  arXiv  Detail & Related papers  (2025-10-02T11:53:20Z)
• OCSVM-Guided Representation Learning for Unsupervised Anomaly Detection [1.0190194769786831]
  Unsupervised anomaly detection (UAD) aims to detect anomalies without labeled data.<n>We propose a novel method that tightly couples representation learning with an analytically solvable one-class SVM.<n>The model is evaluated on two tasks: a new benchmark based on MNIST-C, and a challenging brain MRI subtle lesion detection task.
  arXiv  Detail & Related papers  (2025-07-25T13:00:40Z)
• Spatial-aware Attention Generative Adversarial Network for Semi-supervised Anomaly Detection in Medical Image [63.59114880750643]
  We introduce a novel Spatial-aware Attention Generative Adrialversa Network (SAGAN) for one-class semi-supervised generation of health images. SAGAN generates high-quality health images corresponding to unlabeled data, guided by the reconstruction of normal images and restoration of pseudo-anomaly images. Extensive experiments on three medical datasets demonstrate that the proposed SAGAN outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-05-21T15:41:34Z)
• Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
  This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-19T09:28:19Z)
• Benchmarking Pretrained Vision Embeddings for Near- and Duplicate Detection in Medical Images [0.6827423171182154]
  We present an approach for identifying near- and duplicate 3D medical images leveraging publicly available 2D computer vision embeddings. We generate an experimental benchmark based on the publicly available Medical Decathlon dataset.
  arXiv  Detail & Related papers  (2023-12-12T13:52:55Z)
• Detecting Heart Disease from Multi-View Ultrasound Images via Supervised Attention Multiple Instance Learning [5.201499435914216]
  Aortic stenosis (AS) is a degenerative valve condition that causes substantial morbidity and mortality. Previous approaches to AS detection yield insufficient accuracy due to relying on inflexible averages across images. We contribute a new end-to-end MIL approach with two key methodological innovations.
  arXiv  Detail & Related papers  (2023-05-25T18:22:12Z)
• Unveiling the Unseen: A Comprehensive Survey on Explainable Anomaly Detection in Images and Videos [49.07140708026425]
  Anomaly detection and localization in visual data, including images and videos, are crucial in machine learning and real-world applications.<n>This paper provides the first comprehensive survey focused specifically on explainable 2D visual anomaly detection (X-VAD)<n>We present a literature review of explainable methods, categorized by their underlying techniques.<n>We discuss promising future directions and open problems, including quantifying explanation quality.
  arXiv  Detail & Related papers  (2023-02-13T20:17:41Z)
• The role of noise in denoising models for anomaly detection in medical images [62.0532151156057]
  Pathological brain lesions exhibit diverse appearance in brain images. Unsupervised anomaly detection approaches have been proposed using only normal data for training. We show that optimization of the spatial resolution and magnitude of the noise improves the performance of different model training regimes.
  arXiv  Detail & Related papers  (2023-01-19T21:39:38Z)
• Learning Discriminative Representation via Metric Learning for Imbalanced Medical Image Classification [52.94051907952536]
  We propose embedding metric learning into the first stage of the two-stage framework specially to help the feature extractor learn to extract more discriminative feature representations. Experiments mainly on three medical image datasets show that the proposed approach consistently outperforms existing onestage and two-stage approaches.
  arXiv  Detail & Related papers  (2022-07-14T14:57:01Z)
• Anomaly Detection in Medical Imaging with Deep Perceptual Autoencoders [1.7277957019593995]
  We introduce a new powerful method of image anomaly detection. It relies on the classical autoencoder approach with a re-designed training pipeline. It outperforms state-of-the-art approaches in complex medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-06-23T18:45:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.