MediCLIP: Adapting CLIP for Few-shot Medical Image Anomaly Detection

• URL: http://arxiv.org/abs/2405.11315v1
• Date: Sat, 18 May 2024 15:24:58 GMT
• Title: MediCLIP: Adapting CLIP for Few-shot Medical Image Anomaly Detection
• Authors: Ximiao Zhang, Min Xu, Dehui Qiu, Ruixin Yan, Ning Lang, Xiuzhuang Zhou,
• Abstract summary: This paper first focuses on the task of medical image anomaly detection in the few-shot setting. We propose an innovative approach, MediCLIP, which adapts the CLIP model to few-shot medical image anomaly detection through self-supervised fine-tuning.
• Score: 6.812281925604158
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the field of medical decision-making, precise anomaly detection in medical imaging plays a pivotal role in aiding clinicians. However, previous work is reliant on large-scale datasets for training anomaly detection models, which increases the development cost. This paper first focuses on the task of medical image anomaly detection in the few-shot setting, which is critically significant for the medical field where data collection and annotation are both very expensive. We propose an innovative approach, MediCLIP, which adapts the CLIP model to few-shot medical image anomaly detection through self-supervised fine-tuning. Although CLIP, as a vision-language model, demonstrates outstanding zero-/fewshot performance on various downstream tasks, it still falls short in the anomaly detection of medical images. To address this, we design a series of medical image anomaly synthesis tasks to simulate common disease patterns in medical imaging, transferring the powerful generalization capabilities of CLIP to the task of medical image anomaly detection. When only few-shot normal medical images are provided, MediCLIP achieves state-of-the-art performance in anomaly detection and location compared to other methods. Extensive experiments on three distinct medical anomaly detection tasks have demonstrated the superiority of our approach. The code is available at https://github.com/cnulab/MediCLIP.

Related papers

• Training Medical Large Vision-Language Models with Abnormal-Aware Feedback [57.98393950821579]
  We propose a novel UMed-LVLM designed with Unveiling Medical abnormalities. We propose a prompt method utilizing the GPT-4V to generate diagnoses based on identified abnormal areas in medical images. Experimental results demonstrate that our UMed-LVLM surpasses existing Med-LVLMs in identifying and understanding medical abnormality.
  arXiv  Detail & Related papers  (2025-01-02T17:37:20Z)
• Exploring Zero-Shot Anomaly Detection with CLIP in Medical Imaging: Are We There Yet? [0.0]
  We evaluate CLIP-based models, originally developed for industrial tasks, on brain tumor detection. While these models show promise in transferring general knowledge to medical tasks, their performance falls short of the precision required for clinical use.
  arXiv  Detail & Related papers  (2024-11-14T09:38:29Z)
• Spatial-aware Attention Generative Adversarial Network for Semi-supervised Anomaly Detection in Medical Image [63.59114880750643]
  We introduce a novel Spatial-aware Attention Generative Adrialversa Network (SAGAN) for one-class semi-supervised generation of health images. SAGAN generates high-quality health images corresponding to unlabeled data, guided by the reconstruction of normal images and restoration of pseudo-anomaly images. Extensive experiments on three medical datasets demonstrate that the proposed SAGAN outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-05-21T15:41:34Z)
• MedIAnomaly: A comparative study of anomaly detection in medical images [26.319602363581442]
  Anomaly detection (AD) aims at detecting abnormal samples that deviate from the expected normal patterns. Despite the emergence of numerous methods for medical AD, the lack of a fair and comprehensive evaluation causes ambiguous conclusions. This paper builds a benchmark with unified comparison to address this problem.
  arXiv  Detail & Related papers  (2024-04-06T06:18:11Z)
• Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
  This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-19T09:28:19Z)
• AnoDODE: Anomaly Detection with Diffusion ODE [0.0]
  Anomaly detection is the process of identifying atypical data samples that significantly deviate from the majority of the dataset. We propose a new anomaly detection method based on diffusion ODEs by estimating the density of features extracted from medical images. Our proposed method not only identifie anomalies but also provides interpretability at both the image and pixel levels.
  arXiv  Detail & Related papers  (2023-10-10T08:44:47Z)
• LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching [59.01894976615714]
  We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets. We have collected approximately 1.3 million medical images from 55 publicly available datasets. LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
  arXiv  Detail & Related papers  (2023-06-20T22:21:34Z)
• Explainable multiple abnormality classification of chest CT volumes with AxialNet and HiResCAM [89.2175350956813]
  We introduce the challenging new task of explainable multiple abnormality classification in volumetric medical images. We propose a multiple instance learning convolutional neural network, AxialNet, that allows identification of top slices for each abnormality. We then aim to improve the model's learning through a novel mask loss that leverages HiResCAM and 3D allowed regions.
  arXiv  Detail & Related papers  (2021-11-24T01:14:33Z)
• Convolutional-LSTM for Multi-Image to Single Output Medical Prediction [55.41644538483948]
  A common scenario in developing countries is to have the volume metadata lost due multiple reasons. It is possible to get a multi-image to single diagnostic model which mimics human doctor diagnostic process.
  arXiv  Detail & Related papers  (2020-10-20T04:30:09Z)
• Anomaly Detection in Medical Imaging with Deep Perceptual Autoencoders [1.7277957019593995]
  We introduce a new powerful method of image anomaly detection. It relies on the classical autoencoder approach with a re-designed training pipeline. It outperforms state-of-the-art approaches in complex medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-06-23T18:45:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.