Exploring Zero-Shot Anomaly Detection with CLIP in Medical Imaging: Are We There Yet?

• URL: http://arxiv.org/abs/2411.09310v1
• Date: Thu, 14 Nov 2024 09:38:29 GMT
• Title: Exploring Zero-Shot Anomaly Detection with CLIP in Medical Imaging: Are We There Yet?
• Authors: Aldo Marzullo, Marta Bianca Maria Ranzini,
• Abstract summary: We evaluate CLIP-based models, originally developed for industrial tasks, on brain tumor detection. While these models show promise in transferring general knowledge to medical tasks, their performance falls short of the precision required for clinical use.
• Score: 0.0
• License:
• Abstract: Zero-shot anomaly detection (ZSAD) offers potential for identifying anomalies in medical imaging without task-specific training. In this paper, we evaluate CLIP-based models, originally developed for industrial tasks, on brain tumor detection using the BraTS-MET dataset. Our analysis examines their ability to detect medical-specific anomalies with no or minimal supervision, addressing the challenges posed by limited data annotation. While these models show promise in transferring general knowledge to medical tasks, their performance falls short of the precision required for clinical use. Our findings highlight the need for further adaptation before CLIP-based models can be reliably applied to medical anomaly detection.

Related papers

• Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction [10.403187385041702]
  We introduce MERA, a clinical diagnosis prediction model that bridges pertaining natural language knowledge with medical practice. We apply hierarchical contrastive learning on a disease candidate ranking list to alleviate the large decision space issue.
  arXiv  Detail & Related papers  (2025-01-28T22:38:45Z)
• On the challenges of detecting MCI using EEG in the wild [6.505818939553856]
  Recent studies have shown promising results in the detection of Mild Cognitive Impairment (MCI) using Electroencephalogram (EEG) data. We investigate the potential limitations and challenges in developing a robust MCI detection method using two contrasting datasets.
  arXiv  Detail & Related papers  (2025-01-15T15:20:11Z)
• Training Medical Large Vision-Language Models with Abnormal-Aware Feedback [57.98393950821579]
  We propose a novel UMed-LVLM designed with Unveiling Medical abnormalities. We propose a prompt method utilizing the GPT-4V to generate diagnoses based on identified abnormal areas in medical images. Experimental results demonstrate that our UMed-LVLM surpasses existing Med-LVLMs in identifying and understanding medical abnormality.
  arXiv  Detail & Related papers  (2025-01-02T17:37:20Z)
• FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injection [83.54960238236548]
  FEDMEKI not only preserves data privacy but also enhances the capability of medical foundation models. FEDMEKI allows medical foundation models to learn from a broader spectrum of medical knowledge without direct data exposure.
  arXiv  Detail & Related papers  (2024-08-17T15:18:56Z)
• Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports [51.45762396192655]
  Multimodal large language models (MLLMs) have recently transformed many domains, significantly affecting the medical field. Notably, Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models have epitomized a paradigm shift in Artificial General Intelligence for computer vision. This study evaluated the performance of the Gemini, GPT-4, and 4 popular large models for an exhaustive evaluation across 14 medical imaging datasets.
  arXiv  Detail & Related papers  (2024-07-08T09:08:42Z)
• MediCLIP: Adapting CLIP for Few-shot Medical Image Anomaly Detection [6.812281925604158]
  This paper first focuses on the task of medical image anomaly detection in the few-shot setting. We propose an innovative approach, MediCLIP, which adapts the CLIP model to few-shot medical image anomaly detection through self-supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-05-18T15:24:58Z)
• Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
  This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-19T09:28:19Z)
• Revamping AI Models in Dermatology: Overcoming Critical Challenges for Enhanced Skin Lesion Diagnosis [8.430482797862926]
  We present an All-In-One textbfHierarchical-textbfOut of Distribution-textbfClinical Triage model. For a clinical image, our model generates three outputs: a hierarchical prediction, an alert for out-of-distribution images, and a recommendation for dermoscopy. Our versatile model provides valuable decision support for lesion diagnosis and sets a promising precedent for medical AI applications.
  arXiv  Detail & Related papers  (2023-11-02T06:08:49Z)
• Risk Assessment of Lymph Node Metastases in Endometrial Cancer Patients: A Causal Approach [1.8933952173153485]
  We introduce a causal discovery algorithm for causal Bayesian networks based on bootstrap resampling. We discuss the strengths and limitations of our findings in light of the presence of missing data that may be missing-not-at-random.
  arXiv  Detail & Related papers  (2023-05-17T08:33:32Z)
• Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation [116.87918100031153]
  We propose a Cross-modal clinical Graph Transformer (CGT) for ophthalmic report generation (ORG) CGT injects clinical relation triples into the visual features as prior knowledge to drive the decoding procedure. Experiments on the large-scale FFA-IR benchmark demonstrate that the proposed CGT is able to outperform previous benchmark methods.
  arXiv  Detail & Related papers  (2022-06-04T13:16:30Z)
• Hemogram Data as a Tool for Decision-making in COVID-19 Management: Applications to Resource Scarcity Scenarios [62.997667081978825]
  COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure. This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients. Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
  arXiv  Detail & Related papers  (2020-05-10T01:45:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.