Eyes Tell the Truth: GazeVal Highlights Shortcomings of Generative AI in Medical Imaging

• URL: http://arxiv.org/abs/2503.20967v1
• Date: Wed, 26 Mar 2025 20:11:07 GMT
• Title: Eyes Tell the Truth: GazeVal Highlights Shortcomings of Generative AI in Medical Imaging
• Authors: David Wong, Bin Wang, Gorkem Durak, Marouane Tliba, Akshay Chaudhari, Aladine Chetouani, Ahmet Enis Cetin, Cagdas Topel, Nicolo Gennaro, Camila Lopes Vendrami, Tugce Agirlar Trabzonlu, Amir Ali Rahsepar, Laetitia Perronne, Matthew Antalek, Onural Ozturk, Gokcan Okur, Andrew C. Gordon, Ayis Pyrros, Frank H. Miller, Amir Borhani, Hatice Savas, Eric Hart, Drew Torigian, Jayaram K. Udupa, Elizabeth Krupinski, Ulas Bagci,
• Abstract summary: We introduce GazeVal, a framework that synergizes expert eye-tracking data with direct radiological evaluations to assess the quality of synthetic medical images.<n>Experiments with sixteen radiologists revealed that 96.6% of the generated images were identified as fake.
• Score: 6.9112781921075195
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The demand for high-quality synthetic data for model training and augmentation has never been greater in medical imaging. However, current evaluations predominantly rely on computational metrics that fail to align with human expert recognition. This leads to synthetic images that may appear realistic numerically but lack clinical authenticity, posing significant challenges in ensuring the reliability and effectiveness of AI-driven medical tools. To address this gap, we introduce GazeVal, a practical framework that synergizes expert eye-tracking data with direct radiological evaluations to assess the quality of synthetic medical images. GazeVal leverages gaze patterns of radiologists as they provide a deeper understanding of how experts perceive and interact with synthetic data in different tasks (i.e., diagnostic or Turing tests). Experiments with sixteen radiologists revealed that 96.6% of the generated images (by the most recent state-of-the-art AI algorithm) were identified as fake, demonstrating the limitations of generative AI in producing clinically accurate images.

Related papers

• Advancing AI-Powered Medical Image Synthesis: Insights from MedVQA-GI Challenge Using CLIP, Fine-Tuned Stable Diffusion, and Dream-Booth + LoRA [3.290817968509798]
  The MEDVQA-GI challenge addresses the integration of AI-driven text-to-image generative models in medical diagnostics.<n>This study introduces a novel approach based on fine-tuned generative models to generate dynamic, scalable, and precise images from textual descriptions.<n>Our system integrates Stable Diffusion and DreamBooth models, as well as Low-Rank Adaptation (LORA) to generate high-fidelity medical images.
  arXiv  Detail & Related papers  (2025-02-28T02:49:45Z)
• Detecting Discrepancies Between AI-Generated and Natural Images Using Uncertainty [91.64626435585643]
  We propose a novel approach for detecting AI-generated images by leveraging predictive uncertainty to mitigate misuse and associated risks. The motivation arises from the fundamental assumption regarding the distributional discrepancy between natural and AI-generated images. We propose to leverage large-scale pre-trained models to calculate the uncertainty as the score for detecting AI-generated images.
  arXiv  Detail & Related papers  (2024-12-08T11:32:25Z)
• Privacy-Preserving Federated Foundation Model for Generalist Ultrasound Artificial Intelligence [83.02106623401885]
  We present UltraFedFM, an innovative privacy-preserving ultrasound foundation model. UltraFedFM is collaboratively pre-trained using federated learning across 16 distributed medical institutions in 9 countries. It achieves an average area under the receiver operating characteristic curve of 0.927 for disease diagnosis and a dice similarity coefficient of 0.878 for lesion segmentation.
  arXiv  Detail & Related papers  (2024-11-25T13:40:11Z)
• Clinical Evaluation of Medical Image Synthesis: A Case Study in Wireless Capsule Endoscopy [63.39037092484374]
  Synthetic Data Generation based on Artificial Intelligence (AI) can transform the way clinical medicine is delivered.<n>This study focuses on the clinical evaluation of medical SDG, with a proof-of-concept investigation on diagnosing Inflammatory Bowel Disease (IBD) using Wireless Capsule Endoscopy (WCE) images.<n>The results show that TIDE-II generates clinically plausible, very realistic WCE images, of improved quality compared to relevant state-of-the-art generative models.
  arXiv  Detail & Related papers  (2024-10-31T19:48:50Z)
• MediSyn: A Generalist Text-Guided Latent Diffusion Model For Diverse Medical Image Synthesis [4.541407789437896]
  MediSyn is a text-guided latent diffusion model capable of generating synthetic images from 6 medical specialties and 10 image types.<n>A direct comparison of the synthetic images against the real images confirms that our model synthesizes novel images and, crucially, may preserve patient privacy.<n>Our findings highlight the immense potential for generalist image generative models to accelerate algorithmic research and development in medicine.
  arXiv  Detail & Related papers  (2024-05-16T04:28:44Z)
• Synthetic Medical Imaging Generation with Generative Adversarial Networks For Plain Radiographs [34.98319691651471]
  The purpose of this investigation was to develop a reusable open-source synthetic image generation pipeline, the GAN Image Synthesis Tool (GIST) The pipeline helps to improve and standardize AI algorithms in the digital health space by generating high quality synthetic image data that is not linked to specific patients.
  arXiv  Detail & Related papers  (2024-03-28T02:51:33Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• Virtual imaging trials improved the transparency and reliability of AI systems in COVID-19 imaging [1.6040478776985583]
  This study focuses on using convolutional neural networks (CNNs) for COVID-19 diagnosis using computed tomography (CT) and chest radiography (CXR) We developed and tested multiple AI models, 3D ResNet-like and 2D EfficientNetv2 architectures, across diverse datasets. Models trained on the most diverse datasets showed the highest external testing performance, with AUC values ranging from 0.73 to 0.76 for CT and 0.70 to 0.73 for CXR.
  arXiv  Detail & Related papers  (2023-08-17T19:12:32Z)
• Aligning Synthetic Medical Images with Clinical Knowledge using Human Feedback [22.390670838355295]
  This paper introduces a pathologist-in-the-loop framework for generating clinically-plausible synthetic medical images. We show that human feedback significantly improves the quality of synthetic images in terms of fidelity, diversity, utility in downstream applications, and plausibility as evaluated by experts.
  arXiv  Detail & Related papers  (2023-06-16T21:54:20Z)
• Bi-parametric prostate MR image synthesis using pathology and sequence-conditioned stable diffusion [3.290987481767681]
  We propose an image synthesis mechanism for multi-sequence prostate MR images conditioned on text. We generate paired bi-parametric images conditioned on images conditioned on paired data. We validate our method using 2D image slices from real suspected prostate cancer patients.
  arXiv  Detail & Related papers  (2023-03-03T17:24:39Z)
• Variational Knowledge Distillation for Disease Classification in Chest X-Rays [102.04931207504173]
  We propose itvariational knowledge distillation (VKD), which is a new probabilistic inference framework for disease classification based on X-rays. We demonstrate the effectiveness of our method on three public benchmark datasets with paired X-ray images and EHRs.
  arXiv  Detail & Related papers  (2021-03-19T14:13:56Z)
• Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation and Diagnosis for COVID-19 [71.41929762209328]
  The pandemic of coronavirus disease 2019 (COVID-19) is spreading all over the world. Medical imaging such as X-ray and computed tomography (CT) plays an essential role in the global fight against COVID-19. The recently emerging artificial intelligence (AI) technologies further strengthen the power of the imaging tools and help medical specialists.
  arXiv  Detail & Related papers  (2020-04-06T15:21:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.