CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models

• URL: http://arxiv.org/abs/2406.06007v3
• Date: Sun, 03 Nov 2024 16:54:14 GMT
• Title: CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models
• Authors: Peng Xia, Ze Chen, Juanxi Tian, Yangrui Gong, Ruibo Hou, Yue Xu, Zhenbang Wu, Zhiyuan Fan, Yiyang Zhou, Kangyu Zhu, Wenhao Zheng, Zhaoyang Wang, Xiao Wang, Xuchao Zhang, Chetan Bansal, Marc Niethammer, Junzhou Huang, Hongtu Zhu, Yun Li, Jimeng Sun, Zongyuan Ge, Gang Li, James Zou, Huaxiu Yao,
• Abstract summary: We introduce CARES and aim to evaluate the Trustworthiness of Med-LVLMs across the medical domain. We assess the trustworthiness of Med-LVLMs across five dimensions, including trustfulness, fairness, safety, privacy, and robustness.
• Score: 92.04812189642418
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Artificial intelligence has significantly impacted medical applications, particularly with the advent of Medical Large Vision Language Models (Med-LVLMs), sparking optimism for the future of automated and personalized healthcare. However, the trustworthiness of Med-LVLMs remains unverified, posing significant risks for future model deployment. In this paper, we introduce CARES and aim to comprehensively evaluate the Trustworthiness of Med-LVLMs across the medical domain. We assess the trustworthiness of Med-LVLMs across five dimensions, including trustfulness, fairness, safety, privacy, and robustness. CARES comprises about 41K question-answer pairs in both closed and open-ended formats, covering 16 medical image modalities and 27 anatomical regions. Our analysis reveals that the models consistently exhibit concerns regarding trustworthiness, often displaying factual inaccuracies and failing to maintain fairness across different demographic groups. Furthermore, they are vulnerable to attacks and demonstrate a lack of privacy awareness. We publicly release our benchmark and code in https://cares-ai.github.io/.

Related papers

• REVAL: A Comprehension Evaluation on Reliability and Values of Large Vision-Language Models [59.445672459851274]
  REVAL is a comprehensive benchmark designed to evaluate the textbfREliability and textbfVALue of Large Vision-Language Models. REVAL encompasses over 144K image-text Visual Question Answering (VQA) samples, structured into two primary sections: Reliability and Values. We evaluate 26 models, including mainstream open-source LVLMs and prominent closed-source models like GPT-4o and Gemini-1.5-Pro.
  arXiv  Detail & Related papers  (2025-03-20T07:54:35Z)
• The Reliability of LLMs for Medical Diagnosis: An Examination of Consistency, Manipulation, and Contextual Awareness [0.0]
  Large Language Models (LLMs) offer promise for democratizing healthcare with advanced diagnostics. This study assesses their diagnostic reliability focusing on consistency, manipulation resilience, and contextual integration. LLMs' vulnerability to manipulation and limited contextual awareness pose challenges in clinical use.
  arXiv  Detail & Related papers  (2025-03-02T11:50:16Z)
• Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment [79.41098832007819]
  Medical multimodal large language models (MLLMs) are becoming an instrumental part of healthcare systems. As medical data is scarce and protected by privacy regulations, medical MLLMs represent valuable intellectual property. We introduce Adversarial Domain Alignment (ADA-STEAL), the first stealing attack against medical MLLMs.
  arXiv  Detail & Related papers  (2025-02-04T16:04:48Z)
• AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Driving [106.0319745724181]
  We introduce AutoTrust, a comprehensive trustworthiness benchmark for large vision-language models in autonomous driving (DriveVLMs) We constructed the largest visual question-answering dataset for investigating trustworthiness issues in driving scenarios. Our evaluations have unveiled previously undiscovered vulnerabilities of DriveVLMs to trustworthiness threats.
  arXiv  Detail & Related papers  (2024-12-19T18:59:33Z)
• Ensuring Safety and Trust: Analyzing the Risks of Large Language Models in Medicine [41.71754418349046]
  We propose five key principles for safe and trustworthy medical AI, along with ten specific aspects. Under this comprehensive framework, we introduce a novel MedGuard benchmark with 1,000 expert-verified questions. Our evaluation of 11 commonly used LLMs shows that the current language models, regardless of their safety alignment mechanisms, generally perform poorly on most of our benchmarks. This study underscores a significant safety gap, highlighting the crucial need for human oversight and the implementation of AI safety guardrails.
  arXiv  Detail & Related papers  (2024-11-20T06:34:32Z)
• Which Client is Reliable?: A Reliable and Personalized Prompt-based Federated Learning for Medical Image Question Answering [51.26412822853409]
  We present a novel personalized federated learning (pFL) method for medical visual question answering (VQA) models. Our method introduces learnable prompts into a Transformer architecture to efficiently train it on diverse medical datasets without massive computational costs.
  arXiv  Detail & Related papers  (2024-10-23T00:31:17Z)
• MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models [20.781551849965357]
  We introduce MediConfusion, a challenging medical Visual Question Answering (VQA) benchmark dataset. We reveal that state-of-the-art models are easily confused by image pairs that are otherwise visually dissimilar and clearly distinct for medical experts. We also extract common patterns of model failure that may help the design of a new generation of more trustworthy and reliable MLLMs in healthcare.
  arXiv  Detail & Related papers  (2024-09-23T18:59:37Z)
• A Survey on Trustworthiness in Foundation Models for Medical Image Analysis [27.876946673940452]
  We present a novel taxonomy of foundation models used in medical imaging. We focus on segmentation, medical report generation, medical question and answering (Q&A), and disease diagnosis. Our analysis underscores the imperative for advancing towards trustworthy AI in medical image analysis.
  arXiv  Detail & Related papers  (2024-07-03T18:07:57Z)
• Benchmarking Trustworthiness of Multimodal Large Language Models: A Comprehensive Study [51.19622266249408]
  MultiTrust is the first comprehensive and unified benchmark on the trustworthiness of MLLMs. Our benchmark employs a rigorous evaluation strategy that addresses both multimodal risks and cross-modal impacts. Extensive experiments with 21 modern MLLMs reveal some previously unexplored trustworthiness issues and risks.
  arXiv  Detail & Related papers  (2024-06-11T08:38:13Z)
• Asclepius: A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models [59.60384461302662]
  We introduce Asclepius, a novel benchmark for evaluating Medical Multi-Modal Large Language Models (Med-MLLMs) Asclepius rigorously and comprehensively assesses model capability in terms of distinct medical specialties and different diagnostic capacities. We also provide an in-depth analysis of 6 Med-MLLMs and compare them with 5 human specialists.
  arXiv  Detail & Related papers  (2024-02-17T08:04:23Z)
• OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM [48.16696073640864]
  We introduce OmniMedVQA, a novel comprehensive medical Visual Question Answering (VQA) benchmark. All images in this benchmark are sourced from authentic medical scenarios. We have found that existing LVLMs struggle to address these medical VQA problems effectively.
  arXiv  Detail & Related papers  (2024-02-14T13:51:56Z)
• Medical Foundation Models are Susceptible to Targeted Misinformation Attacks [3.252906830953028]
  Large language models (LLMs) have broad medical knowledge and can reason about medical information across many domains. We demonstrate a concerning vulnerability of LLMs in medicine through targeted manipulation of just 1.1% of the model's weights. We validate our findings in a set of 1,038 incorrect biomedical facts.
  arXiv  Detail & Related papers  (2023-09-29T06:44:36Z)
• Med-Flamingo: a Multimodal Medical Few-shot Learner [58.85676013818811]
  We propose Med-Flamingo, a multimodal few-shot learner adapted to the medical domain. Based on OpenFlamingo-9B, we continue pre-training on paired and interleaved medical image-text data from publications and textbooks. We conduct the first human evaluation for generative medical VQA where physicians review the problems and blinded generations in an interactive app.
  arXiv  Detail & Related papers  (2023-07-27T20:36:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.