Vision Language Models in Medicine

• URL: http://arxiv.org/abs/2503.01863v1
• Date: Mon, 24 Feb 2025 22:53:22 GMT
• Title: Vision Language Models in Medicine
• Authors: Beria Chingnabe Kalpelbe, Angel Gabriel Adaambiik, Wei Peng,
• Abstract summary: Medical Vision-Language Models (Med-VLMs) integrate visual and textual data to enhance healthcare outcomes.<n>The transformative impact of Med-VLMs on clinical practice, education, and patient care is highlighted.<n> challenges like data scarcity, narrow task generalization, interpretability issues, and ethical concerns like fairness, accountability, and privacy are highlighted.<n>Future directions include leveraging large-scale, diverse datasets, improving cross-modal generalization, and enhancing interpretability.
• Score: 3.964982657945488
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: With the advent of Vision-Language Models (VLMs), medical artificial intelligence (AI) has experienced significant technological progress and paradigm shifts. This survey provides an extensive review of recent advancements in Medical Vision-Language Models (Med-VLMs), which integrate visual and textual data to enhance healthcare outcomes. We discuss the foundational technology behind Med-VLMs, illustrating how general models are adapted for complex medical tasks, and examine their applications in healthcare. The transformative impact of Med-VLMs on clinical practice, education, and patient care is highlighted, alongside challenges such as data scarcity, narrow task generalization, interpretability issues, and ethical concerns like fairness, accountability, and privacy. These limitations are exacerbated by uneven dataset distribution, computational demands, and regulatory hurdles. Rigorous evaluation methods and robust regulatory frameworks are essential for safe integration into healthcare workflows. Future directions include leveraging large-scale, diverse datasets, improving cross-modal generalization, and enhancing interpretability. Innovations like federated learning, lightweight architectures, and Electronic Health Record (EHR) integration are explored as pathways to democratize access and improve clinical relevance. This review aims to provide a comprehensive understanding of Med-VLMs' strengths and limitations, fostering their ethical and balanced adoption in healthcare.

Related papers

• HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation [68.4316501012718]
  HealthGPT is a powerful Medical Large Vision-Language Model (Med-LVLM)<n>It integrates medical visual comprehension and generation capabilities within a unified autoregressive paradigm.
  arXiv  Detail & Related papers  (2025-02-14T00:42:36Z)
• Large Language Models in Healthcare [4.119811542729794]
  Large language models (LLMs) hold promise for transforming healthcare. Their successful integration requires rigorous development, adaptation, and evaluation strategies tailored to clinical needs.
  arXiv  Detail & Related papers  (2025-02-06T20:53:33Z)
• A Survey of Medical Vision-and-Language Applications and Their Techniques [48.268198631277315]
  Medical vision-and-language models (MVLMs) have attracted substantial interest due to their capability to offer a natural language interface for interpreting complex medical data. Here, we provide a comprehensive overview of MVLMs and the various medical tasks to which they have been applied. We also examine the datasets used for these tasks and compare the performance of different models based on standardized evaluation metrics.
  arXiv  Detail & Related papers  (2024-11-19T03:27:05Z)
• Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval [61.70489848327436]
  KARE is a novel framework that integrates knowledge graph (KG) community-level retrieval with large language models (LLMs) reasoning. Extensive experiments demonstrate that KARE outperforms leading models by up to 10.8-15.0% on MIMIC-III and 12.6-12.7% on MIMIC-IV for mortality and readmission predictions.
  arXiv  Detail & Related papers  (2024-10-06T18:46:28Z)
• The Role of Language Models in Modern Healthcare: A Comprehensive Review [2.048226951354646]
  The application of large language models (LLMs) in healthcare has gained significant attention. This review examines the trajectory of language models from their early stages to the current state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2024-09-25T12:15:15Z)
• From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practice [14.739357670600103]
  Large Language Models (LLMs) have rapidly evolved from text-based systems to multimodal platforms.<n>We examine the current landscape of MLLMs in healthcare, analyzing their applications across clinical decision support, medical imaging, patient engagement, and research.
  arXiv  Detail & Related papers  (2024-09-14T02:35:29Z)
• STLLaVA-Med: Self-Training Large Language and Vision Assistant for Medical Question-Answering [58.79671189792399]
  STLLaVA-Med is designed to train a policy model capable of auto-generating medical visual instruction data. We validate the efficacy and data efficiency of STLLaVA-Med across three major medical Visual Question Answering (VQA) benchmarks.
  arXiv  Detail & Related papers  (2024-06-28T15:01:23Z)
• A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions [23.36640449085249]
  We trace the recent advances of Medical Large Language Models (Med-LLMs)<n>The wide-ranging applications of Med-LLMs are investigated across various healthcare domains.<n>We discuss the challenges associated with ensuring fairness, accountability, privacy, and robustness.
  arXiv  Detail & Related papers  (2024-06-06T03:15:13Z)
• Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review [0.0]
  Medical vision-language models (VLMs) combine computer vision (CV) and natural language processing (NLP) to analyze medical data. Our paper reviews recent advancements in developing models designed for medical report generation and visual question answering.
  arXiv  Detail & Related papers  (2024-03-04T20:29:51Z)
• MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluation [110.31526448744096]
  We argue that unlocking this potential requires a systematic way to measure the performance of medical AI models on large-scale heterogeneous data. We are building MedPerf, an open framework for benchmarking machine learning in the medical domain.
  arXiv  Detail & Related papers  (2021-09-29T18:09:41Z)
• MIMO: Mutual Integration of Patient Journey and Medical Ontology for Healthcare Representation Learning [49.57261599776167]
  We propose an end-to-end robust Transformer-based solution, Mutual Integration of patient journey and Medical Ontology (MIMO) for healthcare representation learning and predictive analytics.
  arXiv  Detail & Related papers  (2021-07-20T07:04:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.