A Comprehensive Survey of Foundation Models in Medicine

• URL: http://arxiv.org/abs/2406.10729v3
• Date: Thu, 16 Jan 2025 16:04:07 GMT
• Title: A Comprehensive Survey of Foundation Models in Medicine
• Authors: Wasif Khan, Seowung Leem, Kyle B. See, Joshua K. Wong, Shaoting Zhang, Ruogu Fang,
• Abstract summary: Foundation models (FMs) are large-scale deep learning models trained on massive datasets.<n>We present a review of FMs in medicine, focusing on their evolution, learning strategies, flagship models, applications, and associated challenges.<n>We provide a detailed taxonomy of FM-enabled healthcare applications, spanning clinical natural language processing, medical computer vision, graph learning, and other biology- and omics- related tasks.
• Score: 8.879092631568263
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Foundation models (FMs) are large-scale deep learning models trained on massive datasets, often using self-supervised learning techniques. These models serve as a versatile base for a wide range of downstream tasks, including those in medicine and healthcare. FMs have demonstrated remarkable success across multiple healthcare domains. However, existing surveys in this field do not comprehensively cover all areas where FMs have made significant strides. In this survey, we present a comprehensive review of FMs in medicine, focusing on their evolution, learning strategies, flagship models, applications, and associated challenges. We examine how prominent FMs, such as the BERT and GPT families, are transforming various aspects of healthcare, including clinical large language models, medical image analysis, and omics research. Additionally, we provide a detailed taxonomy of FM-enabled healthcare applications, spanning clinical natural language processing, medical computer vision, graph learning, and other biology- and omics- related tasks. Despite the transformative potentials of FMs, they also pose unique challenges. This survey delves into these challenges and highlights open research questions and lessons learned to guide researchers and practitioners. Our goal is to provide valuable insights into the capabilities of FMs in health, facilitating responsible deployment and mitigating associated risks.

Related papers

• Structured Outputs Enable General-Purpose LLMs to be Medical Experts [50.02627258858336]
  Large language models (LLMs) often struggle with open-ended medical questions. We propose a novel approach utilizing structured medical reasoning. Our approach achieves the highest Factuality Score of 85.8, surpassing fine-tuned models.
  arXiv  Detail & Related papers  (2025-03-05T05:24:55Z)
• Biomedical Foundation Model: A Survey [84.26268124754792]
  Foundation models are large-scale pre-trained models that learn from extensive unlabeled datasets. These models can be adapted to various applications such as question answering and visual understanding. This survey explores the potential of foundation models across diverse domains within biomedical fields.
  arXiv  Detail & Related papers  (2025-03-03T22:42:00Z)
• A Survey of Medical Vision-and-Language Applications and Their Techniques [48.268198631277315]
  Medical vision-and-language models (MVLMs) have attracted substantial interest due to their capability to offer a natural language interface for interpreting complex medical data. Here, we provide a comprehensive overview of MVLMs and the various medical tasks to which they have been applied. We also examine the datasets used for these tasks and compare the performance of different models based on standardized evaluation metrics.
  arXiv  Detail & Related papers  (2024-11-19T03:27:05Z)
• Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcare [14.399086205317358]
  Foundation models (FMs) are trained on vast datasets through methods including unsupervised pretraining, self-supervised learning, instructed fine-tuning, and reinforcement learning from human feedback. These models are crucial for biomedical applications that require processing diverse data forms such as clinical reports, diagnostic images, and multimodal patient interactions. The incorporation of FL with these sophisticated models presents a promising strategy to harness their analytical power while safeguarding the privacy of sensitive medical data.
  arXiv  Detail & Related papers  (2024-05-10T19:22:24Z)
• Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directions [11.973160653486433]
  Foundation model, which is pre-trained on broad data and is able to adapt to a wide range of tasks, is advancing healthcare. Much more widespread healthcare scenarios will benefit from the development of a healthcare foundation model (HFM) Despite the impending widespread deployment of HFMs, there is currently a lack of clear understanding about how they work in the healthcare field.
  arXiv  Detail & Related papers  (2024-04-04T07:39:55Z)
• Progress and Opportunities of Foundation Models in Bioinformatics [77.74411726471439]
  Foundations models (FMs) have ushered in a new era in computational biology, especially in the realm of deep learning. Central to our focus is the application of FMs to specific biological problems, aiming to guide the research community in choosing appropriate FMs for their research needs. Review analyses challenges and limitations faced by FMs in biology, such as data noise, model explainability, and potential biases.
  arXiv  Detail & Related papers  (2024-02-06T02:29:17Z)
• Learn From Model Beyond Fine-Tuning: A Survey [78.80920533793595]
  Learn From Model (LFM) focuses on the research, modification, and design of foundation models (FM) based on the model interface. The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing. This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM.
  arXiv  Detail & Related papers  (2023-10-12T10:20:36Z)
• Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data [66.9359934608229]
  This study aims to initiate the development of Radiology Foundation Model, termed as RadFM. To the best of our knowledge, this is the first large-scale, high-quality, medical visual-language dataset, with both 2D and 3D scans. We propose a new evaluation benchmark, RadBench, that comprises five tasks, including modality recognition, disease diagnosis, visual question answering, report generation and rationale diagnosis.
  arXiv  Detail & Related papers  (2023-08-04T17:00:38Z)
• Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health [22.858424132819795]
  ChatGPT has led to the emergence of diverse applications in the field of biomedicine and health. We explore the areas of biomedical information retrieval, question answering, medical text summarization, and medical education. We find that significant advances have been made in the field of text generation tasks, surpassing the previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-15T20:19:08Z)
• XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models [60.437091462613544]
  We introduce XrayGPT, a novel conversational medical vision-language model. It can analyze and answer open-ended questions about chest radiographs. We generate 217k interactive and high-quality summaries from free-text radiology reports.
  arXiv  Detail & Related papers  (2023-06-13T17:59:59Z)
• Artificial General Intelligence for Medical Imaging Analysis [92.3940918983821]
  Large-scale Artificial General Intelligence (AGI) models have achieved unprecedented success in a variety of general domain tasks. These models face notable challenges arising from the medical field's inherent complexities and unique characteristics. This review aims to offer insights into the future implications of AGI in medical imaging, healthcare, and beyond.
  arXiv  Detail & Related papers  (2023-06-08T18:04:13Z)
• Towards Medical Artificial General Intelligence via Knowledge-Enhanced Multimodal Pretraining [121.89793208683625]
  Medical artificial general intelligence (MAGI) enables one foundation model to solve different medical tasks. We propose a new paradigm called Medical-knedge-enhanced mulTimOdal pretRaining (MOTOR)
  arXiv  Detail & Related papers  (2023-04-26T01:26:19Z)
• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis [38.90186125141749]
  Small size of medical datasets remains a major bottleneck in deep learning. Traditional approaches leverage the information from natural images via transfer learning. More recent works utilize the domain knowledge from medical doctors to create networks that resemble how medical doctors are trained.
  arXiv  Detail & Related papers  (2020-04-25T14:27:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.