A Comprehensive Survey of Foundation Models in Medicine

• URL: http://arxiv.org/abs/2406.10729v3
• Date: Thu, 16 Jan 2025 16:04:07 GMT
• Title: A Comprehensive Survey of Foundation Models in Medicine
• Authors: Wasif Khan, Seowung Leem, Kyle B. See, Joshua K. Wong, Shaoting Zhang, Ruogu Fang,
• Abstract summary: Foundation models (FMs) are large-scale deep learning models trained on massive datasets. We present a review of FMs in medicine, focusing on their evolution, learning strategies, flagship models, applications, and associated challenges. We provide a detailed taxonomy of FM-enabled healthcare applications, spanning clinical natural language processing, medical computer vision, graph learning, and other biology- and omics- related tasks.
• Score: 8.879092631568263
• License:
• Abstract: Foundation models (FMs) are large-scale deep learning models trained on massive datasets, often using self-supervised learning techniques. These models serve as a versatile base for a wide range of downstream tasks, including those in medicine and healthcare. FMs have demonstrated remarkable success across multiple healthcare domains. However, existing surveys in this field do not comprehensively cover all areas where FMs have made significant strides. In this survey, we present a comprehensive review of FMs in medicine, focusing on their evolution, learning strategies, flagship models, applications, and associated challenges. We examine how prominent FMs, such as the BERT and GPT families, are transforming various aspects of healthcare, including clinical large language models, medical image analysis, and omics research. Additionally, we provide a detailed taxonomy of FM-enabled healthcare applications, spanning clinical natural language processing, medical computer vision, graph learning, and other biology- and omics- related tasks. Despite the transformative potentials of FMs, they also pose unique challenges. This survey delves into these challenges and highlights open research questions and lessons learned to guide researchers and practitioners. Our goal is to provide valuable insights into the capabilities of FMs in health, facilitating responsible deployment and mitigating associated risks.

Related papers

• A Survey of Medical Vision-and-Language Applications and Their Techniques [48.268198631277315]
  Medical vision-and-language models (MVLMs) have attracted substantial interest due to their capability to offer a natural language interface for interpreting complex medical data. Here, we provide a comprehensive overview of MVLMs and the various medical tasks to which they have been applied. We also examine the datasets used for these tasks and compare the performance of different models based on standardized evaluation metrics.
  arXiv  Detail & Related papers  (2024-11-19T03:27:05Z)
• Foundation Models for Remote Sensing and Earth Observation: A Survey [101.77425018347557]
  This survey systematically reviews the emerging field of Remote Sensing Foundation Models (RSFMs) It begins with an outline of their motivation and background, followed by an introduction of their foundational concepts. We benchmark these models against publicly available datasets, discuss existing challenges, and propose future research directions.
  arXiv  Detail & Related papers  (2024-10-22T01:08:21Z)
• A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data [41.8344712915454]
  Automatic radiology report generation can alleviate the workload for physicians and minimize regional disparities in medical resources. It is a challenging task, as the computational model needs to mimic physicians to obtain information from multi-modal input data. Recent works emerged to address this issue using deep learning-based methods, such as transformers, contrastive learning, and knowledge-base construction. This survey summarizes the key techniques developed in the most recent works and proposes a general workflow for deep learning-based report generation.
  arXiv  Detail & Related papers  (2024-05-21T14:37:35Z)
• Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcare [14.399086205317358]
  Foundation models (FMs) are trained on vast datasets through methods including unsupervised pretraining, self-supervised learning, instructed fine-tuning, and reinforcement learning from human feedback. These models are crucial for biomedical applications that require processing diverse data forms such as clinical reports, diagnostic images, and multimodal patient interactions. The incorporation of FL with these sophisticated models presents a promising strategy to harness their analytical power while safeguarding the privacy of sensitive medical data.
  arXiv  Detail & Related papers  (2024-05-10T19:22:24Z)
• Progress and Opportunities of Foundation Models in Bioinformatics [77.74411726471439]
  Foundations models (FMs) have ushered in a new era in computational biology, especially in the realm of deep learning. Central to our focus is the application of FMs to specific biological problems, aiming to guide the research community in choosing appropriate FMs for their research needs. Review analyses challenges and limitations faced by FMs in biology, such as data noise, model explainability, and potential biases.
  arXiv  Detail & Related papers  (2024-02-06T02:29:17Z)
• Learning from models beyond fine-tuning [78.20895343699658]
  Learn From Model (LFM) focuses on the research, modification, and design of foundation models (FM) based on the model interface. The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing. This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM.
  arXiv  Detail & Related papers  (2023-10-12T10:20:36Z)
• Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health [22.858424132819795]
  ChatGPT has led to the emergence of diverse applications in the field of biomedicine and health. We explore the areas of biomedical information retrieval, question answering, medical text summarization, and medical education. We find that significant advances have been made in the field of text generation tasks, surpassing the previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-15T20:19:08Z)
• Artificial General Intelligence for Medical Imaging Analysis [92.3940918983821]
  Large-scale Artificial General Intelligence (AGI) models have achieved unprecedented success in a variety of general domain tasks. These models face notable challenges arising from the medical field's inherent complexities and unique characteristics. This review aims to offer insights into the future implications of AGI in medical imaging, healthcare, and beyond.
  arXiv  Detail & Related papers  (2023-06-08T18:04:13Z)
• Towards Medical Artificial General Intelligence via Knowledge-Enhanced Multimodal Pretraining [121.89793208683625]
  Medical artificial general intelligence (MAGI) enables one foundation model to solve different medical tasks. We propose a new paradigm called Medical-knedge-enhanced mulTimOdal pretRaining (MOTOR)
  arXiv  Detail & Related papers  (2023-04-26T01:26:19Z)
• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis [38.90186125141749]
  Small size of medical datasets remains a major bottleneck in deep learning. Traditional approaches leverage the information from natural images via transfer learning. More recent works utilize the domain knowledge from medical doctors to create networks that resemble how medical doctors are trained.
  arXiv  Detail & Related papers  (2020-04-25T14:27:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.