A Comprehensive Survey of Foundation Models in Medicine
- URL: http://arxiv.org/abs/2406.10729v1
- Date: Sat, 15 Jun 2024 20:04:06 GMT
- Title: A Comprehensive Survey of Foundation Models in Medicine
- Authors: Wasif Khan, Seowung Leem, Kyle B. See, Joshua K. Wong, Shaoting Zhang, Ruogu Fang,
- Abstract summary: Foundation models (FMs) are large-scale deep-learning models trained on extensive datasets using self-supervised techniques.
We focus on the history, learning strategies, flagship models, applications, and challenges of FMs in healthcare.
- Score: 8.879092631568263
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Foundation models (FMs) are large-scale deep-learning models trained on extensive datasets using self-supervised techniques. These models serve as a base for various downstream tasks, including healthcare. FMs have been adopted with great success across various domains within healthcare, including natural language processing (NLP), computer vision, graph learning, biology, and omics. Existing healthcare-based surveys have not yet included all of these domains. Therefore, this survey provides a comprehensive overview of FMs in healthcare. We focus on the history, learning strategies, flagship models, applications, and challenges of FMs. We explore how FMs such as the BERT and GPT families are reshaping various healthcare domains, including clinical large language models, medical image analysis, and omics data. Furthermore, we provide a detailed taxonomy of healthcare applications facilitated by FMs, such as clinical NLP, medical computer vision, graph learning, and other biology-related tasks. Despite the promising opportunities FMs provide, they also have several associated challenges, which are explained in detail. We also outline potential future directions to provide researchers and practitioners with insights into the potential and limitations of FMs in healthcare to advance their deployment and mitigate associated risks.
Related papers
- A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data [41.8344712915454]
Automatic radiology report generation can alleviate the workload for physicians and minimize regional disparities in medical resources.
It is a challenging task, as the computational model needs to mimic physicians to obtain information from multi-modal input data.
Recent works emerged to address this issue using deep learning-based methods, such as transformers, contrastive learning, and knowledge-base construction.
This survey summarizes the key techniques developed in the most recent works and proposes a general workflow for deep learning-based report generation.
arXiv Detail & Related papers (2024-05-21T14:37:35Z) - Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcare [14.399086205317358]
Foundation models (FMs) are trained on vast datasets through methods including unsupervised pretraining, self-supervised learning, instructed fine-tuning, and reinforcement learning from human feedback.
These models are crucial for biomedical applications that require processing diverse data forms such as clinical reports, diagnostic images, and multimodal patient interactions.
The incorporation of FL with these sophisticated models presents a promising strategy to harness their analytical power while safeguarding the privacy of sensitive medical data.
arXiv Detail & Related papers (2024-05-10T19:22:24Z) - Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directions [11.973160653486433]
Foundation model, which is pre-trained on broad data and is able to adapt to a wide range of tasks, is advancing healthcare.
Much more widespread healthcare scenarios will benefit from the development of a healthcare foundation model (HFM)
Despite the impending widespread deployment of HFMs, there is currently a lack of clear understanding about how they work in the healthcare field.
arXiv Detail & Related papers (2024-04-04T07:39:55Z) - Progress and Opportunities of Foundation Models in Bioinformatics [77.74411726471439]
Foundations models (FMs) have ushered in a new era in computational biology, especially in the realm of deep learning.
Central to our focus is the application of FMs to specific biological problems, aiming to guide the research community in choosing appropriate FMs for their research needs.
Review analyses challenges and limitations faced by FMs in biology, such as data noise, model explainability, and potential biases.
arXiv Detail & Related papers (2024-02-06T02:29:17Z) - Learn From Model Beyond Fine-Tuning: A Survey [78.80920533793595]
Learn From Model (LFM) focuses on the research, modification, and design of foundation models (FM) based on the model interface.
The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing.
This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM.
arXiv Detail & Related papers (2023-10-12T10:20:36Z) - Towards Generalist Foundation Model for Radiology by Leveraging
Web-scale 2D&3D Medical Data [66.9359934608229]
This study aims to initiate the development of Radiology Foundation Model, termed as RadFM.
To the best of our knowledge, this is the first large-scale, high-quality, medical visual-language dataset, with both 2D and 3D scans.
We propose a new evaluation benchmark, RadBench, that comprises five tasks, including modality recognition, disease diagnosis, visual question answering, report generation and rationale diagnosis.
arXiv Detail & Related papers (2023-08-04T17:00:38Z) - Radiology-GPT: A Large Language Model for Radiology [74.07944784968372]
We introduce Radiology-GPT, a large language model for radiology.
It demonstrates superior performance compared to general language models such as StableLM, Dolly and LLaMA.
It exhibits significant versatility in radiological diagnosis, research, and communication.
arXiv Detail & Related papers (2023-06-14T17:57:24Z) - XrayGPT: Chest Radiographs Summarization using Medical Vision-Language
Models [60.437091462613544]
We introduce XrayGPT, a novel conversational medical vision-language model.
It can analyze and answer open-ended questions about chest radiographs.
We generate 217k interactive and high-quality summaries from free-text radiology reports.
arXiv Detail & Related papers (2023-06-13T17:59:59Z) - Privacy-preserving machine learning for healthcare: open challenges and
future perspectives [72.43506759789861]
We conduct a review of recent literature concerning Privacy-Preserving Machine Learning (PPML) for healthcare.
We primarily focus on privacy-preserving training and inference-as-a-service.
The aim of this review is to guide the development of private and efficient ML models in healthcare.
arXiv Detail & Related papers (2023-03-27T19:20:51Z) - A Survey on Incorporating Domain Knowledge into Deep Learning for
Medical Image Analysis [38.90186125141749]
Small size of medical datasets remains a major bottleneck in deep learning.
Traditional approaches leverage the information from natural images via transfer learning.
More recent works utilize the domain knowledge from medical doctors to create networks that resemble how medical doctors are trained.
arXiv Detail & Related papers (2020-04-25T14:27:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.