RudolfV: A Foundation Model by Pathologists for Pathologists

• URL: http://arxiv.org/abs/2401.04079v4
• Date: Tue, 11 Jun 2024 17:46:38 GMT
• Title: RudolfV: A Foundation Model by Pathologists for Pathologists
• Authors: Jonas Dippel, Barbara Feulner, Tobias Winterhoff, Timo Milbich, Stephan Tietz, Simon Schallenberg, Gabriel Dernbach, Andreas Kunft, Simon Heinke, Marie-Lisa Eich, Julika Ribbat-Idel, Rosemarie Krupar, Philipp Anders, Niklas Prenißl, Philipp Jurmeister, David Horst, Lukas Ruff, Klaus-Robert Müller, Frederick Klauschen, Maximilian Alber,
• Abstract summary: We present a novel approach to designing foundation models for computational pathology. Our model "RudolfV" surpasses existing state-of-the-art foundation models across different benchmarks.
• Score: 13.17203220753175
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Artificial intelligence has started to transform histopathology impacting clinical diagnostics and biomedical research. However, while many computational pathology approaches have been proposed, most current AI models are limited with respect to generalization, application variety, and handling rare diseases. Recent efforts introduced self-supervised foundation models to address these challenges, yet existing approaches do not leverage pathologist knowledge by design. In this study, we present a novel approach to designing foundation models for computational pathology, incorporating pathologist expertise, semi-automated data curation, and a diverse dataset from over 15 laboratories, including 58 tissue types, and encompassing 129 different histochemical and immunohistochemical staining modalities. We demonstrate that our model "RudolfV" surpasses existing state-of-the-art foundation models across different benchmarks focused on tumor microenvironment profiling, biomarker evaluation, and reference case search while exhibiting favorable robustness properties. Our study shows how domain-specific knowledge can increase the efficiency and performance of pathology foundation models and enable novel application areas.

Related papers

• CanvOI, an Oncology Intelligence Foundation Model: Scaling FLOPS Differently [0.0]
  We present CanvOI, a ViT-g/10-based foundation model designed to enhance the capabilities of digital pathology. By introducing larger tile sizes (380 x 380 pixels) and smaller patch sizes (10 x 10 pixels), we were able to optimize the model's performance.
  arXiv  Detail & Related papers  (2024-09-04T17:15:44Z)
• Benchmarking foundation models as feature extractors for weakly-supervised computational pathology [0.6151041580858937]
  We benchmarked ten histopathology foundation models on 13 patient cohorts with 6,791 patients and 9,493 slides from lung, colorectal, gastric, and breast cancers. We show that a vision-language foundation model, CONCH, yielded the highest performance in 42% of tasks when compared to vision-only foundation models.
  arXiv  Detail & Related papers  (2024-08-28T14:34:45Z)
• Pathology Foundation Models [0.0354287905099182]
  Development of deep learning technologies have led to extensive research and development in pathology AI (Artificial Intelligence) Large-scale AI models known as Foundation Models (FMs) have emerged and expanded their application scope in the healthcare field. FMs are more accurate and applicable to a wide range of tasks compared to traditional AI.
  arXiv  Detail & Related papers  (2024-07-31T03:58:48Z)
• A Clinical Benchmark of Public Self-Supervised Pathology Foundation Models [2.124312824026935]
  We present a collection of pathology datasets comprising clinical slides associated with clinically relevant endpoints including cancer diagnoses and a variety of biomarkers generated during standard hospital operation from two medical centers. We leverage these datasets to systematically assess the performance of public pathology foundation models and provide insights into best practices for training new foundation models and selecting appropriate pretrained models.
  arXiv  Detail & Related papers  (2024-07-09T02:33:13Z)
• Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports [51.45762396192655]
  Multimodal large language models (MLLMs) have recently transformed many domains, significantly affecting the medical field. Notably, Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models have epitomized a paradigm shift in Artificial General Intelligence for computer vision. This study evaluated the performance of the Gemini, GPT-4, and 4 popular large models for an exhaustive evaluation across 14 medical imaging datasets.
  arXiv  Detail & Related papers  (2024-07-08T09:08:42Z)
• A Survey of Artificial Intelligence in Gait-Based Neurodegenerative Disease Diagnosis [51.07114445705692]
  neurodegenerative diseases (NDs) traditionally require extensive healthcare resources and human effort for medical diagnosis and monitoring. As a crucial disease-related motor symptom, human gait can be exploited to characterize different NDs. The current advances in artificial intelligence (AI) models enable automatic gait analysis for NDs identification and classification.
  arXiv  Detail & Related papers  (2024-05-21T06:44:40Z)
• Knowledge-enhanced Visual-Language Pretraining for Computational Pathology [68.6831438330526]
  We consider the problem of visual representation learning for computational pathology, by exploiting large-scale image-text pairs gathered from public resources. We curate a pathology knowledge tree that consists of 50,470 informative attributes for 4,718 diseases requiring pathology diagnosis from 32 human tissues.
  arXiv  Detail & Related papers  (2024-04-15T17:11:25Z)
• Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation [113.5002649181103]
  Training open-source small multimodal models (SMMs) to bridge competency gaps for unmet clinical needs in radiology. For training, we assemble a large dataset of over 697 thousand radiology image-text pairs. For evaluation, we propose CheXprompt, a GPT-4-based metric for factuality evaluation, and demonstrate its parity with expert evaluation. The inference of LlaVA-Rad is fast and can be performed on a single V100 GPU in private settings, offering a promising state-of-the-art tool for real-world clinical applications.
  arXiv  Detail & Related papers  (2024-03-12T18:12:02Z)
• OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicine [55.29668193415034]
  We present OpenMEDLab, an open-source platform for multi-modality foundation models. It encapsulates solutions of pioneering attempts in prompting and fine-tuning large language and vision models for frontline clinical and bioinformatic applications. It opens access to a group of pre-trained foundation models for various medical image modalities, clinical text, protein engineering, etc.
  arXiv  Detail & Related papers  (2024-02-28T03:51:02Z)
• Domain-specific optimization and diverse evaluation of self-supervised models for histopathology [9.450129206898115]
  Task-specific deep learning models in histopathology offer promising opportunities for improving diagnosis, clinical research, and precision medicine. We describe the development and evaluation of foundation models for histopathology via self-supervised learning (SSL)
  arXiv  Detail & Related papers  (2023-10-20T03:38:07Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.