PathBench: A comprehensive comparison benchmark for pathology foundation models towards precision oncology

• URL: http://arxiv.org/abs/2505.20202v1
• Date: Mon, 26 May 2025 16:42:22 GMT
• Title: PathBench: A comprehensive comparison benchmark for pathology foundation models towards precision oncology
• Authors: Jiabo Ma, Yingxue Xu, Fengtao Zhou, Yihui Wang, Cheng Jin, Zhengrui Guo, Jianfeng Wu, On Ki Tang, Huajun Zhou, Xi Wang, Luyang Luo, Zhengyu Zhang, Du Cai, Zizhao Gao, Wei Wang, Yueping Liu, Jiankun He, Jing Cui, Zhenhui Li, Jing Zhang, Feng Gao, Xiuming Zhang, Li Liang, Ronald Cheong Kin Chan, Zhe Wang, Hao Chen,
• Abstract summary: We present PathBench, the first comprehensive benchmark for pathology foundation models (PFMs)<n>Our framework incorporates large-scale data, enabling objective comparison of PFMs.<n>We have collected 15,888 WSIs from 8,549 patients across 10 hospitals, encompassing over 64 diagnosis and prognosis tasks.
• Score: 33.51485504161335
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The emergence of pathology foundation models has revolutionized computational histopathology, enabling highly accurate, generalized whole-slide image analysis for improved cancer diagnosis, and prognosis assessment. While these models show remarkable potential across cancer diagnostics and prognostics, their clinical translation faces critical challenges including variability in optimal model across cancer types, potential data leakage in evaluation, and lack of standardized benchmarks. Without rigorous, unbiased evaluation, even the most advanced PFMs risk remaining confined to research settings, delaying their life-saving applications. Existing benchmarking efforts remain limited by narrow cancer-type focus, potential pretraining data overlaps, or incomplete task coverage. We present PathBench, the first comprehensive benchmark addressing these gaps through: multi-center in-hourse datasets spanning common cancers with rigorous leakage prevention, evaluation across the full clinical spectrum from diagnosis to prognosis, and an automated leaderboard system for continuous model assessment. Our framework incorporates large-scale data, enabling objective comparison of PFMs while reflecting real-world clinical complexity. All evaluation data comes from private medical providers, with strict exclusion of any pretraining usage to avoid data leakage risks. We have collected 15,888 WSIs from 8,549 patients across 10 hospitals, encompassing over 64 diagnosis and prognosis tasks. Currently, our evaluation of 19 PFMs shows that Virchow2 and H-Optimus-1 are the most effective models overall. This work provides researchers with a robust platform for model development and offers clinicians actionable insights into PFM performance across diverse clinical scenarios, ultimately accelerating the translation of these transformative technologies into routine pathology practice.

Related papers

• A High Magnifications Histopathology Image Dataset for Oral Squamous Cell Carcinoma Diagnosis and Prognosis [18.549808005574985]
  Multi-OSCC is a new histopathology image dataset comprising 1,325 Oral Squamous Cell Carcinoma patients.<n>Each patient is represented by six high resolution histopathology images captured at x200, x400, and x1000 magnifications-two per magnification-covering both the core and edge tumor regions.<n>The dataset is richly annotated for six critical clinical tasks: recurrence prediction (REC), lymph node metastasis (LNM), tumor differentiation (TD), tumor invasion (TI) and perineural invasion (PI)
  arXiv  Detail & Related papers  (2025-07-22T08:48:45Z)
• Benchmarking Foundation Models and Parameter-Efficient Fine-Tuning for Prognosis Prediction in Medical Imaging [26.589728923739596]
  We evaluate and compare the transferability of Convolutional Neural Networks and Foundation Models in predicting clinical outcomes in COVID-19 patients.<n>The evaluations were conducted across multiple learning paradigms, including both extensive full-data scenarios and more clinically realistic Few-Shot Learning settings.
  arXiv  Detail & Related papers  (2025-06-23T09:16:04Z)
• Zero-shot Medical Event Prediction Using a Generative Pre-trained Transformer on Electronic Health Records [8.575985305475355]
  Generative pre-trained transformers (GPT) can leverage Longitudinal Data in EHRs to predict future events.<n> fine-tuning of these models can enhance task-specific performance, but it becomes costly when applied to many clinical prediction tasks.<n>A pretrained foundation model can be used in zero-shot forecasting setting, offering a scalable alternative to fine-tuning separate models for each outcome.
  arXiv  Detail & Related papers  (2025-03-07T19:26:47Z)
• Continually Evolved Multimodal Foundation Models for Cancer Prognosis [50.43145292874533]
  Cancer prognosis is a critical task that involves predicting patient outcomes and survival rates.<n>Previous studies have integrated diverse data modalities, such as clinical notes, medical images, and genomic data, leveraging their complementary information.<n>Existing approaches face two major limitations. First, they struggle to incorporate newly arrived data with varying distributions into training, such as patient records from different hospitals.<n>Second, most multimodal integration methods rely on simplistic concatenation or task-specific pipelines, which fail to capture the complex interdependencies across modalities.
  arXiv  Detail & Related papers  (2025-01-30T06:49:57Z)
• Prediction of Lung Metastasis from Hepatocellular Carcinoma using the SEER Database [0.9055332067000195]
  Hepatocellular carcinoma (HCC) is a leading cause of cancer-related mortality.<n> predictive models for lung metastasis inHCC remain limited in scope and clinical applicability.<n>We develop and validate an end-to-end machine learning pipeline using data from the Surveillance, Epidemiology, and End Results (SEER) database.
  arXiv  Detail & Related papers  (2025-01-20T20:06:31Z)
• Enhancing End Stage Renal Disease Outcome Prediction: A Multi-Sourced Data-Driven Approach [7.212939068975618]
  We utilized data about 10,326 CKD patients, combining their clinical and claims information from 2009 to 2018.<n>A 24-month observation window was identified as optimal for balancing early detection and prediction accuracy.<n>The 2021 eGFR equation improved prediction accuracy and reduced racial bias, notably for African American patients.
  arXiv  Detail & Related papers  (2024-10-02T03:21:01Z)
• Enhancing clinical decision support with physiological waveforms -- a multimodal benchmark in emergency care [0.9503773054285559]
  We present a dataset and benchmarking protocol designed to advance multimodal decision support in emergency care.<n>Our models utilize demographics, biometrics, vital signs, laboratory values, and electrocardiogram (ECG) waveforms as inputs to predict both discharge diagnoses and patient deterioration.
  arXiv  Detail & Related papers  (2024-07-25T08:21:46Z)
• Machine Learning for ALSFRS-R Score Prediction: Making Sense of the Sensor Data [44.99833362998488]
  Amyotrophic Lateral Sclerosis (ALS) is a rapidly progressive neurodegenerative disease that presents individuals with limited treatment options. The present investigation, spearheaded by the iDPP@CLEF 2024 challenge, focuses on utilizing sensor-derived data obtained through an app.
  arXiv  Detail & Related papers  (2024-07-10T19:17:23Z)
• Foresight -- Deep Generative Modelling of Patient Timelines using Electronic Health Records [46.024501445093755]
  Temporal modelling of medical history can be used to forecast and simulate future events, estimate risk, suggest alternative diagnoses or forecast complications. We present Foresight, a novel GPT3-based pipeline that uses NER+L tools (i.e. MedCAT) to convert document text into structured, coded concepts.
  arXiv  Detail & Related papers  (2022-12-13T19:06:00Z)
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data [81.00385374948125]
  We present UNcertaInTy-based hEalth risk prediction (UNITE) model. UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data. We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD) UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
  arXiv  Detail & Related papers  (2020-10-22T02:28:11Z)
• Hemogram Data as a Tool for Decision-making in COVID-19 Management: Applications to Resource Scarcity Scenarios [62.997667081978825]
  COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure. This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients. Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
  arXiv  Detail & Related papers  (2020-05-10T01:45:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.