PhenoBench: Mapping What a Deeply Phenotyped Human Cohort Can Tell Us

• URL: http://arxiv.org/abs/2609.06080v1
• Date: Sat, 05 Sep 2026 13:15:40 GMT
• Title: PhenoBench: Mapping What a Deeply Phenotyped Human Cohort Can Tell Us
• Abstract summary: We present PhenoBench, an executable benchmark built around the Human Phenotype Project.<n>The benchmark defines 90 clinically grounded tasks across 15 domains and 26 input modalities.<n> Measurements showed question- and representation-dependent predictive value, including positive, near-zero, and negative changes in held-out performance.
• Score: 0.7084863262134483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deeply phenotyped cohorts combine clinical, imaging, molecular, and wearable observations across timescales from seconds to years. This breadth can reveal which measurements inform which health-related questions, but heterogeneous analyses are not directly comparable. We present PhenoBench, an executable benchmark built around the Human Phenotype Project, in which more than 13,000 participants have completed the initial visit. Each question fixes the target, eligible population, timing, and allowed information; its evaluation contract specifies the split, metric, baseline, and claim boundary. The benchmark defines 90 clinically grounded tasks across 15 domains and 26 input modalities. Measurements showed question- and representation-dependent predictive value, including positive, near-zero, and negative changes in held-out performance relative to matched baselines. We used PhenoBench to evaluate emerging tabular foundation models across 160 matched regression comparisons spanning 52 tasks. These models ranked above standard task-specific models in aggregate but, averaged across the three pretrained models within each cell, improved on ridge by a median of only 0.004 $R^2$ (95% CI, 0.002--0.006). We then used the same cohort data and evaluation contracts to evaluate 14 language models, collectively covering 40 tasks spanning phenotype recovery, classification, follow-up forecasting, and participant ordering. Without cohort-specific fitting, language models made informative predictions on some tasks, but showed task-specific capability gaps, shared failures of scale, and rarely surpassed models fitted on the same fields. PhenoBench turns a multimodal longitudinal cohort into a versioned, auditable evaluation system where new questions, measurements, and models can be added without redefining existing comparisons.

Related papers

• Simulating clinical interventions with a generative multimodal model of human physiology [49.828582089394445]
  HealthFormer is a decoder-only transformer that models the human physiological trajectory generatively.<n>We tokenise each participant's health trajectory across 667 measurements spanning seven domains.<n>HealthFormer improves prediction for 27 of 30 incident-disease and mortality endpoints, exceeding established clinical risk scores in every comparison.
  arXiv  Detail & Related papers  (2026-04-30T14:10:30Z)
• A Nationwide Japanese Medical Claims Foundation Model: Balancing Model Scaling and Task-Specific Computational Efficiency [2.107969466194361]
  Self-supervised foundation models have emerged as a promising approach for leveraging large-scale unlabeled healthcare records.<n>In natural language processing, scaling laws suggest that larger models achieve predictably lower pretraining losses.<n>We evaluated the relationship between model scale and downstream task performance for structured medical foundation models.
  arXiv  Detail & Related papers  (2026-04-24T08:32:47Z)
• OmniTabBench: Mapping the Empirical Frontiers of GBDTs, Neural Networks, and Foundation Models for Tabular Data at Scale [82.32822441887886]
  We introduce OmniTabBench, the largest benchmark to date, comprising 3030 datasets spanning diverse tasks.<n>We conduct an unprecedented large-scale empirical evaluation of state-of-the-art models from all model families on OmniTabBench, confirming the absence of a dominant winner.
  arXiv  Detail & Related papers  (2026-04-08T08:31:43Z)
• AdaptStress: Online Adaptive Learning for Interpretable and Personalized Stress Prediction Using Multivariate and Sparse Physiological Signals [1.593065406609169]
  This paper presents a novel, explainable, and individualized approach for stress prediction using physiological data from consumer-grade smartwatches.<n>We develop a time series forecasting model that leverages multivariate features, including heart rate variability, activity patterns, and sleep metrics, to predict stress levels.<n>Our model achieved performance with an MSE of 0.053, MAE of 0.190, and RMSE of 0.226 in optimal settings.
  arXiv  Detail & Related papers  (2026-02-19T20:57:35Z)
• GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation [90.53485251837235]
  Time series foundation models excel in zero-shot forecasting, handling diverse tasks without explicit training. GIFT-Eval is a pioneering benchmark aimed at promoting evaluation across diverse datasets. GIFT-Eval encompasses 23 datasets over 144,000 time series and 177 million data points.
  arXiv  Detail & Related papers  (2024-10-14T11:29:38Z)
• Benchmarking foundation models as feature extractors for weakly-supervised computational pathology [0.6151041580858937]
  We benchmarked 19 histopathology foundation models on 13 patient cohorts with 6,818 patients and 9,528 slides from lung, colorectal, gastric, and breast cancers.<n>We show that a vision-language foundation model, CONCH, yielded the highest performance when compared to vision-only foundation models, with Virchow2 as close second.
  arXiv  Detail & Related papers  (2024-08-28T14:34:45Z)
• Using Pre-training and Interaction Modeling for ancestry-specific disease prediction in UK Biobank [69.90493129893112]
  Recent genome-wide association studies (GWAS) have uncovered the genetic basis of complex traits, but show an under-representation of non-European descent individuals. Here, we assess whether we can improve disease prediction across diverse ancestries using multiomic data.
  arXiv  Detail & Related papers  (2024-04-26T16:39:50Z)
• Estimating Structural Disparities for Face Models [54.062512989859265]
  In machine learning, disparity metrics are often defined by measuring the difference in the performance or outcome of a model, across different sub-populations. We explore performing such analysis on computer vision models trained on human faces, and on tasks such as face attribute prediction and affect estimation.
  arXiv  Detail & Related papers  (2022-04-13T05:30:53Z)
• CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation [107.63407690972139]
  Conditional Score-based Diffusion models for Imputation (CSDI) is a novel time series imputation method that utilizes score-based diffusion models conditioned on observed data. CSDI improves by 40-70% over existing probabilistic imputation methods on popular performance metrics. In addition, C reduces the error by 5-20% compared to the state-of-the-art deterministic imputation methods.
  arXiv  Detail & Related papers  (2021-07-07T22:20:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.