Enabling Granular Subgroup Level Model Evaluations by Generating Synthetic Medical Time Series

• URL: http://arxiv.org/abs/2510.19728v1
• Date: Wed, 22 Oct 2025 16:17:29 GMT
• Title: Enabling Granular Subgroup Level Model Evaluations by Generating Synthetic Medical Time Series
• Abstract summary: We introduce textitEnhanced TimeAutoDiff, which augments the latent diffusion objective with distribution-alignment penalties.<n>We benchmark all models on MIMIC-III and eICU, on 24-hour mortality and binary length-of-stay tasks.<n>For 32 intersectional subgroups, large synthetic cohorts cut-level AUROC estimation error by up to 50%.
• Score: 3.5347586732342435
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel framework for leveraging synthetic ICU time-series data not only to train but also to rigorously and trustworthily evaluate predictive models, both at the population level and within fine-grained demographic subgroups. Building on prior diffusion and VAE-based generators (TimeDiff, HealthGen, TimeAutoDiff), we introduce \textit{Enhanced TimeAutoDiff}, which augments the latent diffusion objective with distribution-alignment penalties. We extensively benchmark all models on MIMIC-III and eICU, on 24-hour mortality and binary length-of-stay tasks. Our results show that Enhanced TimeAutoDiff reduces the gap between real-on-synthetic and real-on-real evaluation (``TRTS gap'') by over 70\%, achieving $\Delta_{TRTS} \leq 0.014$ AUROC, while preserving training utility ($\Delta_{TSTR} \approx 0.01$). Crucially, for 32 intersectional subgroups, large synthetic cohorts cut subgroup-level AUROC estimation error by up to 50\% relative to small real test sets, and outperform them in 72--84\% of subgroups. This work provides a practical, privacy-preserving roadmap for trustworthy, granular model evaluation in critical care, enabling robust and reliable performance analysis across diverse patient populations without exposing sensitive EHR data, contributing to the overall trustworthiness of Medical AI.

Related papers

• MedFlow: Class-Aware Multi-Scale Generation for Medical Time-Series Synthesis [61.211117385840396]
  We propose MedFlow, a class-aware flow matching framework for medical time-series synthesis.<n>MedFlow employs a vector-quantized multi-scale tokenizer to represent medical sequences at complementary temporal resolutions.<n> Token Marginal Guidance, which incorporates class-conditional token statistics directly into the flow matching process, steers generation toward class-specific regions.
  arXiv  Detail & Related papers  (2026-09-04T07:02:12Z)
• Evaluating Deep Multivariate Imputation Models on Wearable Device Data [1.9474251417198067]
  Deep imputation methods such as BRITS and SAITS have seen limited evaluation on multimodal physiological data under realistic missingness.<n>We develop an evaluation protocol that mines contiguous missing-run templates from training data, stratifies them by per-feature gap-length quantiles, and injects them as block masks with preserved co-missingness structure.<n>A matched training protocol exposing models to the same missingness distribution reduces BRITS's severe-gap MAE by 43%.
  arXiv  Detail & Related papers  (2026-08-25T11:48:12Z)
• Zero-Shot Heart Rate Variability Forecasting from Consumer Wearables Using Time Series Foundation Models [5.572607261284016]
  Short-term Heart Rate Variability (HRV) forecasting could provide clinicians with actionable lead time for detecting autonomic dysfunction and adverse cardiac events.<n> Consumer wearable devices generate fragmented, artifact-rich HRV signals that challenge conventional forecasting approaches.<n>In this study, we evaluated the forecasting ability of three Time Series Foundation Models (TSFMs), TimesFM, Chronos, and MOIRAI.
  arXiv  Detail & Related papers  (2026-07-22T11:12:07Z)
• TimeLAVA: Learning-Agnostic Data Valuation for Time Series [48.76711740818533]
  TimeLAVA is a learning-agnostic framework that values temporal segments by their marginal contribution to minimize distributional discrepancy between evaluated and reference data.<n>We provide theoretical guarantees linking valuation to model-agnostic generalization and prove bounded sensitivity to contamination.
  arXiv  Detail & Related papers  (2026-06-17T06:16:16Z)
• Towards Brain MRI Foundation Models for the Clinic: Findings from the FOMO25 Challenge [33.95846433611413]
  We organized the FOMO25 challenge as a satellite event at MICCAI 2025.<n>We provided participants with a large pretraining dataset, FOMO60K, and evaluated models on data sourced directly from clinical labels.<n>19 foundation models from sixteen teams were evaluated using a standardized containerized pipeline.
  arXiv  Detail & Related papers  (2026-04-13T16:23:46Z)
• TarDiff: Target-Oriented Diffusion Guidance for Synthetic Electronic Health Record Time Series Generation [26.116599951658454]
  Time-series generation is crucial for advancing clinical machine learning models.<n>We argue that fidelity to observed data alone does not guarantee better model performance.<n>We propose TarDiff, a novel target-oriented diffusion framework that integrates task-specific influence guidance.
  arXiv  Detail & Related papers  (2025-04-24T14:36:10Z)
• A Scalable k-Medoids Clustering via Whale Optimization Algorithm [0.0]
  We introduce WOA-kMedoids, a novel unsupervised clustering method that incorporates the Whale Optimization Algorithm (WOA)<n>By optimizing the centroid selection, WOA-kMedoids reduces the computational complexity from quadratic to near-linear with respect to the number of observations.<n>We show that WOA-kMedoids achieved a clustering performance comparable to PAM, with an average Rand Index (RI) of 0.731 compared to PAM's 0.739, outperforming PAM on 12 out of 25 datasets.
  arXiv  Detail & Related papers  (2024-08-30T03:43:37Z)
• Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution [67.9215891673174]
  We propose score entropy as a novel loss that naturally extends score matching to discrete spaces. We test our Score Entropy Discrete Diffusion models on standard language modeling tasks.
  arXiv  Detail & Related papers  (2023-10-25T17:59:12Z)
• GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models [60.48306899271866]
  We present a new framework, called GREAT Score, for global robustness evaluation of adversarial perturbation using generative models. We show high correlation and significantly reduced cost of GREAT Score when compared to the attack-based model ranking on RobustBench. GREAT Score can be used for remote auditing of privacy-sensitive black-box models.
  arXiv  Detail & Related papers  (2023-04-19T14:58:27Z)
• Revisiting the Evaluation of Image Synthesis with GANs [55.72247435112475]
  This study presents an empirical investigation into the evaluation of synthesis performance, with generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses of various factors, including how to represent a data point in the representation space, how to calculate a fair distance using selected samples, and how many instances to use from each set.
  arXiv  Detail & Related papers  (2023-04-04T17:54:32Z)
• Robustness Analysis of Deep Learning Models for Population Synthesis [5.9106199000537645]
  We present bootstrap confidence interval for the deep generative models to evaluate robustness to multiple datasets. The models are implemented on multiple travel diaries of Montreal Origin- Destination Survey of 2008, 2013, and 2018. Results show that the predictive errors of CTGAN have narrower confidence intervals indicating its robustness to multiple datasets.
  arXiv  Detail & Related papers  (2022-11-23T22:55:55Z)
• Generating Synthetic Mixed-type Longitudinal Electronic Health Records for Artificial Intelligent Applications [9.374416143268892]
  generative adversarial network (GAN) entitled EHR-M-GAN which synthesizes textitmixed-type timeseries EHR data. We have validated EHR-M-GAN on three publicly-available intensive care unit databases with records from a total of 141,488 unique patients.
  arXiv  Detail & Related papers  (2021-12-22T17:17:34Z)
• SANSformers: Self-Supervised Forecasting in Electronic Health Records with Attention-Free Models [48.07469930813923]
  This work aims to forecast the demand for healthcare services, by predicting the number of patient visits to healthcare facilities. We introduce SANSformer, an attention-free sequential model designed with specific inductive biases to cater for the unique characteristics of EHR data. Our results illuminate the promising potential of tailored attention-free models and self-supervised pretraining in refining healthcare utilization predictions across various patient demographics.
  arXiv  Detail & Related papers  (2021-08-31T08:23:56Z)
• Examining and Combating Spurious Features under Distribution Shift [94.31956965507085]
  We define and analyze robust and spurious representations using the information-theoretic concept of minimal sufficient statistics. We prove that even when there is only bias of the input distribution, models can still pick up spurious features from their training data. Inspired by our analysis, we demonstrate that group DRO can fail when groups do not directly account for various spurious correlations.
  arXiv  Detail & Related papers  (2021-06-14T05:39:09Z)
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data [62.29031007761901]
  This paper proposes a novel contrastive regularized clinical classification model. We introduce two unique positive sampling strategies specifically tailored for EHR data. Our framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data.
  arXiv  Detail & Related papers  (2021-04-07T06:02:04Z)
• Temporal Phenotyping using Deep Predictive Clustering of Disease Progression [97.88605060346455]
  We develop a deep learning approach for clustering time-series data, where each cluster comprises patients who share similar future outcomes of interest. Experiments on two real-world datasets show that our model achieves superior clustering performance over state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2020-06-15T20:48:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.