MedGym:A Unified Continuous-Time Benchmark for Dynamic Medical Treatment Reinforcement Learning

• URL: http://arxiv.org/abs/2606.01028v1
• Date: Sun, 31 May 2026 05:36:03 GMT
• Title: MedGym:A Unified Continuous-Time Benchmark for Dynamic Medical Treatment Reinforcement Learning
• Abstract summary: We introduce MedGym, a benchmark environment for dynamic treatment recommendation.<n>MedGym models longitudinal patient evolution in a continuous-time framework.<n>It constructs a medical RL benchmark from clinical data by using Physics-Informed Neural Networks.
• Score: 16.00709468365385
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical treatment recommendation poses several challenges to reinforcement learning (RL): patient physiology evolves in continuous time, measurements and interventions are performed at irregular intervals, and treatment effects vary substantially across individuals. Existing RL formulations and simulated environments, however, are based on discrete-time MDP or POMDP abstractions with fixed or pre-specified decision intervals. Thus, it remains difficult to evaluate whether RL methods can handle time-interval-dependent disease progression, personalized treatment response, and safety between consecutive measurement points. To address this gap, we introduce MedGym, a benchmark environment for dynamic treatment recommendation. MedGym models longitudinal patient evolution in a continuous-time framework and constructs a configurable medical RL benchmark from clinical data by using Physics-Informed Neural Networks. The resulting benchmark supports both offline and online RL, and enables direct comparison between discrete-time and continuous-time methods under irregular treatment timing and patient-specific dynamics. Besides, MedGym supports evaluation from clinically important perspectives, including personalization, trajectory-level safety, and the performance gap between model-based offline learning and online deployment. By providing a standardized and configurable benchmark for continuous-time dynamic treatment, MedGym aims to facilitate more realistic and informative evaluation of medical RL methods.

Related papers

• How Good are Foundation Models in Longitudinal MRI Disease Progression Reasoning? [49.14050864978079]
  We introduce the Time-Aware Multi-View MRI Benchmark, an evaluation framework unifying multi-view anatomical input, temporal reasoning, and structured localization guidance.<n>The benchmark comprises 3,920 expert-verified question-answer pairs derived from 890 patients across over 3,200 longitudinal MRI timepoints.
  arXiv  Detail & Related papers  (2026-08-13T14:35:20Z)
• EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins [3.229417465367373]
  Existing reinforcement learning approaches learn fixed strategies for sepsis treatment.<n>We propose EHRMPC, a framework that decouples learning patient dynamics from optimizing treatment.
  arXiv  Detail & Related papers  (2026-07-07T19:24:41Z)
• From Static Risk to Dynamic Trajectories: Toward World-Model-Inspired Clinical Prediction [61.12883122613684]
  This Review focuses on intervention-aware disease trajectory modeling in clinical AI--methods estimating patient-specific longitudinal disease evolution and assessing trajectory changes under alternative treatments.<n>We organize the field around six linked components: three decision tasks (factual forecasting, counterfactual estimation, policy evaluation) and three data-generating mechanisms (disease evolution, treatment assignment, observation process) that determine identifiability.<n>We present the first unified framework bridging forecasting, counterfactual trajectories, and policy evaluation across discrete/continuous time, explicitly addressing treatment assignment, time-varying confounding, and observation bias.
  arXiv  Detail & Related papers  (2026-05-16T10:45:26Z)
• Agentifying Patient Dynamics within LLMs through Interacting with Clinical World Model [57.78184285979881]
  We introduce SepsisAgent, a world model-augmented LLM agent for sepsis treatment recommendation.<n>SepsisAgent uses a learned Clinical World Model to simulate patient responses under candidate fluid--vasopressor interventions.
  arXiv  Detail & Related papers  (2026-05-14T11:50:00Z)
• Learning Dynamic Representations and Policies from Multimodal Clinical Time-Series with Informative Missingness [5.351519104745287]
  We propose a patient representation learning framework for multimodal clinical records.<n>The framework combines a multimodal encoder that captures signals from structured and textual data together with their observation patterns.<n>We evaluate the framework on ICU sepsis cohorts from MIMIC-III, MIMIC-IV, and eICU.
  arXiv  Detail & Related papers  (2026-04-23T03:07:57Z)
• Beyond the ATE: Interpretable Modelling of Treatment Effects over Dose and Time [46.2482873419289]
  We propose a framework for modelling treatment effect trajectories as smooth surfaces over dose and time.<n>Our approach decouples the estimation of trajectory shape from the specification of clinically relevant properties.<n>We show that our method yields accurate, interpretable, and editable models of treatment dynamics.
  arXiv  Detail & Related papers  (2025-07-09T20:33:33Z)
• medDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Support [3.8382507197481144]
  medDreamer is a novel model-based reinforcement learning framework for personalized treatment recommendation.<n>It simulates latent patient states from irregular data and a two-phase policy trained on a hybrid of real and imagined trajectories.<n>It significantly outperforms model-free and model-based baselines in both clinical outcomes and off-policy metrics.
  arXiv  Detail & Related papers  (2025-05-26T10:16:39Z)
• Dynamic Local Average Treatment Effects [19.014535120129338]
  We consider Dynamic Treatment Regimes (DTRs) with One Sided Noncompliance in digital recommendations and adaptive medical trials.<n>We provide nonparametric identification, estimation, and inference for Dynamic Local Average Treatment Effects (LATEs)<n>We show that the assumptions are sufficient to identify Dynamic LATEs for treating in multiple time periods.
  arXiv  Detail & Related papers  (2024-05-02T16:52:09Z)
• SCouT: Synthetic Counterfactuals via Spatiotemporal Transformers for Actionable Healthcare [6.431557011732579]
  The Synthetic Control method has pioneered a class of powerful data-driven techniques to estimate the counterfactual reality of a unit from donor units. At its core, the technique involves a linear model fitted on the pre-intervention period that combines donor outcomes to yield the counterfactual. We propose an approach to use localtemporal information before the onset of the intervention as a promising way to estimate the counterfactual sequence.
  arXiv  Detail & Related papers  (2022-07-09T07:00:17Z)
• Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations [84.42837346400151]
  Estimating counterfactual outcomes over time has the potential to unlock personalized healthcare. Existing causal inference approaches consider regular, discrete-time intervals between observations and treatment decisions. We propose a controllable simulation environment based on a model of tumor growth for a range of scenarios.
  arXiv  Detail & Related papers  (2022-06-16T17:15:15Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)
• Estimating Counterfactual Treatment Outcomes over Time Through Adversarially Balanced Representations [114.16762407465427]
  We introduce the Counterfactual Recurrent Network (CRN) to estimate treatment effects over time. CRN uses domain adversarial training to build balancing representations of the patient history. We show how our model achieves lower error in estimating counterfactuals and in choosing the correct treatment and timing of treatment.
  arXiv  Detail & Related papers  (2020-02-10T20:47:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.