Censoring-Aware Tree-Based Reinforcement Learning for Estimating Dynamic Treatment Regimes with Censored Outcomes

• URL: http://arxiv.org/abs/2503.06690v1
• Date: Sun, 09 Mar 2025 16:53:09 GMT
• Title: Censoring-Aware Tree-Based Reinforcement Learning for Estimating Dynamic Treatment Regimes with Censored Outcomes
• Abstract summary: Censoring-Aware Tree-Based Reinforcement Learning (CA-TRL) is a novel framework to address the complexities associated with censored data.<n>We demonstrate its effectiveness through extensive simulations and real-world applications using the SANAD epilepsy dataset.
• Score: 4.877686100899469
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dynamic Treatment Regimes (DTRs) provide a systematic approach for making sequential treatment decisions that adapt to individual patient characteristics, particularly in clinical contexts where survival outcomes are of interest. Censoring-Aware Tree-Based Reinforcement Learning (CA-TRL) is a novel framework to address the complexities associated with censored data when estimating optimal DTRs. We explore ways to learn effective DTRs, from observational data. By enhancing traditional tree-based reinforcement learning methods with augmented inverse probability weighting (AIPW) and censoring-aware modifications, CA-TRL delivers robust and interpretable treatment strategies. We demonstrate its effectiveness through extensive simulations and real-world applications using the SANAD epilepsy dataset, where it outperformed the recently proposed ASCL method in key metrics such as restricted mean survival time (RMST) and decision-making accuracy. This work represents a step forward in advancing personalized and data-driven treatment strategies across diverse healthcare settings.

Related papers

• IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning [57.16513298507272]
  We propose IADD-TR, a unified framework combining Intervention-Aware Dynamics Decoupling (IADD) and Targeted Regularization (TR)<n>IADD factorizes transitions into an action-intervention stage and an action-free natural evolution stage, using a zero-action anchor to resolve the non-uniqueness of this two-stage factorization for robust generalization.<n>For policy learning, we derive TR from the efficient influence function of a replay-state policy-gradient functional.
  arXiv  Detail & Related papers  (2026-08-11T08:20:02Z)
• From Static Risk to Dynamic Trajectories: Toward World-Model-Inspired Clinical Prediction [61.12883122613684]
  This Review focuses on intervention-aware disease trajectory modeling in clinical AI--methods estimating patient-specific longitudinal disease evolution and assessing trajectory changes under alternative treatments.<n>We organize the field around six linked components: three decision tasks (factual forecasting, counterfactual estimation, policy evaluation) and three data-generating mechanisms (disease evolution, treatment assignment, observation process) that determine identifiability.<n>We present the first unified framework bridging forecasting, counterfactual trajectories, and policy evaluation across discrete/continuous time, explicitly addressing treatment assignment, time-varying confounding, and observation bias.
  arXiv  Detail & Related papers  (2026-05-16T10:45:26Z)
• Learning Preference-Based Objectives from Clinical Narratives for Sequential Treatment Decision-Making [8.302720355559359]
  We propose a framework for learning reward functions directly from discharge summaries by treating them as scalable supervision for trajectory-level preferences.<n>We derive trajectory quality scores (TQS) and construct pairwise preferences over patient trajectories, enabling reward learning via a structured preference-based objective.
  arXiv  Detail & Related papers  (2026-04-12T19:18:02Z)
• Treatment Stitching with Schrödinger Bridge for Enhancing Offline Reinforcement Learning in Adaptive Treatment Strategies [16.34689464480395]
  We propose Treatment Stitching, a novel data augmentation framework that generates clinically valid treatment trajectories.<n>TreatStitch identifies similar intermediate patient states across different trajectories and stitches their respective segments.<n>By augmenting these synthetic trajectories into the original dataset, offline RL can learn from a more diverse dataset.
  arXiv  Detail & Related papers  (2025-11-15T07:37:49Z)
• Prior-informed optimization of treatment recommendation via bandit algorithms trained on large language model-processed historical records [0.6875312133832079]
  Current medical practice depends on standardized treatment frameworks and empirical methodologies that neglect individual patient variations.<n>We develop a comprehensive system integrating Large Language Models (LLMs), Conditional Tabular Generative Adversarial Networks (CTGAN), T-learner counterfactual models, and contextual bandit approaches.
  arXiv  Detail & Related papers  (2025-10-21T18:57:00Z)
• DTR-Bench: An in silico Environment and Benchmark Platform for Reinforcement Learning Based Dynamic Treatment Regime [18.443316087890324]
  Reinforcement learning (RL) has garnered increasing recognition for its potential to optimise dynamic treatment regimes (DTRs) in personalised medicine. We introduce textitDTR-Bench, a benchmarking platform for simulating diverse healthcare scenarios. We evaluate various state-of-the-art RL algorithms across these settings, particularly highlighting their performance amidst real-world challenges.
  arXiv  Detail & Related papers  (2024-05-28T21:40:00Z)
• Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites: A Federated Learning Approach with Noise-Resilient Training [75.40980802817349]
  Deep learning models have shown promise for automatically segmenting MS lesions, but the scarcity of accurately annotated data hinders progress in this area. We introduce a Decoupled Hard Label Correction (DHLC) strategy that considers the imbalanced distribution and fuzzy boundaries of MS lesions. We also introduce a Centrally Enhanced Label Correction (CELC) strategy, which leverages the aggregated central model as a correction teacher for all sites.
  arXiv  Detail & Related papers  (2023-08-31T00:36:10Z)
• TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic Tree-Based Memory Network [54.332862955411656]
  Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment. In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials. We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
  arXiv  Detail & Related papers  (2023-07-19T12:35:09Z)
• Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Care [46.2482873419289]
  We introduce a deep Q-learning approach to obtain more reliable critical care policies. We evaluate our method in off-policy and offline settings using simulated environments and real health records from intensive care units.
  arXiv  Detail & Related papers  (2023-06-13T18:02:57Z)
• Kernel-Based Distributed Q-Learning: A Scalable Reinforcement Learning Approach for Dynamic Treatment Regimes [20.62850899223748]
  We present a scalable kernel-based distributed Q-learning algorithm for generating dynamic treatment regimes (DTRs) The results demonstrate that our algorithm significantly reduces the computational complexity associated with the state-of-the-art deep reinforcement learning methods.
  arXiv  Detail & Related papers  (2023-02-21T04:15:34Z)
• Meta-analysis of individualized treatment rules via sign-coherency [3.432284729311483]
  We develop a method for individual-level meta-analysis of ITRs, which jointly learns site-specific ITRs. We also develop an adaptive procedure for model tuning, using information criteria tailored to the ITR learning problem.
  arXiv  Detail & Related papers  (2022-11-28T15:55:55Z)
• Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability [82.29775890542967]
  Estimating personalized effects of treatments is a complex, yet pervasive problem. Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools. We use post-hoc feature importance methods to identify features that influence the model's predictions.
  arXiv  Detail & Related papers  (2022-06-16T17:59:05Z)
• Learning Optimal Dynamic Treatment Regimes Using Causal Tree Methods in Medicine [20.401805132360654]
  We develop two novel methods for learning optimal dynamic treatment regimes (DTRs) Our methods are based on a data-driven estimation of heterogeneous treatment effects using causal tree methods. We evaluate our proposed methods using synthetic data and then apply them to real-world data from intensive care units.
  arXiv  Detail & Related papers  (2022-04-14T17:27:08Z)
• Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approach [0.0]
  Dynamic Treatment Regimes (DTRs) are widely studied to formalize this process. We develop Reinforcement Learning methods to efficiently learn optimal treatment regimes.
  arXiv  Detail & Related papers  (2021-12-08T20:22:04Z)
• Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network [51.44506007844284]
  We propose a shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection surgery. We first devise an algorithm to automatically generate relation keypoint heatmaps, which intuitively represent the prior knowledge of spatial relations among landmarks. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process.
  arXiv  Detail & Related papers  (2021-11-08T07:57:30Z)
• COVI-AgentSim: an Agent-based Model for Evaluating Methods of Digital Contact Tracing [68.68882022019272]
  COVI-AgentSim is an agent-based compartmental simulator based on virology, disease progression, social contact networks, and mobility patterns. We use COVI-AgentSim to perform cost-adjusted analyses comparing no DCT to: 1) standard binary contact tracing (BCT) that assigns binary recommendations based on binary test results; and 2) a rule-based method for feature-based contact tracing (FCT) that assigns a graded level of recommendation based on diverse individual features.
  arXiv  Detail & Related papers  (2020-10-30T00:47:01Z)
• DTR Bandit: Learning to Make Response-Adaptive Decisions With Low Regret [59.81290762273153]
  Dynamic treatment regimes (DTRs) are personalized, adaptive, multi-stage treatment plans that adapt treatment decisions to an individual's initial features and to intermediate outcomes and features at each subsequent stage. We propose a novel algorithm that, by carefully balancing exploration and exploitation, is guaranteed to achieve rate-optimal regret when the transition and reward models are linear.
  arXiv  Detail & Related papers  (2020-05-06T13:03:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.