Counterfactual Survival Q Learning for Longitudinal Randomized Trials via Buckley James Boosting

• URL: http://arxiv.org/abs/2508.11060v1
• Date: Thu, 14 Aug 2025 20:43:56 GMT
• Title: Counterfactual Survival Q Learning for Longitudinal Randomized Trials via Buckley James Boosting
• Abstract summary: We propose a Buckley James (BJ) Boost Q learning framework for estimating optimal dynamic treatment regimes.<n>The method integrates accelerated failure time models with iterative boosting techniques, including componentwise least squares and regression trees.<n>By directly modeling conditional survival time, BJ Boost Q learning avoids the restrictive proportional hazards assumption.
• Score: 2.3246627516636935
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a Buckley James (BJ) Boost Q learning framework for estimating optimal dynamic treatment regimes under right censored survival data, tailored for longitudinal randomized clinical trial settings. The method integrates accelerated failure time models with iterative boosting techniques, including componentwise least squares and regression trees, within a counterfactual Q learning framework. By directly modeling conditional survival time, BJ Boost Q learning avoids the restrictive proportional hazards assumption and enables unbiased estimation of stage specific Q functions. Grounded in potential outcomes, this framework ensures identifiability of the optimal treatment regime under standard causal assumptions. Compared to Cox based Q learning, which relies on hazard modeling and may suffer from bias under misspecification, our approach provides robust and flexible estimation. Simulation studies and analysis of the ACTG175 HIV trial demonstrate that BJ Boost Q learning yields higher accuracy in treatment decision making, especially in multistage settings where bias can accumulate.

Related papers

• Robust Dynamic Expansion for Continual Learning under Backdoor Attacks via Purification and Selective Recovery [21.762207770794916]
  In practical scenarios, task streams collected from untrusted sources may contain backdoor-poisoned samples.<n>We propose a robust dynamic-expansion framework that integrates sample purification, selective recovery, and robust expert routing.
  arXiv  Detail & Related papers  (2026-09-06T02:52:44Z)
• Understanding and Stabilizing Deep Q-Learning via Controlled Bootstrapping and Regulated Value Dynamics [2.358468106592515]
  We provide a systematic analysis of instability in deep Q-learning from three complementary perspectives.<n>We identify a reward-triggered self-reinforcing trap and characteristic parameter spike dynamics.<n>Experiments on Atari-100K and Procgen demonstrate competitive performance and improved training stability.
  arXiv  Detail & Related papers  (2026-08-17T06:55:41Z)
• Revisiting TD Target Aggregation under Uncertainty in Q-Learning [2.4908682918195804]
  Deep Q-Networks (DQNs) learn value functions through bootstrapped temporal-difference updates, where future returns are approximated using a greedy aggregation over next-state values.<n>We propose textbfSuccessor Rollout textbfAggregation textbfDeep textbfQ-Network (SADQ), a simple rollout to Q-learning that regularizes how the TD target is formed.
  arXiv  Detail & Related papers  (2026-08-04T03:31:24Z)
• Doubly Robust Proxy Causal Learning with Neural Mean Embeddings [58.8808794752741]
  We develop a neural doubly robust framework for proxy causal learning with continuous and structured treatments.<n>The framework covers population, heterogeneous, and conditional dose-response functions, yielding full response-curve estimators.<n>Across synthetic and image-valued benchmarks, the proposed estimators outperform existing baselines and single-bridge neural estimators.
  arXiv  Detail & Related papers  (2026-05-10T12:50:25Z)
• Cost-optimal Sequential Testing via Doubly Robust Q-learning [17.865867651025717]
  We study the problem of learning cost-optimal sequential decision policies from retrospective data.<n>We develop a doubly robust Q-learning framework for estimating optimal policies.
  arXiv  Detail & Related papers  (2026-04-13T08:26:27Z)
• SurvKAN: A Fully Parametric Survival Model Based on Kolmogorov-Arnold Networks [7.352227733654751]
  We introduce SurvKAN, a fully parametric, time-continuous survival model based on Kolmogorov-Arnold Networks (KANs)<n>SurvKAN treats time as an explicit input to a KAN that directly predicts the log-hazard function, enabling end-to-end training on the full survival likelihood.
  arXiv  Detail & Related papers  (2026-02-02T14:49:14Z)
• Uncertainty Quantification for Regression using Proper Scoring Rules [76.24649098854219]
  We introduce a unified UQ framework for regression based on proper scoring rules, such as CRPS, logarithmic, squared error, and quadratic scores.<n>We derive closed-form expressions for the uncertainty measures under practical parametric assumptions and show how to estimate them using ensembles of models.<n>Our broad evaluation on synthetic and real-world regression datasets provides guidance for selecting reliable UQ measures.
  arXiv  Detail & Related papers  (2025-09-30T17:52:12Z)
• Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework [2.5070297884580874]
  This study introduces ConformalDQN, a distribution-free conformal deep Q-learning approach for optimizing mechanical ventilation in intensive care units.<n>We trained and evaluated our model using ICU patient records from the MIMIC-IV database.
  arXiv  Detail & Related papers  (2024-12-17T06:55:20Z)
• FPBoost: Fully Parametric Gradient Boosting for Survival Analysis [4.09225917049674]
  FPBoost is a survival model that combines a weighted sum of fully parametric hazard functions with gradient boosting.<n>We show how FPBoost is a universal approximator of hazard functions, offering full event-time modeling flexibility.
  arXiv  Detail & Related papers  (2024-09-20T09:57:17Z)
• Selective Learning: Towards Robust Calibration with Dynamic Regularization [79.92633587914659]
  Miscalibration in deep learning refers to there is a discrepancy between the predicted confidence and performance. We introduce Dynamic Regularization (DReg) which aims to learn what should be learned during training thereby circumventing the confidence adjusting trade-off.
  arXiv  Detail & Related papers  (2024-02-13T11:25:20Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RL [86.0987896274354]
  We first identify a fundamental pattern, self-excitation, as the primary cause of Q-value estimation divergence in offline RL. We then propose a novel Self-Excite Eigenvalue Measure (SEEM) metric to measure the evolving property of Q-network at training. For the first time, our theory can reliably decide whether the training will diverge at an early stage.
  arXiv  Detail & Related papers  (2023-10-06T17:57:44Z)
• Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regression [47.1405366895538]
  Sparse linear regression methods for high-dimensional data commonly assume that residuals have constant variance, which can be violated in practice. This paper proposes estimating high-dimensional heteroscedastic linear regression models using a heteroscedastic partitioned empirical Bayes Expectation Conditional Maximization algorithm.
  arXiv  Detail & Related papers  (2023-09-15T22:06:29Z)
• A Flexible Framework for Incorporating Patient Preferences Into Q-Learning [1.2999518604217852]
  We propose a latent model approach that adapts Q-learning to tackle the aforementioned difficulties.<n>We conduct simulation experiments based on an ongoing trial for low back pain as well as a well-known trial for schizophrenia.<n>LUQ-Learning achieves highly competitive performance compared to alternative baselines.
  arXiv  Detail & Related papers  (2023-07-22T08:58:07Z)
• Robust and Agnostic Learning of Conditional Distributional Treatment Effects [44.31792000298105]
  We provide a new robust and model-agnostic methodology for learning the conditional DTE (CDTE) for a class of problems.<n>Our method is model-agnostic in that it can provide the best projection of CDTE onto the regression model class.<n>We investigate the behavior of our proposal in simulations, as well as in a case study of 401(k) eligibility effects on wealth.
  arXiv  Detail & Related papers  (2022-05-23T17:40:31Z)
• Temporal-Difference Value Estimation via Uncertainty-Guided Soft Updates [110.92598350897192]
  Q-Learning has proven effective at learning a policy to perform control tasks. estimation noise becomes a bias after the max operator in the policy improvement step. We present Unbiased Soft Q-Learning (UQL), which extends the work of EQL from two action, finite state spaces to multi-action, infinite state Markov Decision Processes.
  arXiv  Detail & Related papers  (2021-10-28T00:07:19Z)
• Cross Learning in Deep Q-Networks [82.20059754270302]
  We propose a novel cross Q-learning algorithm, aim at alleviating the well-known overestimation problem in value-based reinforcement learning methods. Our algorithm builds on double Q-learning, by maintaining a set of parallel models and estimate the Q-value based on a randomly selected network.
  arXiv  Detail & Related papers  (2020-09-29T04:58:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.