Cost-optimal Sequential Testing via Doubly Robust Q-learning

• URL: http://arxiv.org/abs/2604.11165v2
• Date: Wed, 15 Apr 2026 01:51:43 GMT
• Title: Cost-optimal Sequential Testing via Doubly Robust Q-learning
• Abstract summary: We study the problem of learning cost-optimal sequential decision policies from retrospective data.<n>We develop a doubly robust Q-learning framework for estimating optimal policies.
• Score: 17.865867651025717
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clinical decision-making often involves selecting tests that are costly, invasive, or time-consuming, motivating individualized, sequential strategies for what to measure and when to stop ascertaining. We study the problem of learning cost-optimal sequential decision policies from retrospective data, where test availability depends on prior results, inducing informative missingness. Under a sequential missing-at-random mechanism, we develop a doubly robust Q-learning framework for estimating optimal policies. The method introduces path-specific inverse probability weights that account for heterogeneous test trajectories and satisfy a normalization property conditional on the observed history. By combining these weights with auxiliary contrast models, we construct orthogonal pseudo-outcomes that enable unbiased policy learning when either the acquisition model or the contrast model is correctly specified. We establish oracle inequalities for the stage-wise contrast estimators, along with convergence rates, regret bounds, and misclassification rates for the learned policy. Simulations demonstrate improved cost-adjusted performance over weighted and complete-case baselines, and an application to a prostate cancer cohort study illustrates how the method reduces testing cost without compromising predictive accuracy.

Related papers

• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• Machine learning to optimize precision in the analysis of randomized trials: A journey in pre-specified, yet data-adaptive learning [2.6827221447298406]
  We tell our story of developing, evaluating, and implementing a machine learning-based approach for covariate adjustment.<n>We provide the rationale for as well as the practical concerns with such an approach for estimating marginal effects.<n>We present the results from applying our approach in the primary, pre-specified analysis of 8 recently published trials.
  arXiv  Detail & Related papers  (2025-12-15T18:05:45Z)
• A systematic evaluation of uncertainty quantification techniques in deep learning: a case study in photoplethysmography signal analysis [1.6690512882610855]
  Deep learning models can be used to continuously monitor physiological parameters outside of clinical settings.<n>There is risk of poor performance when deployed in practical measurement scenarios leading to negative patient outcomes.<n>Here we implement eight uncertainty (UQ) techniques to models trained on two clinically relevant prediction tasks.
  arXiv  Detail & Related papers  (2025-10-31T22:54:13Z)
• Counterfactual Survival Q Learning for Longitudinal Randomized Trials via Buckley James Boosting [2.3246627516636935]
  We propose a Buckley James (BJ) Boost Q learning framework for estimating optimal dynamic treatment regimes.<n>The method integrates accelerated failure time models with iterative boosting techniques, including componentwise least squares and regression trees.<n>By directly modeling conditional survival time, BJ Boost Q learning avoids the restrictive proportional hazards assumption.
  arXiv  Detail & Related papers  (2025-08-14T20:43:56Z)
• Model-free Methods for Event History Analysis and Efficient Adjustment (PhD Thesis) [55.2480439325792]
  This thesis is a series of independent contributions to statistics unified by a model-free perspective.<n>The first chapter elaborates on how a model-free perspective can be used to formulate flexible methods that leverage prediction techniques from machine learning.<n>The second chapter studies the concept of local independence, which describes whether the evolution of one process is directly influenced by another.
  arXiv  Detail & Related papers  (2025-02-11T19:24:09Z)
• Sublinear Regret for a Class of Continuous-Time Linear-Quadratic Reinforcement Learning Problems [10.404992912881601]
  We study reinforcement learning (RL) for a class of continuous-time linear-quadratic (LQ) control problems for diffusions.<n>We apply a model-free approach that relies neither on knowledge of model parameters nor on their estimations, and devise an RL algorithm to learn the optimal policy parameter directly.
  arXiv  Detail & Related papers  (2024-07-24T12:26:21Z)
• Learning Optimal Deterministic Policies with Stochastic Policy Gradients [62.81324245896716]
  Policy gradient (PG) methods are successful approaches to deal with continuous reinforcement learning (RL) problems. In common practice, convergence (hyper)policies are learned only to deploy their deterministic version. We show how to tune the exploration level used for learning to optimize the trade-off between the sample complexity and the performance of the deployed deterministic policy.
  arXiv  Detail & Related papers  (2024-05-03T16:45:15Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• Improved Policy Evaluation for Randomized Trials of Algorithmic Resource Allocation [54.72195809248172]
  We present a new estimator leveraging our proposed novel concept, that involves retrospective reshuffling of participants across experimental arms at the end of an RCT. We prove theoretically that such an estimator is more accurate than common estimators based on sample means.
  arXiv  Detail & Related papers  (2023-02-06T05:17:22Z)
• Robust and Agnostic Learning of Conditional Distributional Treatment Effects [44.31792000298105]
  We provide a new robust and model-agnostic methodology for learning the conditional DTE (CDTE) for a class of problems.<n>Our method is model-agnostic in that it can provide the best projection of CDTE onto the regression model class.<n>We investigate the behavior of our proposal in simulations, as well as in a case study of 401(k) eligibility effects on wealth.
  arXiv  Detail & Related papers  (2022-05-23T17:40:31Z)
• Bayesian Nonparametric Cost-Effectiveness Analyses: Causal Estimation and Adaptive Subgroup Discovery [0.0]
  We develop a nonparametric Bayesian model for joint cost-survival distributions in the presence of censoring. We use our model to assess the cost-efficacy of endometrial chemotherapy versus radiation adjuvant therapy for treating cancer in the SEER-Medicare database.
  arXiv  Detail & Related papers  (2020-02-11T21:51:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.