Robust Learning for Optimal Dynamic Treatment Regimes with Observational Data

• URL: http://arxiv.org/abs/2404.00221v4
• Date: Wed, 20 Nov 2024 05:50:48 GMT
• Title: Robust Learning for Optimal Dynamic Treatment Regimes with Observational Data
• Authors: Shosei Sakaguchi,
• Abstract summary: We study the statistical learning of optimal dynamic treatment regimes (DTRs) that guide the optimal treatment assignment for each individual at each stage based on the individual's evolving history.
• Score: 0.0
• License:
• Abstract: Public policies and medical interventions often involve dynamics in their treatment assignments, where individuals receive a series of interventions over multiple stages. We study the statistical learning of optimal dynamic treatment regimes (DTRs) that guide the optimal treatment assignment for each individual at each stage based on the individual's evolving history. We propose a doubly robust, classification-based approach to learning the optimal DTR using observational data under the assumption of sequential ignorability. This approach learns the optimal DTR through backward induction. At each step, it constructs an augmented inverse probability weighting (AIPW) estimator of the policy value function and maximizes it to learn the optimal policy for the corresponding stage. We show that the resulting DTR can achieve an optimal convergence rate of $n^{-1/2}$ for welfare regret under mild convergence conditions on estimators of the nuisance components.

Related papers

• Truncating Trajectories in Monte Carlo Policy Evaluation: an Adaptive Approach [51.76826149868971]
  Policy evaluation via Monte Carlo simulation is at the core of many MC Reinforcement Learning (RL) algorithms. We propose as a quality index a surrogate of the mean squared error of a return estimator that uses trajectories of different lengths. We present an adaptive algorithm called Robust and Iterative Data collection strategy Optimization (RIDO)
  arXiv  Detail & Related papers  (2024-10-17T11:47:56Z)
• Denoising Pre-Training and Customized Prompt Learning for Efficient Multi-Behavior Sequential Recommendation [69.60321475454843]
  We propose DPCPL, the first pre-training and prompt-tuning paradigm tailored for Multi-Behavior Sequential Recommendation. In the pre-training stage, we propose a novel Efficient Behavior Miner (EBM) to filter out the noise at multiple time scales. Subsequently, we propose to tune the pre-trained model in a highly efficient manner with the proposed Customized Prompt Learning (CPL) module.
  arXiv  Detail & Related papers  (2024-08-21T06:48:38Z)
• Efficient and robust transfer learning of optimal individualized treatment regimes with right-censored survival data [7.308241944759317]
  An individualized treatment regime (ITR) is a decision rule that assigns treatments based on patients' characteristics. We propose a doubly robust estimator of the value function, and the optimal ITR is learned by maximizing the value function within a pre-specified class of ITRs. We evaluate the empirical performance of the proposed method by simulation studies and a real data application of sodium bicarbonate therapy for patients with severe metabolic acidaemia.
  arXiv  Detail & Related papers  (2023-01-13T11:47:10Z)
• When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee [51.527543027813344]
  We propose systematic and efficient gradient-based methods for both one-way and two-way partial AUC (pAUC) For both one-way and two-way pAUC, we propose two algorithms and prove their convergence for optimizing their two formulations, respectively.
  arXiv  Detail & Related papers  (2022-03-01T01:59:53Z)
• Policy Learning for Optimal Individualized Dose Intervals [3.9801611649762263]
  We propose a new method to estimate such a policy. We prove that our estimated policy is consistent, and its risk converges to that of the best-in-class policy at a root-nn rate.
  arXiv  Detail & Related papers  (2022-02-24T17:59:20Z)
• Variance-Aware Off-Policy Evaluation with Linear Function Approximation [85.75516599931632]
  We study the off-policy evaluation problem in reinforcement learning with linear function approximation. We propose an algorithm, VA-OPE, which uses the estimated variance of the value function to reweight the Bellman residual in Fitted Q-Iteration.
  arXiv  Detail & Related papers  (2021-06-22T17:58:46Z)
• Estimation of Optimal Dynamic Treatment Assignment Rules under Policy Constraints [0.0]
  We study estimation of an optimal dynamic treatment regime that guides the optimal treatment assignment for each individual at each stage based on their history. The paper proposes two estimation methods: one solves the treatment assignment problem sequentially through backward induction, and the other solves the entire problem simultaneously across all stages.
  arXiv  Detail & Related papers  (2021-06-09T12:42:53Z)
• DTR Bandit: Learning to Make Response-Adaptive Decisions With Low Regret [59.81290762273153]
  Dynamic treatment regimes (DTRs) are personalized, adaptive, multi-stage treatment plans that adapt treatment decisions to an individual's initial features and to intermediate outcomes and features at each subsequent stage. We propose a novel algorithm that, by carefully balancing exploration and exploitation, is guaranteed to achieve rate-optimal regret when the transition and reward models are linear.
  arXiv  Detail & Related papers  (2020-05-06T13:03:42Z)
• Comment: Entropy Learning for Dynamic Treatment Regimes [58.442274475425144]
  JSLZ's approach leverages a rejection-and-sampling estimate of the value of a given decision rule based on inverse probability (IPW) and its interpretation as a weighted (or cost-sensitive) classification. Their use of smooth classification surrogates enables their careful approach to analyzing distributions. The IPW estimate is problematic as it leads to weights that discard most of the data and are extremely variable on whatever remains.
  arXiv  Detail & Related papers  (2020-04-06T16:11:05Z)
• Multicategory Angle-based Learning for Estimating Optimal Dynamic Treatment Regimes with Censored Data [12.499787110182632]
  An optimal treatment regime (DTR) consists of a sequence of decision rules in maximizing long-term benefits. In this paper, we develop a novel angle-based approach to target the optimal DTR under a multicategory treatment framework. Our numerical studies show that the proposed method outperforms competing methods in terms of maximizing the conditional survival function.
  arXiv  Detail & Related papers  (2020-01-14T05:19:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.