Stage-Aware Learning for Dynamic Treatments
- URL: http://arxiv.org/abs/2310.19300v1
- Date: Mon, 30 Oct 2023 06:35:31 GMT
- Title: Stage-Aware Learning for Dynamic Treatments
- Authors: Hanwen Ye, Wenzhuo Zhou, Ruoqing Zhu, Annie Qu
- Abstract summary: We propose a novel individualized learning method for dynamic treatment regimes.
We focus on prioritizing alignment between the observed treatment trajectory and the one obtained by the optimal regime across decision stages.
By relaxing the restriction that the observed trajectory must be fully aligned with the optimal treatments, our approach substantially improves the sample efficiency and stability of inverse probability weighted methods.
- Score: 4.033641609534417
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent advances in dynamic treatment regimes (DTRs) provide powerful optimal
treatment searching algorithms, which are tailored to individuals' specific
needs and able to maximize their expected clinical benefits. However, existing
algorithms could suffer from insufficient sample size under optimal treatments,
especially for chronic diseases involving long stages of decision-making. To
address these challenges, we propose a novel individualized learning method
which estimates the DTR with a focus on prioritizing alignment between the
observed treatment trajectory and the one obtained by the optimal regime across
decision stages. By relaxing the restriction that the observed trajectory must
be fully aligned with the optimal treatments, our approach substantially
improves the sample efficiency and stability of inverse probability weighted
based methods. In particular, the proposed learning scheme builds a more
general framework which includes the popular outcome weighted learning
framework as a special case of ours. Moreover, we introduce the notion of stage
importance scores along with an attention mechanism to explicitly account for
heterogeneity among decision stages. We establish the theoretical properties of
the proposed approach, including the Fisher consistency and finite-sample
performance bound. Empirically, we evaluate the proposed method in extensive
simulated environments and a real case study for COVID-19 pandemic.
Related papers
- Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences.
Our method is especially suitable for problems with well-specified likelihoods.
We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
arXiv Detail & Related papers (2023-11-08T00:10:21Z) - A Flexible Framework for Incorporating Patient Preferences Into
Q-Learning [1.2891210250935146]
In real-world healthcare problems, there are often multiple competing outcomes of interest, such as treatment efficacy and side effect severity.
statistical methods for estimating dynamic treatment regimes (DTRs) usually assume a single outcome of interest.
This includes restrictions to a single time point and two outcomes, the inability to incorporate self-reported patient preferences and limited theoretical guarantees.
arXiv Detail & Related papers (2023-07-22T08:58:07Z) - Provably Efficient UCB-type Algorithms For Learning Predictive State
Representations [55.00359893021461]
The sequential decision-making problem is statistically learnable if it admits a low-rank structure modeled by predictive state representations (PSRs)
This paper proposes the first known UCB-type approach for PSRs, featuring a novel bonus term that upper bounds the total variation distance between the estimated and true models.
In contrast to existing approaches for PSRs, our UCB-type algorithms enjoy computational tractability, last-iterate guaranteed near-optimal policy, and guaranteed model accuracy.
arXiv Detail & Related papers (2023-07-01T18:35:21Z) - Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning
Approach to Critical Care [68.8204255655161]
We introduce a deep Q-learning approach able to obtain more reliable critical care policies.
We achieve this by first pruning the action set based on all available rewards, and second training a final model based on the sparse main reward but with a restricted action set.
arXiv Detail & Related papers (2023-06-13T18:02:57Z) - Quasi-optimal Reinforcement Learning with Continuous Actions [8.17049210746654]
We develop a novel emphquasi-optimal learning algorithm, which can be easily optimized in off-policy settings.
We evaluate our algorithm with comprehensive simulated experiments and a dose suggestion real application to Ohio Type 1 diabetes dataset.
arXiv Detail & Related papers (2023-01-21T11:30:13Z) - TCFimt: Temporal Counterfactual Forecasting from Individual Multiple
Treatment Perspective [50.675845725806724]
We propose a comprehensive framework of temporal counterfactual forecasting from an individual multiple treatment perspective (TCFimt)
TCFimt constructs adversarial tasks in a seq2seq framework to alleviate selection and time-varying bias and designs a contrastive learning-based block to decouple a mixed treatment effect into separated main treatment effects and causal interactions.
The proposed method shows satisfactory performance in predicting future outcomes with specific treatments and in choosing optimal treatment type and timing than state-of-the-art methods.
arXiv Detail & Related papers (2022-12-17T15:01:05Z) - SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event
Data [83.50281440043241]
We study the problem of inferring heterogeneous treatment effects from time-to-event data.
We propose a novel deep learning method for treatment-specific hazard estimation based on balancing representations.
arXiv Detail & Related papers (2021-10-26T20:13:17Z) - Estimating Optimal Infinite Horizon Dynamic Treatment Regimes via
pT-Learning [2.0625936401496237]
Recent advances in mobile health (mHealth) technology provide an effective way to monitor individuals' health statuses and deliver just-in-time personalized interventions.
The practical use of mHealth technology raises unique challenges to existing methodologies on learning an optimal dynamic treatment regime.
We propose a Proximal Temporal Learning framework to estimate an optimal regime adaptively adjusted between deterministic and sparse policy models.
arXiv Detail & Related papers (2021-10-20T18:38:22Z) - Stochastic Intervention for Causal Inference via Reinforcement Learning [7.015556609676951]
Central to causal inference is the treatment effect estimation of intervention strategies.
Existing methods are mostly restricted to the deterministic treatment and compare outcomes under different treatments.
We propose a new effective framework to estimate the treatment effect on intervention.
arXiv Detail & Related papers (2021-05-28T00:11:22Z) - Resource Planning for Hospitals Under Special Consideration of the
COVID-19 Pandemic: Optimization and Sensitivity Analysis [87.31348761201716]
Crises like the COVID-19 pandemic pose a serious challenge to health-care institutions.
BaBSim.Hospital is a tool for capacity planning based on discrete event simulation.
We aim to investigate and optimize these parameters to improve BaBSim.Hospital.
arXiv Detail & Related papers (2021-05-16T12:38:35Z) - Multicategory Angle-based Learning for Estimating Optimal Dynamic
Treatment Regimes with Censored Data [12.499787110182632]
An optimal treatment regime (DTR) consists of a sequence of decision rules in maximizing long-term benefits.
In this paper, we develop a novel angle-based approach to target the optimal DTR under a multicategory treatment framework.
Our numerical studies show that the proposed method outperforms competing methods in terms of maximizing the conditional survival function.
arXiv Detail & Related papers (2020-01-14T05:19:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.