Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approach
- URL: http://arxiv.org/abs/2112.04571v4
- Date: Mon, 5 Jun 2023 16:32:55 GMT
- Title: Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approach
- Authors: Soroush Saghafian
- Abstract summary: Dynamic Treatment Regimes (DTRs) are widely studied to formalize this process.
We develop Reinforcement Learning methods to efficiently learn optimal treatment regimes.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: A main research goal in various studies is to use an observational data set
and provide a new set of counterfactual guidelines that can yield causal
improvements. Dynamic Treatment Regimes (DTRs) are widely studied to formalize
this process. However, available methods in finding optimal DTRs often rely on
assumptions that are violated in real-world applications (e.g., medical
decision-making or public policy), especially when (a) the existence of
unobserved confounders cannot be ignored, and (b) the unobserved confounders
are time-varying (e.g., affected by previous actions). When such assumptions
are violated, one often faces ambiguity regarding the underlying causal model.
This ambiguity is inevitable, since the dynamics of unobserved confounders and
their causal impact on the observed part of the data cannot be understood from
the observed data. Motivated by a case study of finding superior treatment
regimes for patients who underwent transplantation in our partner hospital and
faced a medical condition known as New Onset Diabetes After Transplantation
(NODAT), we extend DTRs to a new class termed Ambiguous Dynamic Treatment
Regimes (ADTRs), in which the causal impact of treatment regimes is evaluated
based on a "cloud" of causal models. We then connect ADTRs to Ambiguous
Partially Observable Mark Decision Processes (APOMDPs) and develop
Reinforcement Learning methods, which enable using the observed data to
efficiently learn an optimal treatment regime. We establish theoretical results
for these learning methods, including (weak) consistency and asymptotic
normality. We further evaluate the performance of these learning methods both
in our case study and in simulation experiments.
Related papers
- Deep State-Space Generative Model For Correlated Time-to-Event Predictions [54.3637600983898]
We propose a deep latent state-space generative model to capture the interactions among different types of correlated clinical events.
Our method also uncovers meaningful insights about the latent correlations among mortality and different types of organ failures.
arXiv Detail & Related papers (2024-07-28T02:42:36Z) - Benchmarking Heterogeneous Treatment Effect Models through the Lens of
Interpretability [82.29775890542967]
Estimating personalized effects of treatments is a complex, yet pervasive problem.
Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools.
We use post-hoc feature importance methods to identify features that influence the model's predictions.
arXiv Detail & Related papers (2022-06-16T17:59:05Z) - Deconfounding Actor-Critic Network with Policy Adaptation for Dynamic
Treatment Regimes [8.705574459727202]
We develop a new deconfounding actor-critic network (DAC) to learn optimal treatment policies for patients.
To avoid punishing effective treatment actions non-survivors received, we design a short-term reward to capture patients' immediate health state changes.
The experimental results on one semi-synthetic and two different real-world datasets show the proposed model outperforms the state-of-the-art models.
arXiv Detail & Related papers (2022-05-19T20:53:03Z) - Learning Optimal Dynamic Treatment Regimes Using Causal Tree Methods in
Medicine [20.401805132360654]
We develop two novel methods for learning optimal dynamic treatment regimes (DTRs)
Our methods are based on a data-driven estimation of heterogeneous treatment effects using causal tree methods.
We evaluate our proposed methods using synthetic data and then apply them to real-world data from intensive care units.
arXiv Detail & Related papers (2022-04-14T17:27:08Z) - SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event
Data [83.50281440043241]
We study the problem of inferring heterogeneous treatment effects from time-to-event data.
We propose a novel deep learning method for treatment-specific hazard estimation based on balancing representations.
arXiv Detail & Related papers (2021-10-26T20:13:17Z) - Deep Bayesian Estimation for Dynamic Treatment Regimes with a Long
Follow-up Time [28.11470886127216]
Causal effect estimation for dynamic treatment regimes (DTRs) contributes to sequential decision making.
We combine outcome regression models with treatment models for high dimensional features using uncensored subjects that are small in sample size.
Also, the developed deep Bayesian models can model uncertainty and output the prediction variance which is essential for the safety-aware applications, such as self-driving cars and medical treatment design.
arXiv Detail & Related papers (2021-09-20T13:21:39Z) - Proximal Learning for Individualized Treatment Regimes Under Unmeasured
Confounding [3.020737957610002]
We develop approaches to estimating optimal individualized treatment regimes (ITRs) in the presence of unmeasured confounding.
Based on these results, we propose several classification-based approaches to finding a variety of restricted in-class optimal ITRs.
arXiv Detail & Related papers (2021-05-03T21:49:49Z) - MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
This paper aims at a unified deep learning approach to predict patient prognosis and therapy response.
We formalize the prognosis modeling as a multi-modal asynchronous time series classification task.
Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
arXiv Detail & Related papers (2020-10-08T15:30:17Z) - Estimating Individual Treatment Effects with Time-Varying Confounders [9.784193264717098]
Estimating individual treatment effect (ITE) from observational data is meaningful and practical in healthcare.
Existing work mainly relies on the strong ignorability assumption that no hidden confounders exist.
We propose Deep Sequential Weighting (DSW) for estimating ITE with time-varying confounders.
arXiv Detail & Related papers (2020-08-27T02:21:56Z) - DTR Bandit: Learning to Make Response-Adaptive Decisions With Low Regret [59.81290762273153]
Dynamic treatment regimes (DTRs) are personalized, adaptive, multi-stage treatment plans that adapt treatment decisions to an individual's initial features and to intermediate outcomes and features at each subsequent stage.
We propose a novel algorithm that, by carefully balancing exploration and exploitation, is guaranteed to achieve rate-optimal regret when the transition and reward models are linear.
arXiv Detail & Related papers (2020-05-06T13:03:42Z) - Generalization Bounds and Representation Learning for Estimation of
Potential Outcomes and Causal Effects [61.03579766573421]
We study estimation of individual-level causal effects, such as a single patient's response to alternative medication.
We devise representation learning algorithms that minimize our bound, by regularizing the representation's induced treatment group distance.
We extend these algorithms to simultaneously learn a weighted representation to further reduce treatment group distances.
arXiv Detail & Related papers (2020-01-21T10:16:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.