Deconfounding Actor-Critic Network with Policy Adaptation for Dynamic Treatment Regimes

• URL: http://arxiv.org/abs/2205.09852v1
• Date: Thu, 19 May 2022 20:53:03 GMT
• Title: Deconfounding Actor-Critic Network with Policy Adaptation for Dynamic Treatment Regimes
• Authors: Changchang Yin, Ruoqi Liu, Jeffrey Caterino, Ping Zhang
• Abstract summary: We develop a new deconfounding actor-critic network (DAC) to learn optimal treatment policies for patients. To avoid punishing effective treatment actions non-survivors received, we design a short-term reward to capture patients' immediate health state changes. The experimental results on one semi-synthetic and two different real-world datasets show the proposed model outperforms the state-of-the-art models.
• Score: 8.705574459727202
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Despite intense efforts in basic and clinical research, an individualized ventilation strategy for critically ill patients remains a major challenge. Recently, dynamic treatment regime (DTR) with reinforcement learning (RL) on electronic health records (EHR) has attracted interest from both the healthcare industry and machine learning research community. However, most learned DTR policies might be biased due to the existence of confounders. Although some treatment actions non-survivors received may be helpful, if confounders cause the mortality, the training of RL models guided by long-term outcomes (e.g., 90-day mortality) would punish those treatment actions causing the learned DTR policies to be suboptimal. In this study, we develop a new deconfounding actor-critic network (DAC) to learn optimal DTR policies for patients. To alleviate confounding issues, we incorporate a patient resampling module and a confounding balance module into our actor-critic framework. To avoid punishing the effective treatment actions non-survivors received, we design a short-term reward to capture patients' immediate health state changes. Combining short-term with long-term rewards could further improve the model performance. Moreover, we introduce a policy adaptation method to successfully transfer the learned model to new-source small-scale datasets. The experimental results on one semi-synthetic and two different real-world datasets show the proposed model outperforms the state-of-the-art models. The proposed model provides individualized treatment decisions for mechanical ventilation that could improve patient outcomes.

Related papers

• Assessing Reusability of Deep Learning-Based Monotherapy Drug Response Prediction Models Trained with Omics Data [43.57729817547386]
  Cancer drug response prediction models present a promising approach towards precision oncology. Deep learning (DL) methods have shown great potential in this area. This highlights the need for reusable and adaptable models that can be improved and tested by the wider scientific community.
  arXiv  Detail & Related papers  (2024-09-18T16:08:28Z)
• MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data Augmentation [58.93221876843639]
  This paper introduces a novel, end-to-end diffusion-based risk prediction model, named MedDiffusion. It enhances risk prediction performance by creating synthetic patient data during training to enlarge sample space. It discerns hidden relationships between patient visits using a step-wise attention mechanism, enabling the model to automatically retain the most vital information for generating high-quality data.
  arXiv  Detail & Related papers  (2023-10-04T01:36:30Z)
• TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic Tree-Based Memory Network [54.332862955411656]
  Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment. In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials. We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
  arXiv  Detail & Related papers  (2023-07-19T12:35:09Z)
• Deep Attention Q-Network for Personalized Treatment Recommendation [1.6631602844999724]
  We propose the Deep Attention Q-Network for personalized treatment recommendations. The Transformer architecture within a deep reinforcement learning framework efficiently incorporates all past patient observations. We evaluated the model on real-world sepsis and acute hypotension cohorts, demonstrating its superiority to state-of-the-art models.
  arXiv  Detail & Related papers  (2023-07-04T07:00:19Z)
• Policy Optimization for Personalized Interventions in Behavioral Health [8.10897203067601]
  Behavioral health interventions, delivered through digital platforms, have the potential to significantly improve health outcomes. We study the problem of optimizing personalized interventions for patients to maximize a long-term outcome. We present a new approach for this problem that we dub DecompPI, which decomposes the state space for a system of patients to the individual level.
  arXiv  Detail & Related papers  (2023-03-21T21:42:03Z)
• Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approach [0.0]
  Dynamic Treatment Regimes (DTRs) are widely studied to formalize this process. We develop Reinforcement Learning methods to efficiently learn optimal treatment regimes.
  arXiv  Detail & Related papers  (2021-12-08T20:22:04Z)
• Deep Bayesian Estimation for Dynamic Treatment Regimes with a Long Follow-up Time [28.11470886127216]
  Causal effect estimation for dynamic treatment regimes (DTRs) contributes to sequential decision making. We combine outcome regression models with treatment models for high dimensional features using uncensored subjects that are small in sample size. Also, the developed deep Bayesian models can model uncertainty and output the prediction variance which is essential for the safety-aware applications, such as self-driving cars and medical treatment design.
  arXiv  Detail & Related papers  (2021-09-20T13:21:39Z)
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records [57.75125067744978]
  We propose a data augmentation method to facilitate domain adaptation. adversarially generated samples are used during domain adaptation. Results confirm the effectiveness of our method and the generality on different tasks.
  arXiv  Detail & Related papers  (2021-01-13T03:20:20Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)
• Hemogram Data as a Tool for Decision-making in COVID-19 Management: Applications to Resource Scarcity Scenarios [62.997667081978825]
  COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure. This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients. Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
  arXiv  Detail & Related papers  (2020-05-10T01:45:03Z)
• DTR Bandit: Learning to Make Response-Adaptive Decisions With Low Regret [59.81290762273153]
  Dynamic treatment regimes (DTRs) are personalized, adaptive, multi-stage treatment plans that adapt treatment decisions to an individual's initial features and to intermediate outcomes and features at each subsequent stage. We propose a novel algorithm that, by carefully balancing exploration and exploitation, is guaranteed to achieve rate-optimal regret when the transition and reward models are linear.
  arXiv  Detail & Related papers  (2020-05-06T13:03:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.