Trajectory Inspection: A Method for Iterative Clinician-Driven Design of Reinforcement Learning Studies

• URL: http://arxiv.org/abs/2010.04279v2
• Date: Mon, 21 Dec 2020 19:28:25 GMT
• Title: Trajectory Inspection: A Method for Iterative Clinician-Driven Design of Reinforcement Learning Studies
• Authors: Christina X. Ji, Michael Oberst, Sanjat Kanjilal, David Sontag
• Abstract summary: We highlight a simple approach, trajectory inspection, to bring clinicians into an iterative design process for model-based RL studies. We identify where the model recommends unexpectedly aggressive treatments or expects surprisingly positive outcomes from its recommendations.
• Score: 5.5302127686575435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning (RL) has the potential to significantly improve clinical decision making. However, treatment policies learned via RL from observational data are sensitive to subtle choices in study design. We highlight a simple approach, trajectory inspection, to bring clinicians into an iterative design process for model-based RL studies. We identify where the model recommends unexpectedly aggressive treatments or expects surprisingly positive outcomes from its recommendations. Then, we examine clinical trajectories simulated with the learned model and policy alongside the actual hospital course. Applying this approach to recent work on RL for sepsis management, we uncover a model bias towards discharge, a preference for high vasopressor doses that may be linked to small sample sizes, and clinically implausible expectations of discharge without weaning off vasopressors. We hope that iterations of detecting and addressing the issues unearthed by our method will result in RL policies that inspire more confidence in deployment.

Related papers

• OMG-RL:Offline Model-based Guided Reward Learning for Heparin Treatment [0.4998632546280975]
  This study focuses on developing a reward function that reflects the clinician's intentions. We learn a parameterized reward function that includes the expert's intentions from limited data. This approach can be broadly utilized not only for the heparin dosing problem but also for RL-based medication dosing tasks in general.
  arXiv  Detail & Related papers  (2024-09-20T07:51:37Z)
• Exploiting Causal Graph Priors with Posterior Sampling for Reinforcement Learning [86.22660674919746]
  Posterior sampling allows exploitation of prior knowledge on the environment's transition dynamics. We propose a novel posterior sampling approach in which the prior is given as a causal graph over the environment's variables.
  arXiv  Detail & Related papers  (2023-10-11T14:16:04Z)
• Multitask Deep Learning for Accurate Risk Stratification and Prediction of Next Steps for Coronary CT Angiography Patients [26.50934421749854]
  We propose a multi-task deep learning model to support risk stratification and down-stream test selection. Our model achieved an Area Under the receiver operating characteristic Curve (AUC) of 0.76 in CAD risk stratification, and 0.72 AUC in predicting downstream tests.
  arXiv  Detail & Related papers  (2023-09-01T08:34:13Z)
• Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Care [46.2482873419289]
  We introduce a deep Q-learning approach to obtain more reliable critical care policies. We evaluate our method in off-policy and offline settings using simulated environments and real health records from intensive care units.
  arXiv  Detail & Related papers  (2023-06-13T18:02:57Z)
• Provable Reward-Agnostic Preference-Based Reinforcement Learning [61.39541986848391]
  Preference-based Reinforcement Learning (PbRL) is a paradigm in which an RL agent learns to optimize a task using pair-wise preference-based feedback over trajectories. We propose a theoretical reward-agnostic PbRL framework where exploratory trajectories that enable accurate learning of hidden reward functions are acquired.
  arXiv  Detail & Related papers  (2023-05-29T15:00:09Z)
• Deep learning methods for drug response prediction in cancer: predominant and emerging trends [50.281853616905416]
  Exploiting computational predictive models to study and treat cancer holds great promise in improving drug development and personalized design of treatment plans. A wave of recent papers demonstrates promising results in predicting cancer response to drug treatments while utilizing deep learning methods. This review allows to better understand the current state of the field and identify major challenges and promising solution paths.
  arXiv  Detail & Related papers  (2022-11-18T03:26:31Z)
• Reinforcement Learning For Survival, A Clinically Motivated Method For Critically Ill Patients [0.0]
  We propose a clinically motivated control objective for critically ill patients, for which the value functions have a simple medical interpretation. We experiment on a large cohort and show that our method produces results consistent with clinical knowledge.
  arXiv  Detail & Related papers  (2022-07-17T00:06:09Z)
• Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability [82.29775890542967]
  Estimating personalized effects of treatments is a complex, yet pervasive problem. Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools. We use post-hoc feature importance methods to identify features that influence the model's predictions.
  arXiv  Detail & Related papers  (2022-06-16T17:59:05Z)
• Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approach [0.0]
  Dynamic Treatment Regimes (DTRs) are widely studied to formalize this process. We develop Reinforcement Learning methods to efficiently learn optimal treatment regimes.
  arXiv  Detail & Related papers  (2021-12-08T20:22:04Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)
• Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation [2.908482270923597]
  Our aim is to establish a framework where reinforcement learning (RL) of optimizing interventions retrospectively allows us a regulatory compliant pathway to prospective clinical testing of the learned policies. We focus on infections in intensive care units which are one of the major causes of death and difficult to treat because of the complex and opaque patient dynamics.
  arXiv  Detail & Related papers  (2020-03-13T20:31:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.