Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation

• URL: http://arxiv.org/abs/2003.06474v2
• Date: Wed, 18 Mar 2020 19:42:46 GMT
• Title: Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation
• Authors: Luchen Li, Ignacio Albert-Smet, and Aldo A. Faisal
• Abstract summary: Our aim is to establish a framework where reinforcement learning (RL) of optimizing interventions retrospectively allows us a regulatory compliant pathway to prospective clinical testing of the learned policies. We focus on infections in intensive care units which are one of the major causes of death and difficult to treat because of the complex and opaque patient dynamics.
• Score: 2.908482270923597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Our aim is to establish a framework where reinforcement learning (RL) of optimizing interventions retrospectively allows us a regulatory compliant pathway to prospective clinical testing of the learned policies in a clinical deployment. We focus on infections in intensive care units which are one of the major causes of death and difficult to treat because of the complex and opaque patient dynamics, and the clinically debated, highly-divergent set of intervention policies required by each individual patient, yet intensive care units are naturally data rich. In our work, we build on RL approaches in healthcare ("AI Clinicians"), and learn off-policy continuous dosing policy of pharmaceuticals for sepsis treatment using historical intensive care data under partially observable MDPs (POMDPs). POMPDs capture uncertainty in patient state better by taking in all historical information, yielding an efficient representation, which we investigate through ablations. We compensate for the lack of exploration in our retrospective data by evaluating each encountered state with a best-first tree search. We mitigate state distributional shift by optimizing our policy in the vicinity of the clinicians' compound policy. Crucially, we evaluate our model recommendations using not only conventional policy evaluations but a novel framework that incorporates human experts: a model-agnostic pre-clinical evaluation method to estimate the accuracy and uncertainty of clinician's decisions versus our system recommendations when confronted with the same individual patient history ("shadow mode").

Related papers

• Safe and Interpretable Estimation of Optimal Treatment Regimes [54.257304443780434]
  We operationalize a safe and interpretable framework to identify optimal treatment regimes. Our findings support personalized treatment strategies based on a patient's medical history and pharmacological features.
  arXiv  Detail & Related papers  (2023-10-23T19:59:10Z)
• Validating polyp and instrument segmentation methods in colonoscopy through Medico 2020 and MedAI 2021 Challenges [58.32937972322058]
  "Medico automatic polyp segmentation (Medico 2020)" and "MedAI: Transparency in Medical Image (MedAI 2021)" competitions. We present a comprehensive summary and analyze each contribution, highlight the strength of the best-performing methods, and discuss the possibility of clinical translations of such methods into the clinic.
  arXiv  Detail & Related papers  (2023-07-30T16:08:45Z)
• Deep Attention Q-Network for Personalized Treatment Recommendation [1.6631602844999724]
  We propose the Deep Attention Q-Network for personalized treatment recommendations. The Transformer architecture within a deep reinforcement learning framework efficiently incorporates all past patient observations. We evaluated the model on real-world sepsis and acute hypotension cohorts, demonstrating its superiority to state-of-the-art models.
  arXiv  Detail & Related papers  (2023-07-04T07:00:19Z)
• Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Care [68.8204255655161]
  We introduce a deep Q-learning approach able to obtain more reliable critical care policies. We achieve this by first pruning the action set based on all available rewards, and second training a final model based on the sparse main reward but with a restricted action set.
  arXiv  Detail & Related papers  (2023-06-13T18:02:57Z)
• Policy Optimization for Personalized Interventions in Behavioral Health [8.10897203067601]
  Behavioral health interventions, delivered through digital platforms, have the potential to significantly improve health outcomes. We study the problem of optimizing personalized interventions for patients to maximize a long-term outcome. We present a new approach for this problem that we dub DecompPI, which decomposes the state space for a system of patients to the individual level.
  arXiv  Detail & Related papers  (2023-03-21T21:42:03Z)
• Learning Optimal Treatment Strategies for Sepsis Using Offline Reinforcement Learning in Continuous Space [4.031538204818658]
  We propose a new medical decision model based on historical data to help clinicians recommend the best reference option for real-time treatment. Our model combines offline reinforcement learning with deep reinforcement learning to address the problem that traditional reinforcement learning in healthcare cannot interact with the environment.
  arXiv  Detail & Related papers  (2022-06-22T16:17:21Z)
• POETREE: Interpretable Policy Learning with Adaptive Decision Trees [78.6363825307044]
  POETREE is a novel framework for interpretable policy learning. It builds probabilistic tree policies determining physician actions based on patients' observations and medical history. It outperforms the state-of-the-art on real and synthetic medical datasets.
  arXiv  Detail & Related papers  (2022-03-15T16:50:52Z)
• Optimal discharge of patients from intensive care via a data-driven policy learning framework [58.720142291102135]
  It is important that the patient discharge task addresses the nuanced trade-off between decreasing a patient's length of stay and the risk of readmission or even death following the discharge decision. This work introduces an end-to-end general framework for capturing this trade-off to recommend optimal discharge timing decisions. A data-driven approach is used to derive a parsimonious, discrete state space representation that captures a patient's physiological condition.
  arXiv  Detail & Related papers  (2021-12-17T04:39:33Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)
• Learning Individualized Treatment Rules with Estimated Translated Inverse Propensity Score [29.606141542532356]
  In this paper, we focus on learning individualized treatment rules (ITRs) to derive a treatment policy. In our framework, we cast ITRs learning as a contextual bandit problem and minimize the expected risk of the treatment policy. As a long-term goal, our derived policy might eventually lead to better clinical guidelines for the administration of IV and VP.
  arXiv  Detail & Related papers  (2020-07-02T13:13:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.