Towards Safe Mechanical Ventilation Treatment Using Deep Offline Reinforcement Learning

• URL: http://arxiv.org/abs/2210.02552v1
• Date: Wed, 5 Oct 2022 20:41:17 GMT
• Title: Towards Safe Mechanical Ventilation Treatment Using Deep Offline Reinforcement Learning
• Authors: Flemming Kondrup, Thomas Jiralerspong, Elaine Lau, Nathan de Lara, Jacob Shkrob, My Duc Tran, Doina Precup, Sumana Basu
• Abstract summary: DeepVent is a Conservative Q-Learning (CQL) based offline Deep Reinforcement Learning (DRL) agent that learns to predict the optimal ventilator parameters for a patient to promote 90 day survival. We find that DeepVent recommends ventilation parameters within safe ranges, as outlined in recent clinical trials. The CQL algorithm offers additional safety by mitigating the overestimation of the value estimates of out-of-distribution states/actions.
• Score: 35.10140674005337
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mechanical ventilation is a key form of life support for patients with pulmonary impairment. Healthcare workers are required to continuously adjust ventilator settings for each patient, a challenging and time consuming task. Hence, it would be beneficial to develop an automated decision support tool to optimize ventilation treatment. We present DeepVent, a Conservative Q-Learning (CQL) based offline Deep Reinforcement Learning (DRL) agent that learns to predict the optimal ventilator parameters for a patient to promote 90 day survival. We design a clinically relevant intermediate reward that encourages continuous improvement of the patient vitals as well as addresses the challenge of sparse reward in RL. We find that DeepVent recommends ventilation parameters within safe ranges, as outlined in recent clinical trials. The CQL algorithm offers additional safety by mitigating the overestimation of the value estimates of out-of-distribution states/actions. We evaluate our agent using Fitted Q Evaluation (FQE) and demonstrate that it outperforms physicians from the MIMIC-III dataset.

Related papers

• Machine learning-based algorithms for at-home respiratory disease monitoring and respiratory assessment [45.104212062055424]
  This work aims to develop machine learning-based algorithms to facilitate at-home respiratory disease monitoring and assessment. Data were collected from 30 healthy adults, encompassing respiratory pressure, flow, and dynamic thoraco-abdominal circumferential measurements. Various machine learning models, including the random forest classifier, logistic regression, and support vector machine (SVM), were trained to predict breathing types.
  arXiv  Detail & Related papers  (2024-09-05T02:14:31Z)
• Methodology for Interpretable Reinforcement Learning for Optimizing Mechanical Ventilation [2.3349787245442966]
  This paper proposes a methodology for interpretable reinforcement learning using decision trees for mechanical ventilation control. Numerical experiments using MIMIC-III data on the stays of real patients' intensive care unit stays demonstrate that the decision tree policy outperforms the behavior cloning policy.
  arXiv  Detail & Related papers  (2024-04-03T23:07:24Z)
• Adaptive Multi-Agent Deep Reinforcement Learning for Timely Healthcare Interventions [17.405080523382235]
  We propose a novel AI-driven patient monitoring framework using multi-agent deep reinforcement learning (DRL) Our approach deploys multiple learning agents, each dedicated to monitoring a specific physiological feature, such as heart rate, respiration, and temperature. We evaluate the performance of the proposed multi-agent DRL framework using real-world physiological and motion data from two datasets.
  arXiv  Detail & Related papers  (2023-09-20T00:42:08Z)
• Deep Attention Q-Network for Personalized Treatment Recommendation [1.6631602844999724]
  We propose the Deep Attention Q-Network for personalized treatment recommendations. The Transformer architecture within a deep reinforcement learning framework efficiently incorporates all past patient observations. We evaluated the model on real-world sepsis and acute hypotension cohorts, demonstrating its superiority to state-of-the-art models.
  arXiv  Detail & Related papers  (2023-07-04T07:00:19Z)
• Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Care [46.2482873419289]
  We introduce a deep Q-learning approach to obtain more reliable critical care policies. We evaluate our method in off-policy and offline settings using simulated environments and real health records from intensive care units.
  arXiv  Detail & Related papers  (2023-06-13T18:02:57Z)
• U-PASS: an Uncertainty-guided deep learning Pipeline for Automated Sleep Staging [61.6346401960268]
  We propose a machine learning pipeline called U-PASS tailored for clinical applications that incorporates uncertainty estimation at every stage of the process. We apply our uncertainty-guided deep learning pipeline to the challenging problem of sleep staging and demonstrate that it systematically improves performance at every stage.
  arXiv  Detail & Related papers  (2023-06-07T08:27:36Z)
• Optimal discharge of patients from intensive care via a data-driven policy learning framework [58.720142291102135]
  It is important that the patient discharge task addresses the nuanced trade-off between decreasing a patient's length of stay and the risk of readmission or even death following the discharge decision. This work introduces an end-to-end general framework for capturing this trade-off to recommend optimal discharge timing decisions. A data-driven approach is used to derive a parsimonious, discrete state space representation that captures a patient's physiological condition.
  arXiv  Detail & Related papers  (2021-12-17T04:39:33Z)
• Machine Learning for Mechanical Ventilation Control (Extended Abstract) [52.65490904484772]
  Mechanical ventilation is one of the most widely used therapies in the ICU. We frame these as a control problem: ventilators must let air in and out of the patient's lungs according to a prescribed trajectory of airway pressure. Our data-driven approach learns to control an invasive ventilator by training on a simulator itself trained on data collected from the ventilator. This method outperforms popular reinforcement learning algorithms and even controls the physical ventilator more accurately and robustly than PID.
  arXiv  Detail & Related papers  (2021-11-19T20:54:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.