Controlling Level of Unconsciousness by Titrating Propofol with Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2008.12333v2
• Date: Wed, 9 Sep 2020 14:24:47 GMT
• Title: Controlling Level of Unconsciousness by Titrating Propofol with Deep Reinforcement Learning
• Authors: Gabe Schamberg, Marcus Badgeley, and Emery N. Brown
• Abstract summary: Reinforcement Learning can be used to fit a mapping from patient state to a medication regimen. Deep RL replaces the table with a deep neural network and has been used to learn medication regimens from registry databases.
• Score: 5.276232626689567
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement Learning (RL) can be used to fit a mapping from patient state to a medication regimen. Prior studies have used deterministic and value-based tabular learning to learn a propofol dose from an observed anesthetic state. Deep RL replaces the table with a deep neural network and has been used to learn medication regimens from registry databases. Here we perform the first application of deep RL to closed-loop control of anesthetic dosing in a simulated environment. We use the cross-entropy method to train a deep neural network to map an observed anesthetic state to a probability of infusing a fixed propofol dosage. During testing, we implement a deterministic policy that transforms the probability of infusion to a continuous infusion rate. The model is trained and tested on simulated pharmacokinetic/pharmacodynamic models with randomized parameters to ensure robustness to patient variability. The deep RL agent significantly outperformed a proportional-integral-derivative controller (median absolute performance error 1.7% +/- 0.6 and 3.4% +/- 1.2). Modeling continuous input variables instead of a table affords more robust pattern recognition and utilizes our prior domain knowledge. Deep RL learned a smooth policy with a natural interpretation to data scientists and anesthesia care providers alike.

Related papers

• Regressor-free Molecule Generation to Support Drug Response Prediction [83.25894107956735]
  Conditional generation based on the target IC50 score can obtain a more effective sampling space. Regressor-free guidance combines a diffusion model's score estimation with a regression controller model's gradient based on number labels.
  arXiv  Detail & Related papers  (2024-05-23T13:22:17Z)
• Physical formula enhanced multi-task learning for pharmacokinetics prediction [54.13787789006417]
  A major challenge for AI-driven drug discovery is the scarcity of high-quality data. We develop a formula enhanced mul-ti-task learning (PEMAL) method that predicts four key parameters of pharmacokinetics simultaneously. Our experiments reveal that PEMAL significantly lowers the data demand, compared to typical Graph Neural Networks.
  arXiv  Detail & Related papers  (2024-04-16T07:42:55Z)
• Instance-based Learning with Prototype Reduction for Real-Time Proportional Myocontrol: A Randomized User Study Demonstrating Accuracy-preserving Data Reduction for Prosthetic Embedded Systems [0.0]
  This work presents the design, implementation and validation of learning techniques based on the kNN scheme for gesture detection in prosthetic control. The influence of parameterization and varying proportionality schemes is analyzed, utilizing an eight-channel-sEMG armband.
  arXiv  Detail & Related papers  (2023-08-21T20:15:35Z)
• A Transformer-based Prediction Method for Depth of Anesthesia During Target-controlled Infusion of Propofol and Remifentanil [2.960339091215942]
  We propose a transformer-based method for predicting the depth of anesthesia using drug infusions of propofol and remifentanil. Experimental results show that our proposed method outperforms traditional PK-PD models and previous deep learning methods.
  arXiv  Detail & Related papers  (2023-08-02T13:49:44Z)
• Continuous time recurrent neural networks: overview and application to forecasting blood glucose in the intensive care unit [56.801856519460465]
  Continuous time autoregressive recurrent neural networks (CTRNNs) are a deep learning model that account for irregular observations. We demonstrate the application of these models to probabilistic forecasting of blood glucose in a critical care setting.
  arXiv  Detail & Related papers  (2023-04-14T09:39:06Z)
• Towards Real-World Applications of Personalized Anesthesia Using Policy Constraint Q Learning for Propofol Infusion Control [11.07500653906137]
  Policy Constraint Q-Learning (PCQL) is a data-driven reinforcement learning algorithm for solving the problem of learning anesthesia strategies on real clinical datasets. PCQL is validated by extensive experiments on a real clinical anesthesia dataset.
  arXiv  Detail & Related papers  (2023-03-17T10:05:20Z)
• SOUL: An Energy-Efficient Unsupervised Online Learning Seizure Detection Classifier [68.8204255655161]
  Implantable devices that record neural activity and detect seizures have been adopted to issue warnings or trigger neurostimulation to suppress seizures. For an implantable seizure detection system, a low power, at-the-edge, online learning algorithm can be employed to dynamically adapt to neural signal drifts. SOUL was fabricated in TSMC's 28 nm process occupying 0.1 mm2 and achieves 1.5 nJ/classification energy efficiency, which is at least 24x more efficient than state-of-the-art.
  arXiv  Detail & Related papers  (2021-10-01T23:01:20Z)
• Adaptive conversion of real-valued input into spike trains [91.3755431537592]
  This paper presents a biologically plausible method for converting real-valued input into spike trains for processing with spiking neural networks. The proposed method mimics the adaptive behaviour of retinal ganglion cells and allows input neurons to adapt their response to changes in the statistics of the input.
  arXiv  Detail & Related papers  (2021-04-12T12:33:52Z)
• DeepRite: Deep Recurrent Inverse TreatmEnt Weighting for Adjusting Time-varying Confounding in Modern Longitudinal Observational Data [68.29870617697532]
  We propose Deep Recurrent Inverse TreatmEnt weighting (DeepRite) for time-varying confounding in longitudinal data. DeepRite is shown to recover the ground truth from synthetic data, and estimate unbiased treatment effects from real data.
  arXiv  Detail & Related papers  (2020-10-28T15:05:08Z)
• Estimating Individual Treatment Effects with Time-Varying Confounders [9.784193264717098]
  Estimating individual treatment effect (ITE) from observational data is meaningful and practical in healthcare. Existing work mainly relies on the strong ignorability assumption that no hidden confounders exist. We propose Deep Sequential Weighting (DSW) for estimating ITE with time-varying confounders.
  arXiv  Detail & Related papers  (2020-08-27T02:21:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.