MPC-guided Imitation Learning of Neural Network Policies for the Artificial Pancreas

• URL: http://arxiv.org/abs/2003.01283v1
• Date: Tue, 3 Mar 2020 01:25:45 GMT
• Title: MPC-guided Imitation Learning of Neural Network Policies for the Artificial Pancreas
• Authors: Hongkai Chen, Nicola Paoletti, Scott A. Smolka, Shan Lin
• Abstract summary: We introduce a novel approach to AP control that uses Imitation Learning to synthesize neural-network insulin policies. Such policies are computationally efficient and, by instrumenting MPC at training time with full state information, they can directly map measurements into optimal therapy decisions. We show that our control policies trained under a specific patient model readily generalize (in terms of model parameters and disturbance distributions) to patient cohorts.
• Score: 7.019683407682642
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Even though model predictive control (MPC) is currently the main algorithm for insulin control in the artificial pancreas (AP), it usually requires complex online optimizations, which are infeasible for resource-constrained medical devices. MPC also typically relies on state estimation, an error-prone process. In this paper, we introduce a novel approach to AP control that uses Imitation Learning to synthesize neural-network insulin policies from MPC-computed demonstrations. Such policies are computationally efficient and, by instrumenting MPC at training time with full state information, they can directly map measurements into optimal therapy decisions, thus bypassing state estimation. We apply Bayesian inference via Monte Carlo Dropout to learn policies, which allows us to quantify prediction uncertainty and thereby derive safer therapy decisions. We show that our control policies trained under a specific patient model readily generalize (in terms of model parameters and disturbance distributions) to patient cohorts, consistently outperforming traditional MPC with state estimation.

Related papers

• Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments [50.310636905746975]
  Real-world machine learning systems often encounter model performance degradation due to distributional shifts in the underlying data generating process. Existing approaches to addressing shifts, such as concept drift adaptation, are limited by their reason-agnostic nature. We propose self-healing machine learning (SHML) to overcome these limitations.
  arXiv  Detail & Related papers  (2024-10-31T20:05:51Z)
• Dropout MPC: An Ensemble Neural MPC Approach for Systems with Learned Dynamics [0.0]
  We propose a novel sampling-based ensemble neural MPC algorithm that employs the Monte-Carlo dropout technique on the learned system model. The method aims in general at uncertain systems with complex dynamics, where models derived from first principles are hard to infer.
  arXiv  Detail & Related papers  (2024-06-04T17:15:25Z)
• Data-Driven Simulator for Mechanical Circulatory Support with Domain Adversarial Neural Process [15.562905335917408]
  Existing mechanical simulators for MCS rely on oversimplifying assumptions and are insensitive to patient-specific behavior. We use a neural process architecture to capture the probabilistic relationship between MCS pump levels and aortic pressure measurements with uncertainty. Empirical results with an improvement of 19% in non-stationary trend prediction establish DANP as an effective tool for clinicians.
  arXiv  Detail & Related papers  (2024-05-28T19:07:12Z)
• Nonlinear sparse variational Bayesian learning based model predictive control with application to PEMFC temperature control [16.703859991393568]
  This study develops a nonlinear sparse variational Bayesian learning based MPC (NSVB-MPC) for nonlinear systems. Variational inference is used by NSVB-MPC to assess the predictive accuracy and make the necessary corrections to quantify system uncertainty.
  arXiv  Detail & Related papers  (2024-04-15T07:30:26Z)
• Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining [50.00291020618743]
  This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU) Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems.
  arXiv  Detail & Related papers  (2024-04-08T20:02:19Z)
• Policy Optimization for Personalized Interventions in Behavioral Health [8.10897203067601]
  Behavioral health interventions, delivered through digital platforms, have the potential to significantly improve health outcomes. We study the problem of optimizing personalized interventions for patients to maximize a long-term outcome. We present a new approach for this problem that we dub DecompPI, which decomposes the state space for a system of patients to the individual level.
  arXiv  Detail & Related papers  (2023-03-21T21:42:03Z)
• Safe AI for health and beyond -- Monitoring to transform a health service [51.8524501805308]
  We will assess the infrastructure required to monitor the outputs of a machine learning algorithm. We will present two scenarios with examples of monitoring and updates of models.
  arXiv  Detail & Related papers  (2023-03-02T17:27:45Z)
• POETREE: Interpretable Policy Learning with Adaptive Decision Trees [78.6363825307044]
  POETREE is a novel framework for interpretable policy learning. It builds probabilistic tree policies determining physician actions based on patients' observations and medical history. It outperforms the state-of-the-art on real and synthetic medical datasets.
  arXiv  Detail & Related papers  (2022-03-15T16:50:52Z)
• Proximal Reinforcement Learning: Efficient Off-Policy Evaluation in Partially Observed Markov Decision Processes [65.91730154730905]
  In applications of offline reinforcement learning to observational data, such as in healthcare or education, a general concern is that observed actions might be affected by unobserved factors. Here we tackle this by considering off-policy evaluation in a partially observed Markov decision process (POMDP) We extend the framework of proximal causal inference to our POMDP setting, providing a variety of settings where identification is made possible.
  arXiv  Detail & Related papers  (2021-10-28T17:46:14Z)
• Stein Variational Model Predictive Control [130.60527864489168]
  Decision making under uncertainty is critical to real-world, autonomous systems. Model Predictive Control (MPC) methods have demonstrated favorable performance in practice, but remain limited when dealing with complex distributions. We show that this framework leads to successful planning in challenging, non optimal control problems.
  arXiv  Detail & Related papers  (2020-11-15T22:36:59Z)
• Planning as Inference in Epidemiological Models [15.097226158765334]
  We demonstrate how to automate parts of the infectious disease-control policy-making process via performing inference in existing epidemiological models. We illustrate the use of a probabilistic programming language that automates inference in existing simulators.
  arXiv  Detail & Related papers  (2020-03-30T05:10:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.