An Improved Strategy for Blood Glucose Control Using Multi-Step Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2403.07566v2
• Date: Fri, 15 Mar 2024 09:48:34 GMT
• Title: An Improved Strategy for Blood Glucose Control Using Multi-Step Deep Reinforcement Learning
• Authors: Weiwei Gu, Senquan Wang,
• Abstract summary: Blood Glucose (BG) control involves keeping an individual's BG within a healthy range through extracorporeal insulin injections. Recent research has been devoted to exploring individualized and automated BG control approaches. Deep Reinforcement Learning (DRL) shows potential as an emerging approach.
• Score: 3.5757761767474876
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Blood Glucose (BG) control involves keeping an individual's BG within a healthy range through extracorporeal insulin injections is an important task for people with type 1 diabetes. However,traditional patient self-management is cumbersome and risky. Recent research has been devoted to exploring individualized and automated BG control approaches, among which Deep Reinforcement Learning (DRL) shows potential as an emerging approach. In this paper, we use an exponential decay model of drug concentration to convert the formalization of the BG control problem, which takes into account the delay and prolongedness of drug effects, from a PAE-POMDP (Prolonged Action Effect-Partially Observable Markov Decision Process) to a MDP, and we propose a novel multi-step DRL-based algorithm to solve the problem. The Prioritized Experience Replay (PER) sampling method is also used in it. Compared to single-step bootstrapped updates, multi-step learning is more efficient and reduces the influence from biasing targets. Our proposed method converges faster and achieves higher cumulative rewards compared to the benchmark in the same training environment, and improves the time-in-range (TIR), the percentage of time the patient's BG is within the target range, in the evaluation phase. Our work validates the effectiveness of multi-step reinforcement learning in BG control, which may help to explore the optimal glycemic control measure and improve the survival of diabetic patients.

Related papers

• GlucoBench: Curated List of Continuous Glucose Monitoring Datasets with Prediction Benchmarks [0.12564343689544843]
  Continuous glucose monitors (CGM) are small medical devices that measure blood glucose levels at regular intervals. Forecasting of glucose trajectories based on CGM data holds the potential to substantially improve diabetes management.
  arXiv  Detail & Related papers  (2024-10-08T08:01:09Z)
• Blood Glucose Control Via Pre-trained Counterfactual Invertible Neural Networks [3.7217371773133325]
  We propose an introspective reinforcement learning (RL) based on Counterfactual Invertible Neural Networks (CINN) We use the pre-trained CINN as a frozen introspective block of the RL agent, which integrates forward prediction and counterfactual inference to guide the policy updates. We experimentally validate the accuracy and generalization ability of the pre-trained CINN in BG prediction and counterfactual inference for action.
  arXiv  Detail & Related papers  (2024-05-23T01:34:59Z)
• Towards Understanding the Robustness of Diffusion-Based Purification: A Stochastic Perspective [65.10019978876863]
  Diffusion-Based Purification (DBP) has emerged as an effective defense mechanism against adversarial attacks. In this paper, we argue that the inherentity in the DBP process is the primary driver of its robustness.
  arXiv  Detail & Related papers  (2024-04-22T16:10:38Z)
• GARNN: An Interpretable Graph Attentive Recurrent Neural Network for Predicting Blood Glucose Levels via Multivariate Time Series [12.618792803757714]
  We propose interpretable graph attentive neural networks (GARNNs) to model multi-modal data. GARNNs achieve the best prediction accuracy and provide high-quality temporal interpretability. These findings underline the potential of GARNN as a robust tool for improving diabetes care.
  arXiv  Detail & Related papers  (2024-02-26T01:18:53Z)
• Basal-Bolus Advisor for Type 1 Diabetes (T1D) Patients Using Multi-Agent Reinforcement Learning (RL) Methodology [0.0]
  This paper presents a novel multi-agent reinforcement learning (RL) approach for personalized glucose control in individuals with type 1 diabetes (T1D) The method employs a closed-loop system consisting of a blood glucose (BG) metabolic model and a multi-agent soft actor-critic RL model acting as the basal-bolus advisor. Results demonstrate that the RL-based basal-bolus advisor significantly improves glucose control, reducing glycemic variability and increasing time spent within the target range.
  arXiv  Detail & Related papers  (2023-07-17T23:50:51Z)
• Hybrid Control Policy for Artificial Pancreas via Ensemble Deep Reinforcement Learning [13.783833824324333]
  We propose a hybrid control policy for the artificial pancreas (HyCPAP) to address the challenges of closed-loop glucose control. We conduct extensive experiments using the FDA-accepted UVA/Padova T1DM simulator. Our approaches achieve the highest percentage of time spent in the desired euglycemic range and the lowest occurrences of hypoglycemia.
  arXiv  Detail & Related papers  (2023-07-13T00:53:09Z)
• Automatic diagnosis of knee osteoarthritis severity using Swin transformer [55.01037422579516]
  Knee osteoarthritis (KOA) is a widespread condition that can cause chronic pain and stiffness in the knee joint. We propose an automated approach that employs the Swin Transformer to predict the severity of KOA.
  arXiv  Detail & Related papers  (2023-07-10T09:49:30Z)
• How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval [80.54532535622988]
  We show that a generalizable dense retriever can be trained to achieve high accuracy in both supervised and zero-shot retrieval. DRAGON, our dense retriever trained with diverse augmentation, is the first BERT-base-sized DR to achieve state-of-the-art effectiveness in both supervised and zero-shot evaluations.
  arXiv  Detail & Related papers  (2023-02-15T03:53:26Z)
• SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity Prediction [127.43571146741984]
  Drug-Target Affinity (DTA) is of vital importance in early-stage drug discovery. wet experiments remain the most reliable method, but they are time-consuming and resource-intensive. Existing methods have primarily focused on developing techniques based on the available DTA data, without adequately addressing the data scarcity issue. We present the SSM-DTA framework, which incorporates three simple yet highly effective strategies.
  arXiv  Detail & Related papers  (2022-06-20T14:53:25Z)
• Persistent Reinforcement Learning via Subgoal Curricula [114.83989499740193]
  Value-accelerated Persistent Reinforcement Learning (VaPRL) generates a curriculum of initial states. VaPRL reduces the interventions required by three orders of magnitude compared to episodic reinforcement learning.
  arXiv  Detail & Related papers  (2021-07-27T16:39:45Z)
• Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations [88.94162416324505]
  A deep reinforcement learning (DRL) agent observes its states through observations, which may contain natural measurement errors or adversarial noises. Since the observations deviate from the true states, they can mislead the agent into making suboptimal actions. We show that naively applying existing techniques on improving robustness for classification tasks, like adversarial training, is ineffective for many RL tasks.
  arXiv  Detail & Related papers  (2020-03-19T17:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.