Offline Reinforcement Learning for Safer Blood Glucose Control in People with Type 1 Diabetes

• URL: http://arxiv.org/abs/2204.03376v2
• Date: Fri, 5 May 2023 10:04:01 GMT
• Title: Offline Reinforcement Learning for Safer Blood Glucose Control in People with Type 1 Diabetes
• Authors: Harry Emerson, Matthew Guy and Ryan McConville
• Abstract summary: Online reinforcement learning (RL) has been utilised as a method for further enhancing glucose control in diabetes devices. This paper examines the utility of BCQ, CQL and TD3-BC in managing the blood glucose of the 30 virtual patients available within the FDA-approved UVA/Padova glucose dynamics simulator. offline RL can significantly increase time in the healthy blood glucose range from 61.6 +- 0.3% to 65.3 +/- 0.5% when compared to the strongest state-of-art baseline.
• Score: 1.1859913430860336
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The widespread adoption of effective hybrid closed loop systems would represent an important milestone of care for people living with type 1 diabetes (T1D). These devices typically utilise simple control algorithms to select the optimal insulin dose for maintaining blood glucose levels within a healthy range. Online reinforcement learning (RL) has been utilised as a method for further enhancing glucose control in these devices. Previous approaches have been shown to reduce patient risk and improve time spent in the target range when compared to classical control algorithms, but are prone to instability in the learning process, often resulting in the selection of unsafe actions. This work presents an evaluation of offline RL for developing effective dosing policies without the need for potentially dangerous patient interaction during training. This paper examines the utility of BCQ, CQL and TD3-BC in managing the blood glucose of the 30 virtual patients available within the FDA-approved UVA/Padova glucose dynamics simulator. When trained on less than a tenth of the total training samples required by online RL to achieve stable performance, this work shows that offline RL can significantly increase time in the healthy blood glucose range from 61.6 +\- 0.3% to 65.3 +/- 0.5% when compared to the strongest state-of-art baseline (p < 0.001). This is achieved without any associated increase in low blood glucose events. Offline RL is also shown to be able to correct for common and challenging control scenarios such as incorrect bolus dosing, irregular meal timings and compression errors.

Related papers

• From Glucose Patterns to Health Outcomes: A Generalizable Foundation Model for Continuous Glucose Monitor Data Analysis [50.80532910808962]
  We present GluFormer, a generative foundation model on biomedical temporal data based on a transformer architecture. GluFormer generalizes to 15 different external datasets, including 4936 individuals across 5 different geographical regions. It can also predict onset of future health outcomes even 4 years in advance.
  arXiv  Detail & Related papers  (2024-08-20T13:19:06Z)
• DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning [61.10299147201369]
  This paper introduces a novel autonomous RL approach, called DigiRL, for training in-the-wild device control agents. We build a scalable and parallelizable Android learning environment equipped with a VLM-based evaluator. We demonstrate the effectiveness of DigiRL using the Android-in-the-Wild dataset, where our 1.3B VLM trained with RL achieves a 49.5% absolute improvement.
  arXiv  Detail & Related papers  (2024-06-14T17:49:55Z)
• Using Reinforcement Learning to Simplify Mealtime Insulin Dosing for People with Type 1 Diabetes: In-Silico Experiments [0.40792653193642503]
  People with type 1 diabetes (T1D) struggle to calculate the optimal insulin dose at mealtime. We propose an RL agent that recommends the optimal meal-accompanying insulin dose corresponding to a qualitative meal (QM) strategy.
  arXiv  Detail & Related papers  (2023-09-17T01:34:02Z)
• Self-Supervised Pretraining Improves Performance and Inference Efficiency in Multiple Lung Ultrasound Interpretation Tasks [65.23740556896654]
  We investigated whether self-supervised pretraining could produce a neural network feature extractor applicable to multiple classification tasks in lung ultrasound analysis. When fine-tuning on three lung ultrasound tasks, pretrained models resulted in an improvement of the average across-task area under the receiver operating curve (AUC) by 0.032 and 0.061 on local and external test sets respectively.
  arXiv  Detail & Related papers  (2023-09-05T21:36:42Z)
• Basal-Bolus Advisor for Type 1 Diabetes (T1D) Patients Using Multi-Agent Reinforcement Learning (RL) Methodology [0.0]
  This paper presents a novel multi-agent reinforcement learning (RL) approach for personalized glucose control in individuals with type 1 diabetes (T1D) The method employs a closed-loop system consisting of a blood glucose (BG) metabolic model and a multi-agent soft actor-critic RL model acting as the basal-bolus advisor. Results demonstrate that the RL-based basal-bolus advisor significantly improves glucose control, reducing glycemic variability and increasing time spent within the target range.
  arXiv  Detail & Related papers  (2023-07-17T23:50:51Z)
• Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies [62.840338941861134]
  We propose to leverage transfer learning from large datasets annotated by radiologists, to predict the histological score available on a small annex dataset. We compare different pretraining methods, namely weakly-supervised and self-supervised ones, to improve the prediction of the cirrhosis. This method outperforms the baseline classification of the METAVIR score, reaching an AUC of 0.84 and a balanced accuracy of 0.75.
  arXiv  Detail & Related papers  (2023-02-16T17:06:23Z)
• Building Brains: Subvolume Recombination for Data Augmentation in Large Vessel Occlusion Detection [56.67577446132946]
  A large training data set is required for a standard deep learning-based model to learn this strategy from data. We propose an augmentation method that generates artificial training samples by recombining vessel tree segmentations of the hemispheres from different patients. In line with the augmentation scheme, we use a 3D-DenseNet fed with task-specific input, fostering a side-by-side comparison between the hemispheres.
  arXiv  Detail & Related papers  (2022-05-05T10:31:57Z)
• LSTMs and Deep Residual Networks for Carbohydrate and Bolus Recommendations in Type 1 Diabetes Management [4.01573226844961]
  We introduce an LSTM-based approach to blood glucose level prediction aimed at "what if" scenarios. We then derive a novel architecture for the same recommendation task. Experimental evaluations using real patient data from the OhioT1DM dataset show that the new integrated architecture compares favorably with the previous LSTM-based approach.
  arXiv  Detail & Related papers  (2021-03-06T19:06:14Z)
• Deep Reinforcement Learning for Closed-Loop Blood Glucose Control [12.989855325491163]
  We develop reinforcement learning techniques for automated blood glucose control. On over 2.1 million hours of data from 30 simulated patients, our RL approach outperforms baseline control algorithms.
  arXiv  Detail & Related papers  (2020-09-18T20:15:02Z)
• Basal Glucose Control in Type 1 Diabetes using Deep Reinforcement Learning: An In Silico Validation [16.93692520921499]
  We propose a novel deep reinforcement learning model for single-hormone (insulin) and dual-hormone (insulin and glucagon) delivery. In the adult cohort, percentage time in target range improved from 77.6% to 80.9% with single-hormone control. In the adolescent cohort, percentage time in target range improved from 55.5% to 65.9% with single-hormone control.
  arXiv  Detail & Related papers  (2020-05-18T20:13:16Z)
• Short Term Blood Glucose Prediction based on Continuous Glucose Monitoring Data [53.01543207478818]
  This study explores the use of Continuous Glucose Monitoring (CGM) data as input for digital decision support tools. We investigate how Recurrent Neural Networks (RNNs) can be used for Short Term Blood Glucose (STBG) prediction.
  arXiv  Detail & Related papers  (2020-02-06T16:39:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.