Flexible Blood Glucose Control: Offline Reinforcement Learning from Human Feedback

• URL: http://arxiv.org/abs/2501.15972v1
• Date: Mon, 27 Jan 2025 11:31:40 GMT
• Title: Flexible Blood Glucose Control: Offline Reinforcement Learning from Human Feedback
• Authors: Harry Emerson, Sam Gordon James, Matthew Guy, Ryan McConville,
• Abstract summary: Paint is an original RL framework for learning flexible insulin dosing policies from patient records. Labelled data trains a reward model, informing the actions of a novel safety-constrained offline RL algorithm. In-silico evaluation shows Paint achieves common glucose goals through simple labelling of desired states, reducing glycaemic risk by 15% over a commercial benchmark.
• Score: 3.3457851904072595
• License:
• Abstract: Reinforcement learning (RL) has demonstrated success in automating insulin dosing in simulated type 1 diabetes (T1D) patients but is currently unable to incorporate patient expertise and preference. This work introduces PAINT (Preference Adaptation for INsulin control in T1D), an original RL framework for learning flexible insulin dosing policies from patient records. PAINT employs a sketch-based approach for reward learning, where past data is annotated with a continuous reward signal to reflect patient's desired outcomes. Labelled data trains a reward model, informing the actions of a novel safety-constrained offline RL algorithm, designed to restrict actions to a safe strategy and enable preference tuning via a sliding scale. In-silico evaluation shows PAINT achieves common glucose goals through simple labelling of desired states, reducing glycaemic risk by 15% over a commercial benchmark. Action labelling can also be used to incorporate patient expertise, demonstrating an ability to pre-empt meals (+10% time-in-range post-meal) and address certain device errors (-1.6% variance post-error) with patient guidance. These results hold under realistic conditions, including limited samples, labelling errors, and intra-patient variability. This work illustrates PAINT's potential in real-world T1D management and more broadly any tasks requiring rapid and precise preference learning under safety constraints.

Related papers

• Domain Adaptive Diabetic Retinopathy Grading with Model Absence and Flowing Data [45.75724873443564]
  Domain shift poses a significant challenge in clinical applications, e.g., Diabetic Retinopathy grading. We propose a novel approach, Generative Unadversarial ExampleS (GUES), which enables adaptation from a data-centric perspective.
  arXiv  Detail & Related papers  (2024-12-02T07:14:25Z)
• Nurse-in-the-Loop Artificial Intelligence for Precision Management of Type 2 Diabetes in a Clinical Trial Utilizing Transfer-Learned Predictive Digital Twin [5.521385406191426]
  The study developed an online nurse-in-the-loop predictive control (ONLC) model that utilizes a predictive digital twin (PDT) The PDT was trained on participants self-monitoring data (weight, food logs, physical activity, glucose) from the first three months. The ONLC provided the intervention group with individualized feedback and recommendations via text messages.
  arXiv  Detail & Related papers  (2024-01-05T06:38:50Z)
• TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic Tree-Based Memory Network [54.332862955411656]
  Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment. In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials. We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
  arXiv  Detail & Related papers  (2023-07-19T12:35:09Z)
• Hybrid Control Policy for Artificial Pancreas via Ensemble Deep Reinforcement Learning [13.783833824324333]
  We propose a hybrid control policy for the artificial pancreas (HyCPAP) to address the challenges of closed-loop glucose control. We conduct extensive experiments using the FDA-accepted UVA/Padova T1DM simulator. Our approaches achieve the highest percentage of time spent in the desired euglycemic range and the lowest occurrences of hypoglycemia.
  arXiv  Detail & Related papers  (2023-07-13T00:53:09Z)
• Automatic diagnosis of knee osteoarthritis severity using Swin transformer [55.01037422579516]
  Knee osteoarthritis (KOA) is a widespread condition that can cause chronic pain and stiffness in the knee joint. We propose an automated approach that employs the Swin Transformer to predict the severity of KOA.
  arXiv  Detail & Related papers  (2023-07-10T09:49:30Z)
• Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies [62.840338941861134]
  We propose to leverage transfer learning from large datasets annotated by radiologists, to predict the histological score available on a small annex dataset. We compare different pretraining methods, namely weakly-supervised and self-supervised ones, to improve the prediction of the cirrhosis. This method outperforms the baseline classification of the METAVIR score, reaching an AUC of 0.84 and a balanced accuracy of 0.75.
  arXiv  Detail & Related papers  (2023-02-16T17:06:23Z)
• SynthA1c: Towards Clinically Interpretable Patient Representations for Diabetes Risk Stratification [0.5551483435671848]
  Early diagnosis of Type 2 Diabetes Mellitus (T2DM) is crucial to enable timely therapeutic interventions and lifestyle modifications. We show that image-derived phenotypes and physical examination data together can accurately predict diabetes risk.
  arXiv  Detail & Related papers  (2022-09-20T23:39:52Z)
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data [62.29031007761901]
  This paper proposes a novel contrastive regularized clinical classification model. We introduce two unique positive sampling strategies specifically tailored for EHR data. Our framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data.
  arXiv  Detail & Related papers  (2021-04-07T06:02:04Z)
• Controlling Level of Unconsciousness by Titrating Propofol with Deep Reinforcement Learning [5.276232626689567]
  Reinforcement Learning can be used to fit a mapping from patient state to a medication regimen. Deep RL replaces the table with a deep neural network and has been used to learn medication regimens from registry databases.
  arXiv  Detail & Related papers  (2020-08-27T18:47:08Z)
• DTR Bandit: Learning to Make Response-Adaptive Decisions With Low Regret [59.81290762273153]
  Dynamic treatment regimes (DTRs) are personalized, adaptive, multi-stage treatment plans that adapt treatment decisions to an individual's initial features and to intermediate outcomes and features at each subsequent stage. We propose a novel algorithm that, by carefully balancing exploration and exploitation, is guaranteed to achieve rate-optimal regret when the transition and reward models are linear.
  arXiv  Detail & Related papers  (2020-05-06T13:03:42Z)
• Detecting Parkinsonian Tremor from IMU Data Collected In-The-Wild using Deep Multiple-Instance Learning [59.74684475991192]
  Parkinson's Disease (PD) is a slowly evolving neuro-logical disease that affects about 1% of the population above 60 years old. PD symptoms include tremor, rigidity and braykinesia. We present a method for automatically identifying tremorous episodes related to PD, based on IMU signals captured via a smartphone device.
  arXiv  Detail & Related papers  (2020-05-06T09:02:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.