Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials

• URL: http://arxiv.org/abs/2402.17003v2
• Date: Mon, 12 Aug 2024 16:56:11 GMT
• Title: Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials
• Authors: Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Iris Yan, Finale Doshi-Velez, Susan A. Murphy,
• Abstract summary: This paper proposes algorithm fidelity as a critical requirement for deploying online RL algorithms in clinical trials. We present a framework for pre-deployment planning and real-time monitoring to help algorithm developers and clinical researchers ensure algorithm fidelity.
• Score: 20.944037982124037
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online reinforcement learning (RL) algorithms offer great potential for personalizing treatment for participants in clinical trials. However, deploying an online, autonomous algorithm in the high-stakes healthcare setting makes quality control and data quality especially difficult to achieve. This paper proposes algorithm fidelity as a critical requirement for deploying online RL algorithms in clinical trials. It emphasizes the responsibility of the algorithm to (1) safeguard participants and (2) preserve the scientific utility of the data for post-trial analyses. We also present a framework for pre-deployment planning and real-time monitoring to help algorithm developers and clinical researchers ensure algorithm fidelity. To illustrate our framework's practical application, we present real-world examples from the Oralytics clinical trial. Since Spring 2023, this trial successfully deployed an autonomous, online RL algorithm to personalize behavioral interventions for participants at risk for dental disease.

Related papers

• A Deployed Online Reinforcement Learning Algorithm In An Oral Health Clinical Trial [20.944037982124037]
  Dental disease is a chronic condition associated with substantial financial burden, personal suffering, and increased risk of systemic diseases. Despite widespread recommendations for twice-daily tooth brushing, adherence to recommended oral self-care behaviors remains sub-optimal due to factors such as forgetfulness and disengagement. We developed Oralytics, a mHealth intervention system designed to complement clinician-delivered preventative care for marginalized individuals at risk for dental disease.
  arXiv  Detail & Related papers  (2024-09-03T17:16:01Z)
• Effective Monitoring of Online Decision-Making Algorithms in Digital Intervention Implementation [18.712461287884917]
  This paper provides guidelines for enabling effective monitoring of online decision-making algorithms. We discuss our experience in monitoring online decision-making algorithms in two digital intervention clinical trials.
  arXiv  Detail & Related papers  (2024-08-30T15:13:58Z)
• Oralytics Reinforcement Learning Algorithm [5.54328512723076]
  Dental disease is one of the most common chronic diseases in the United States. We have developed Oralytics, an online, reinforcement learning (RL) algorithm that optimize the delivery of personalized intervention prompts to improve oral self-care (OSCB) The finalized RL algorithm was deployed in the Oralytics clinical trial, conducted from fall 2023 to summer 2024.
  arXiv  Detail & Related papers  (2024-06-19T00:44:11Z)
• Provably Efficient Iterated CVaR Reinforcement Learning with Function Approximation and Human Feedback [57.6775169085215]
  Risk-sensitive reinforcement learning aims to optimize policies that balance the expected reward and risk. We present a novel framework that employs an Iterated Conditional Value-at-Risk (CVaR) objective under both linear and general function approximations. We propose provably sample-efficient algorithms for this Iterated CVaR RL and provide rigorous theoretical analysis.
  arXiv  Detail & Related papers  (2023-07-06T08:14:54Z)
• PyTrial: Machine Learning Software and Benchmark for Clinical Trial Applications [49.69824178329405]
  PyTrial provides benchmarks and open-source implementations of a series of machine learning algorithms for clinical trial design and operations. We thoroughly investigate 34 ML algorithms for clinical trials across 6 different tasks, including patient outcome prediction, trial site selection, trial outcome prediction, patient-trial matching, trial similarity search, and synthetic data generation. PyTrial defines each task through a simple four-step process: data loading, model specification, model training, and model evaluation, all achievable with just a few lines of code.
  arXiv  Detail & Related papers  (2023-06-06T21:19:03Z)
• Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling [9.745543921550748]
  Reinforcement learning (RL) can be used to personalize sequences of treatments in digital health to support users in adopting healthier behaviors. Online RL is a promising data-driven approach for this problem as it learns based on each user's historical responses. We assess whether the RL algorithm should be included in an optimized'' intervention for real-world deployment.
  arXiv  Detail & Related papers  (2023-04-11T17:20:37Z)
• Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing [53.096237570992294]
  Strategy training is a rehabilitation approach that teaches skills to reduce disability among those with cognitive impairments following a stroke. Standardized fidelity assessment is used to measure adherence to treatment principles. We developed a rule-based NLP algorithm, a long-short term memory (LSTM) model, and a bidirectional encoder representation from transformers (BERT) model for this task.
  arXiv  Detail & Related papers  (2022-09-14T15:33:30Z)
• Federated Offline Reinforcement Learning [55.326673977320574]
  We propose a multi-site Markov decision process model that allows for both homogeneous and heterogeneous effects across sites. We design the first federated policy optimization algorithm for offline RL with sample complexity. We give a theoretical guarantee for the proposed algorithm, where the suboptimality for the learned policies is comparable to the rate as if data is not distributed.
  arXiv  Detail & Related papers  (2022-06-11T18:03:26Z)
• Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelines [24.283342018185028]
  Online reinforcement learning algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints. We extend the PCS (Predictability, Computability, Stability) framework, a data science framework that incorporates best practices from machine learning and statistics in supervised learning.
  arXiv  Detail & Related papers  (2022-06-08T15:05:28Z)
• Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentation [48.821062916381685]
  Federated learning (FL) is a distributed machine learning technique that enables collaborative model training while avoiding explicit data sharing. In this work, we propose an efficient reinforcement learning(RL)-based federated hyperparameter optimization algorithm, termed Auto-FedRL. The effectiveness of the proposed method is validated on a heterogeneous data split of the CIFAR-10 dataset and two real-world medical image segmentation datasets.
  arXiv  Detail & Related papers  (2022-03-12T04:11:42Z)
• Personalized Rehabilitation Robotics based on Online Learning Control [62.6606062732021]
  We propose a novel online learning control architecture, which is able to personalize the control force at run time to each individual user. We evaluate our method in an experimental user study, where the learning controller is shown to provide personalized control, while also obtaining safe interaction forces.
  arXiv  Detail & Related papers  (2021-10-01T15:28:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.