Monitoring Fidelity of Online Reinforcement Learning Algorithms in
Clinical Trials
- URL: http://arxiv.org/abs/2402.17003v1
- Date: Mon, 26 Feb 2024 20:19:14 GMT
- Title: Monitoring Fidelity of Online Reinforcement Learning Algorithms in
Clinical Trials
- Authors: Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Iris
Yan, Finale Doshi-Velez, Susan A. Murphy
- Abstract summary: This paper proposes algorithm fidelity as a critical requirement for deploying online RL algorithms in clinical trials.
We present a framework for pre-deployment planning and real-time monitoring to help algorithm developers and clinical researchers ensure algorithm fidelity.
- Score: 21.994291859964875
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Online reinforcement learning (RL) algorithms offer great potential for
personalizing treatment for participants in clinical trials. However, deploying
an online, autonomous algorithm in the high-stakes healthcare setting makes
quality control and data quality especially difficult to achieve. This paper
proposes algorithm fidelity as a critical requirement for deploying online RL
algorithms in clinical trials. It emphasizes the responsibility of the
algorithm to (1) safeguard participants and (2) preserve the scientific utility
of the data for post-trial analyses. We also present a framework for
pre-deployment planning and real-time monitoring to help algorithm developers
and clinical researchers ensure algorithm fidelity. To illustrate our
framework's practical application, we present real-world examples from the
Oralytics clinical trial. Since Spring 2023, this trial successfully deployed
an autonomous, online RL algorithm to personalize behavioral interventions for
participants at risk for dental disease.
Related papers
- Oralytics Reinforcement Learning Algorithm [5.54328512723076]
Dental disease is one of the most common chronic diseases in the United States.
We have developed Oralytics, an online, reinforcement learning (RL) algorithm that optimize the delivery of personalized intervention prompts to improve oral self-care (OSCB)
The finalized RL algorithm was deployed in the Oralytics clinical trial, conducted from fall 2023 to summer 2024.
arXiv Detail & Related papers (2024-06-19T00:44:11Z) - Provably Efficient Iterated CVaR Reinforcement Learning with Function
Approximation and Human Feedback [57.6775169085215]
Risk-sensitive reinforcement learning aims to optimize policies that balance the expected reward and risk.
We present a novel framework that employs an Iterated Conditional Value-at-Risk (CVaR) objective under both linear and general function approximations.
We propose provably sample-efficient algorithms for this Iterated CVaR RL and provide rigorous theoretical analysis.
arXiv Detail & Related papers (2023-07-06T08:14:54Z) - PyTrial: Machine Learning Software and Benchmark for Clinical Trial
Applications [49.69824178329405]
PyTrial provides benchmarks and open-source implementations of a series of machine learning algorithms for clinical trial design and operations.
We thoroughly investigate 34 ML algorithms for clinical trials across 6 different tasks, including patient outcome prediction, trial site selection, trial outcome prediction, patient-trial matching, trial similarity search, and synthetic data generation.
PyTrial defines each task through a simple four-step process: data loading, model specification, model training, and model evaluation, all achievable with just a few lines of code.
arXiv Detail & Related papers (2023-06-06T21:19:03Z) - Contextual Bandits with Budgeted Information Reveal [3.861395476387163]
Contextual bandit algorithms are commonly used in digital health to recommend personalized treatments.
To ensure the effectiveness of the treatments, patients are often requested to take actions that have no immediate benefit to them.
We introduce a novel optimization and learning algorithm to address this problem.
arXiv Detail & Related papers (2023-05-29T16:18:28Z) - Did we personalize? Assessing personalization by an online reinforcement
learning algorithm using resampling [9.745543921550748]
Reinforcement learning (RL) can be used to personalize sequences of treatments in digital health to support users in adopting healthier behaviors.
Online RL is a promising data-driven approach for this problem as it learns based on each user's historical responses.
We assess whether the RL algorithm should be included in an optimized'' intervention for real-world deployment.
arXiv Detail & Related papers (2023-04-11T17:20:37Z) - Automated Fidelity Assessment for Strategy Training in Inpatient
Rehabilitation using Natural Language Processing [53.096237570992294]
Strategy training is a rehabilitation approach that teaches skills to reduce disability among those with cognitive impairments following a stroke.
Standardized fidelity assessment is used to measure adherence to treatment principles.
We developed a rule-based NLP algorithm, a long-short term memory (LSTM) model, and a bidirectional encoder representation from transformers (BERT) model for this task.
arXiv Detail & Related papers (2022-09-14T15:33:30Z) - Reward Design For An Online Reinforcement Learning Algorithm Supporting
Oral Self-Care [24.283342018185028]
Dental disease is one of the most common chronic diseases despite being largely preventable.
We develop an online reinforcement learning (RL) algorithm for use in optimizing the delivery of mobile-based prompts to encourage oral hygiene behaviors.
The RL algorithm discussed in this paper will be deployed in Oralytics, an oral self-care app that provides behavioral strategies to boost patient engagement in oral hygiene practices.
arXiv Detail & Related papers (2022-08-15T18:47:09Z) - Federated Offline Reinforcement Learning [55.326673977320574]
We propose a multi-site Markov decision process model that allows for both homogeneous and heterogeneous effects across sites.
We design the first federated policy optimization algorithm for offline RL with sample complexity.
We give a theoretical guarantee for the proposed algorithm, where the suboptimality for the learned policies is comparable to the rate as if data is not distributed.
arXiv Detail & Related papers (2022-06-11T18:03:26Z) - Designing Reinforcement Learning Algorithms for Digital Interventions:
Pre-implementation Guidelines [24.283342018185028]
Online reinforcement learning algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education.
Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints.
We extend the PCS (Predictability, Computability, Stability) framework, a data science framework that incorporates best practices from machine learning and statistics in supervised learning.
arXiv Detail & Related papers (2022-06-08T15:05:28Z) - Auto-FedRL: Federated Hyperparameter Optimization for
Multi-institutional Medical Image Segmentation [48.821062916381685]
Federated learning (FL) is a distributed machine learning technique that enables collaborative model training while avoiding explicit data sharing.
In this work, we propose an efficient reinforcement learning(RL)-based federated hyperparameter optimization algorithm, termed Auto-FedRL.
The effectiveness of the proposed method is validated on a heterogeneous data split of the CIFAR-10 dataset and two real-world medical image segmentation datasets.
arXiv Detail & Related papers (2022-03-12T04:11:42Z) - Personalized Rehabilitation Robotics based on Online Learning Control [62.6606062732021]
We propose a novel online learning control architecture, which is able to personalize the control force at run time to each individual user.
We evaluate our method in an experimental user study, where the learning controller is shown to provide personalized control, while also obtaining safe interaction forces.
arXiv Detail & Related papers (2021-10-01T15:28:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.