Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care

• URL: http://arxiv.org/abs/2208.07406v1
• Date: Mon, 15 Aug 2022 18:47:09 GMT
• Title: Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care
• Authors: Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A. Murphy
• Abstract summary: Dental disease is one of the most common chronic diseases despite being largely preventable. We develop an online reinforcement learning (RL) algorithm for use in optimizing the delivery of mobile-based prompts to encourage oral hygiene behaviors. The RL algorithm discussed in this paper will be deployed in Oralytics, an oral self-care app that provides behavioral strategies to boost patient engagement in oral hygiene practices.
• Score: 24.283342018185028
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dental disease is one of the most common chronic diseases despite being largely preventable. However, professional advice on optimal oral hygiene practices is often forgotten or abandoned by patients. Therefore patients may benefit from timely and personalized encouragement to engage in oral self-care behaviors. In this paper, we develop an online reinforcement learning (RL) algorithm for use in optimizing the delivery of mobile-based prompts to encourage oral hygiene behaviors. One of the main challenges in developing such an algorithm is ensuring that the algorithm considers the impact of the current action on the effectiveness of future actions (i.e., delayed effects), especially when the algorithm has been made simple in order to run stably and autonomously in a constrained, real-world setting (i.e., highly noisy, sparse data). We address this challenge by designing a quality reward which maximizes the desired health outcome (i.e., high-quality brushing) while minimizing user burden. We also highlight a procedure for optimizing the hyperparameters of the reward by building a simulation environment test bed and evaluating candidates using the test bed. The RL algorithm discussed in this paper will be deployed in Oralytics, an oral self-care app that provides behavioral strategies to boost patient engagement in oral hygiene practices.

Related papers

• Random-Key Algorithms for Optimizing Integrated Operating Room Scheduling [0.16385815610837165]
  This study introduces a novel concept of Random-Key (RKO), rigorously tested on literature and new real-world inspired instances. Our literature optimization problem incorporates multi-room scheduling, equipment scheduling, and complex availability constraints. The RKO approach represents solutions as points in a continuous space, which are then mapped in the problem solution space via a deterministic function known as a decoder.
  arXiv  Detail & Related papers  (2025-01-17T15:11:30Z)
• A Deployed Online Reinforcement Learning Algorithm In An Oral Health Clinical Trial [20.944037982124037]
  Dental disease is a chronic condition associated with substantial financial burden, personal suffering, and increased risk of systemic diseases. Despite widespread recommendations for twice-daily tooth brushing, adherence to recommended oral self-care behaviors remains sub-optimal due to factors such as forgetfulness and disengagement. We developed Oralytics, a mHealth intervention system designed to complement clinician-delivered preventative care for marginalized individuals at risk for dental disease.
  arXiv  Detail & Related papers  (2024-09-03T17:16:01Z)
• Oralytics Reinforcement Learning Algorithm [5.54328512723076]
  Dental disease is one of the most common chronic diseases in the United States. We have developed Oralytics, an online, reinforcement learning (RL) algorithm that optimize the delivery of personalized intervention prompts to improve oral self-care (OSCB) The finalized RL algorithm was deployed in the Oralytics clinical trial, conducted from fall 2023 to summer 2024.
  arXiv  Detail & Related papers  (2024-06-19T00:44:11Z)
• Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials [20.944037982124037]
  This paper proposes algorithm fidelity as a critical requirement for deploying online RL algorithms in clinical trials. We present a framework for pre-deployment planning and real-time monitoring to help algorithm developers and clinical researchers ensure algorithm fidelity.
  arXiv  Detail & Related papers  (2024-02-26T20:19:14Z)
• REBEL: Reward Regularization-Based Approach for Robotic Reinforcement Learning from Human Feedback [61.54791065013767]
  A misalignment between the reward function and human preferences can lead to catastrophic outcomes in the real world. Recent methods aim to mitigate misalignment by learning reward functions from human preferences. We propose a novel concept of reward regularization within the robotic RLHF framework.
  arXiv  Detail & Related papers  (2023-12-22T04:56:37Z)
• Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing [53.096237570992294]
  Strategy training is a rehabilitation approach that teaches skills to reduce disability among those with cognitive impairments following a stroke. Standardized fidelity assessment is used to measure adherence to treatment principles. We developed a rule-based NLP algorithm, a long-short term memory (LSTM) model, and a bidirectional encoder representation from transformers (BERT) model for this task.
  arXiv  Detail & Related papers  (2022-09-14T15:33:30Z)
• Adherence Forecasting for Guided Internet-Delivered Cognitive Behavioral Therapy: A Minimally Data-Sensitive Approach [59.535699822923]
  Internet-delivered psychological treatments (IDPT) are seen as an effective and scalable pathway to improving the accessibility of mental healthcare. This work proposes a deep-learning approach to perform automatic adherence forecasting, while relying on minimally sensitive login/logout data. The proposed Self-Attention Network achieved over 70% average balanced accuracy, when only 1/3 of the treatment duration had elapsed.
  arXiv  Detail & Related papers  (2022-01-11T13:55:57Z)
• Personalized Rehabilitation Robotics based on Online Learning Control [62.6606062732021]
  We propose a novel online learning control architecture, which is able to personalize the control force at run time to each individual user. We evaluate our method in an experimental user study, where the learning controller is shown to provide personalized control, while also obtaining safe interaction forces.
  arXiv  Detail & Related papers  (2021-10-01T15:28:44Z)
• Persistent Reinforcement Learning via Subgoal Curricula [114.83989499740193]
  Value-accelerated Persistent Reinforcement Learning (VaPRL) generates a curriculum of initial states. VaPRL reduces the interventions required by three orders of magnitude compared to episodic reinforcement learning.
  arXiv  Detail & Related papers  (2021-07-27T16:39:45Z)
• Resource Planning for Hospitals Under Special Consideration of the COVID-19 Pandemic: Optimization and Sensitivity Analysis [87.31348761201716]
  Crises like the COVID-19 pandemic pose a serious challenge to health-care institutions. BaBSim.Hospital is a tool for capacity planning based on discrete event simulation. We aim to investigate and optimize these parameters to improve BaBSim.Hospital.
  arXiv  Detail & Related papers  (2021-05-16T12:38:35Z)
• Streamlined Empirical Bayes Fitting of Linear Mixed Models in Mobile Health [3.8974425658660596]
  A mobile health (mHealth) application designed to increase physical activity must make contextually relevant suggestions to motivate users. We propose an algorithm which provides users with contextualized and personalized physical activity suggestions. We show improvements over state of the art approaches both in speed and accuracy of up to 99% and 56% respectively.
  arXiv  Detail & Related papers  (2020-03-28T19:57:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.