Designing and evaluating an online reinforcement learning agent for physical exercise recommendations in N-of-1 trials

• URL: http://arxiv.org/abs/2309.14156v2
• Date: Thu, 23 Nov 2023 19:50:21 GMT
• Title: Designing and evaluating an online reinforcement learning agent for physical exercise recommendations in N-of-1 trials
• Authors: Dominik Meier, Ipek Ensari, Stefan Konigorski
• Abstract summary: We present an innovative N-of-1 trial study design testing whether implementing a personalized intervention by an online reinforcement learning agent is feasible and effective. The results show that, first, implementing a personalized intervention by an online reinforcement learning agent is feasible. Second, such adaptive interventions have the potential to improve patients' benefits even if only few observations are available.
• Score: 0.9865722130817715
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Personalized adaptive interventions offer the opportunity to increase patient benefits, however, there are challenges in their planning and implementation. Once implemented, it is an important question whether personalized adaptive interventions are indeed clinically more effective compared to a fixed gold standard intervention. In this paper, we present an innovative N-of-1 trial study design testing whether implementing a personalized intervention by an online reinforcement learning agent is feasible and effective. Throughout, we use a new study on physical exercise recommendations to reduce pain in endometriosis for illustration. We describe the design of a contextual bandit recommendation agent and evaluate the agent in simulation studies. The results show that, first, implementing a personalized intervention by an online reinforcement learning agent is feasible. Second, such adaptive interventions have the potential to improve patients' benefits even if only few observations are available. As one challenge, they add complexity to the design and implementation process. In order to quantify the expected benefit, data from previous interventional studies is required. We expect our approach to be transferable to other interventions and clinical interventions.

Related papers

• Adaptive Interventions with User-Defined Goals for Health Behavior Change [17.688448640253494]
  Mobile health applications present a promising avenue for low-cost, scalable health behavior change promotion. tailoring advice to a person's unique goals, preferences, and life circumstances is a critical component of health coaching. We introduce a new Thompson sampling algorithm that can accommodate personalized reward functions.
  arXiv  Detail & Related papers  (2023-11-16T01:00:04Z)
• Nonparametric Additive Value Functions: Interpretable Reinforcement Learning with an Application to Surgical Recovery [8.890206493793878]
  We propose a nonparametric additive model for estimating interpretable value functions in reinforcement learning. We validate the proposed approach with a simulation study, and, in an application to spine disease, uncover recovery recommendations that are inline with related clinical knowledge.
  arXiv  Detail & Related papers  (2023-08-25T02:05:51Z)
• A Meta-Learning Method for Estimation of Causal Excursion Effects to Assess Time-Varying Moderation [0.0]
  This paper revisits the estimation of causal excursion effects from a meta-learner perspective. We present the properties of the proposed estimators and compare them both theoretically and through extensive simulations. The results show relative efficiency gains and support the suggestion of a doubly robust alternative to existing methods.
  arXiv  Detail & Related papers  (2023-06-28T15:19:33Z)
• Policy Optimization for Personalized Interventions in Behavioral Health [8.10897203067601]
  Behavioral health interventions, delivered through digital platforms, have the potential to significantly improve health outcomes. We study the problem of optimizing personalized interventions for patients to maximize a long-term outcome. We present a new approach for this problem that we dub DecompPI, which decomposes the state space for a system of patients to the individual level.
  arXiv  Detail & Related papers  (2023-03-21T21:42:03Z)
• Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing [53.096237570992294]
  Strategy training is a rehabilitation approach that teaches skills to reduce disability among those with cognitive impairments following a stroke. Standardized fidelity assessment is used to measure adherence to treatment principles. We developed a rule-based NLP algorithm, a long-short term memory (LSTM) model, and a bidirectional encoder representation from transformers (BERT) model for this task.
  arXiv  Detail & Related papers  (2022-09-14T15:33:30Z)
• Adaptive Identification of Populations with Treatment Benefit in Clinical Trials: Machine Learning Challenges and Solutions [78.31410227443102]
  We study the problem of adaptively identifying patient subpopulations that benefit from a given treatment during a confirmatory clinical trial. We propose AdaGGI and AdaGCPI, two meta-algorithms for subpopulation construction.
  arXiv  Detail & Related papers  (2022-08-11T14:27:49Z)
• Personalized Rehabilitation Robotics based on Online Learning Control [62.6606062732021]
  We propose a novel online learning control architecture, which is able to personalize the control force at run time to each individual user. We evaluate our method in an experimental user study, where the learning controller is shown to provide personalized control, while also obtaining safe interaction forces.
  arXiv  Detail & Related papers  (2021-10-01T15:28:44Z)
• On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy [70.71457102672545]
  We compare the impact of different training procedures for diabetic retinopathy grading. We investigate different aspects such as quantitative performance, statistics of the learned feature representations, interpretability and robustness to image distortions. Our results indicate that models from ImageNet pretraining report a significant increase in performance, generalization and robustness to image distortions.
  arXiv  Detail & Related papers  (2021-06-25T08:32:45Z)
• Semi-Supervised Variational Reasoning for Medical Dialogue Generation [70.838542865384]
  Two key characteristics are relevant for medical dialogue generation: patient states and physician actions. We propose an end-to-end variational reasoning approach to medical dialogue generation. A physician policy network composed of an action-classifier and two reasoning detectors is proposed for augmented reasoning ability.
  arXiv  Detail & Related papers  (2021-05-13T04:14:35Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)
• Improving healthcare access management by predicting patient no-show behaviour [0.0]
  This work develops a Decision Support System (DSS) to support the implementation of strategies to encourage attendance. We assess the effectiveness of different machine learning approaches to improve the accuracy of regression models. In addition to quantifying relationships reported in previous studies, we find that income and neighbourhood crime statistics affect no-show probabilities.
  arXiv  Detail & Related papers  (2020-12-10T14:57:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.