StepCountJITAI: simulation environment for RL with application to physical activity adaptive intervention

• URL: http://arxiv.org/abs/2411.00336v1
• Date: Fri, 01 Nov 2024 03:31:39 GMT
• Title: StepCountJITAI: simulation environment for RL with application to physical activity adaptive intervention
• Authors: Karine Karine, Benjamin M. Marlin,
• Abstract summary: We introduce StepCountJITAI, an RL environment designed to foster research on RL methods. In this paper, we introduce StepCountJITAI, an RL environment designed to foster research on RL methods for adaptive behavioral interventions.
• Score: 9.395236804312496
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The use of reinforcement learning (RL) to learn policies for just-in-time adaptive interventions (JITAIs) is of significant interest in many behavioral intervention domains including improving levels of physical activity. In a messaging-based physical activity JITAI, a mobile health app is typically used to send messages to a participant to encourage engagement in physical activity. In this setting, RL methods can be used to learn what intervention options to provide to a participant in different contexts. However, deploying RL methods in real physical activity adaptive interventions comes with challenges: the cost and time constraints of real intervention studies result in limited data to learn adaptive intervention policies. Further, commonly used RL simulation environments have dynamics that are of limited relevance to physical activity adaptive interventions and thus shed little light on what RL methods may be optimal for this challenging application domain. In this paper, we introduce StepCountJITAI, an RL environment designed to foster research on RL methods that address the significant challenges of policy learning for adaptive behavioral interventions.

Related papers

• Enhancing Adaptive Behavioral Interventions with LLM Inference from Participant-Described States [9.395236804312496]
  We develop a novel physical activity intervention simulation environment that generates text-based state descriptions conditioned on latent state variables.<n>We show that this approach has the potential to significantly improve the performance of online policy learning methods.
  arXiv  Detail & Related papers  (2025-07-05T02:52:51Z)
• Reinforcement Learning for Active Matter [3.152018389781338]
  reinforcement learning (RL) has emerged as a promising framework for addressing the complexities of active matter. This review systematically introduces the integration of RL for guiding and controlling active matter systems. We discuss the use of RL to optimize the navigation, foraging, and locomotion strategies for individual active particles.
  arXiv  Detail & Related papers  (2025-03-30T04:27:17Z)
• Action Mapping for Reinforcement Learning in Continuous Environments with Constraints [4.521631014571241]
  We propose a novel DRL training strategy utilizing action mapping to streamline the learning process. We demonstrate through experiments that action mapping significantly improves training performance in constrained environments.
  arXiv  Detail & Related papers  (2024-12-05T16:42:45Z)
• Data-Efficient Task Generalization via Probabilistic Model-based Meta Reinforcement Learning [58.575939354953526]
  PACOH-RL is a novel model-based Meta-Reinforcement Learning (Meta-RL) algorithm designed to efficiently adapt control policies to changing dynamics. Existing Meta-RL methods require abundant meta-learning data, limiting their applicability in settings such as robotics. Our experiment results demonstrate that PACOH-RL outperforms model-based RL and model-based Meta-RL baselines in adapting to new dynamic conditions.
  arXiv  Detail & Related papers  (2023-11-13T18:51:57Z)
• Stepsize Learning for Policy Gradient Methods in Contextual Markov Decision Processes [35.889129338603446]
  Policy-based algorithms are among the most widely adopted techniques in model-free RL. They tend to struggle when asked to accomplish a series of heterogeneous tasks. We introduce a new formulation, known as meta-MDP, that can be used to solve any hyper parameter selection problem in RL.
  arXiv  Detail & Related papers  (2023-06-13T12:58:12Z)
• Model-Based Reinforcement Learning with Multi-Task Offline Pretraining [59.82457030180094]
  We present a model-based RL method that learns to transfer potentially useful dynamics and action demonstrations from offline data to a novel task. The main idea is to use the world models not only as simulators for behavior learning but also as tools to measure the task relevance. We demonstrate the advantages of our approach compared with the state-of-the-art methods in Meta-World and DeepMind Control Suite.
  arXiv  Detail & Related papers  (2023-06-06T02:24:41Z)
• Human-Inspired Framework to Accelerate Reinforcement Learning [1.6317061277457001]
  Reinforcement learning (RL) is crucial for data science decision-making but suffers from sample inefficiency. This paper introduces a novel human-inspired framework to enhance RL algorithm sample efficiency.
  arXiv  Detail & Related papers  (2023-02-28T13:15:04Z)
• Reinforcement Learning with Stepwise Fairness Constraints [50.538878453547966]
  We introduce the study of reinforcement learning with stepwise fairness constraints. We provide learning algorithms with strong theoretical guarantees in regard to policy optimality and fairness violation.
  arXiv  Detail & Related papers  (2022-11-08T04:06:23Z)
• Jump-Start Reinforcement Learning [68.82380421479675]
  We present a meta algorithm that can use offline data, demonstrations, or a pre-existing policy to initialize an RL policy. In particular, we propose Jump-Start Reinforcement Learning (JSRL), an algorithm that employs two policies to solve tasks. We show via experiments that JSRL is able to significantly outperform existing imitation and reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-04-05T17:25:22Z)
• Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventions [6.9295094033607825]
  reinforcement learning (RL) has acquired a prominent position in health-related sequential decision-making problems. However, its real-life application is still limited and its potential is still to be realized.
  arXiv  Detail & Related papers  (2022-03-04T23:14:02Z)
• Learning Complex Spatial Behaviours in ABM: An Experimental Observational Study [0.0]
  This paper explores how Reinforcement Learning can be applied to create emergent agent behaviours. Running a series of simulations, we demonstrate that agents trained using the novel Proximal Policy optimisation algorithm behave in ways that exhibit properties of real-world intelligent adaptive behaviours.
  arXiv  Detail & Related papers  (2022-01-04T11:56:11Z)
• Autonomous Reinforcement Learning: Formalism and Benchmarking [106.25788536376007]
  Real-world embodied learning, such as that performed by humans and animals, is situated in a continual, non-episodic world. Common benchmark tasks in RL are episodic, with the environment resetting between trials to provide the agent with multiple attempts. This discrepancy presents a major challenge when attempting to take RL algorithms developed for episodic simulated environments and run them on real-world platforms.
  arXiv  Detail & Related papers  (2021-12-17T16:28:06Z)
• A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments [1.713291434132985]
  Reinforcement learning (RL) algorithms find applications in inventory control, recommender systems, vehicular traffic management, cloud computing and robotics. Real-world complications of many tasks arising in these domains makes them difficult to solve with the basic assumptions underlying classical RL algorithms. This paper provides a survey of RL methods developed for handling dynamically varying environment models. A representative collection of these algorithms is discussed in detail in this work along with their categorization and their relative merits and demerits.
  arXiv  Detail & Related papers  (2020-05-19T09:42:42Z)
• Discrete Action On-Policy Learning with Action-Value Critic [72.20609919995086]
  Reinforcement learning (RL) in discrete action space is ubiquitous in real-world applications, but its complexity grows exponentially with the action-space dimension. We construct a critic to estimate action-value functions, apply it on correlated actions, and combine these critic estimated action values to control the variance of gradient estimation. These efforts result in a new discrete action on-policy RL algorithm that empirically outperforms related on-policy algorithms relying on variance control techniques.
  arXiv  Detail & Related papers  (2020-02-10T04:23:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.