Diffusion Policies with Offline and Inverse Reinforcement Learning for Promoting Physical Activity in Older Adults Using Wearable Sensors

• URL: http://arxiv.org/abs/2509.18433v1
• Date: Mon, 22 Sep 2025 21:36:34 GMT
• Title: Diffusion Policies with Offline and Inverse Reinforcement Learning for Promoting Physical Activity in Older Adults Using Wearable Sensors
• Authors: Chang Liu, Ladda Thiamwong, Yanjie Fu, Rui Xie,
• Abstract summary: offline reinforcement learning (RL) with real-world clinical data is getting increasing attention in AI for healthcare.<n>We introduce Kolmogorov-Arnold Networks and Diffusion Policies for Offline Inverse Reinforcement Learning (KANDI)<n>We evaluate KANDI using wearable activity monitoring data in a two-arm clinical trial from our Physio-feedback Exercise Program (PEER) study.
• Score: 22.781709642461564
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Utilizing offline reinforcement learning (RL) with real-world clinical data is getting increasing attention in AI for healthcare. However, implementation poses significant challenges. Defining direct rewards is difficult, and inverse RL (IRL) struggles to infer accurate reward functions from expert behavior in complex environments. Offline RL also encounters challenges in aligning learned policies with observed human behavior in healthcare applications. To address challenges in applying offline RL to physical activity promotion for older adults at high risk of falls, based on wearable sensor activity monitoring, we introduce Kolmogorov-Arnold Networks and Diffusion Policies for Offline Inverse Reinforcement Learning (KANDI). By leveraging the flexible function approximation in Kolmogorov-Arnold Networks, we estimate reward functions by learning free-living environment behavior from low-fall-risk older adults (experts), while diffusion-based policies within an Actor-Critic framework provide a generative approach for action refinement and efficiency in offline RL. We evaluate KANDI using wearable activity monitoring data in a two-arm clinical trial from our Physio-feedback Exercise Program (PEER) study, emphasizing its practical application in a fall-risk intervention program to promote physical activity among older adults. Additionally, KANDI outperforms state-of-the-art methods on the D4RL benchmark. These results underscore KANDI's potential to address key challenges in offline RL for healthcare applications, offering an effective solution for activity promotion intervention strategies in healthcare.

Related papers

• Causal Reinforcement Learning based Agent-Patient Interaction with Clinical Domain Knowledge [3.2434118923825483]
  Reinforcement Learning (RL) faces significant challenges in adaptive healthcare interventions, such as dementia care, where data is scarce.<n>We present a novel framework called Causal structure-aware Reinforcement Learning (CRL) that explicitly integrates causal discovery and reasoning into policy optimization.
  arXiv  Detail & Related papers  (2025-11-17T22:38:03Z)
• Guardian-regularized Safe Offline Reinforcement Learning for Smart Weaning of Mechanical Circulatory Devices [19.512275639322638]
  We study the sequential decision-making problem for automated weaning of mechanical circulatory support devices in cardiogenic shock patients.<n> offline reinforcement learning has proven to be successful in sequential decision-making tasks.<n>We develop an end-to-end machine learning framework with two key contributions.
  arXiv  Detail & Related papers  (2025-11-08T19:32:31Z)
• Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models [71.34520793462069]
  Unsupervised reinforcement learning (RL) aims at pre-training agents that can solve a wide range of downstream tasks in complex environments.<n>We introduce a novel algorithm regularizing unsupervised RL towards imitating trajectories from unlabeled behavior datasets.<n>We demonstrate the effectiveness of this new approach in a challenging humanoid control problem.
  arXiv  Detail & Related papers  (2025-04-15T10:41:11Z)
• Active Advantage-Aligned Online Reinforcement Learning with Offline Data [56.98480620108727]
  We introduce A3RL, which incorporates a novel confidence aware Active Advantage Aligned sampling strategy.<n>We demonstrate that our method outperforms competing online RL techniques that leverage offline data.
  arXiv  Detail & Related papers  (2025-02-11T20:31:59Z)
• StepCountJITAI: simulation environment for RL with application to physical activity adaptive intervention [9.395236804312496]
  We introduce StepCountJITAI, an RL environment designed to foster research on RL methods. In this paper, we introduce StepCountJITAI, an RL environment designed to foster research on RL methods for adaptive behavioral interventions.
  arXiv  Detail & Related papers  (2024-11-01T03:31:39Z)
• Dyadic Reinforcement Learning [7.105179961841919]
  Mobile health aims to enhance health outcomes by delivering interventions to individuals as they go about their daily life. Dyadic RL is an online reinforcement learning algorithm designed to personalize intervention delivery based on contextual factors and past responses. We demonstrate dyadic RL's empirical performance through simulation studies on both toy scenarios and on a realistic test bed constructed from data collected in a mobile health study.
  arXiv  Detail & Related papers  (2023-08-15T15:43:12Z)
• Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Care [46.2482873419289]
  We introduce a deep Q-learning approach to obtain more reliable critical care policies. We evaluate our method in off-policy and offline settings using simulated environments and real health records from intensive care units.
  arXiv  Detail & Related papers  (2023-06-13T18:02:57Z)
• Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applications [3.770564448216192]
  We introduce a practical and theoretically grounded transition sampling approach to address action imbalance during offline RL training. We perform extensive experiments on two real-world tasks for diabetes and sepsis treatment optimization. Across a range of principled and clinically relevant metrics, we show that our proposed approach enables substantial improvements in expected health outcomes.
  arXiv  Detail & Related papers  (2023-02-15T09:30:57Z)
• On Pathologies in KL-Regularized Reinforcement Learning from Expert Demonstrations [79.49929463310588]
  We show that KL-regularized reinforcement learning with behavioral reference policies can suffer from pathological training dynamics. We show that the pathology can be remedied by non-parametric behavioral reference policies.
  arXiv  Detail & Related papers  (2022-12-28T16:29:09Z)
• Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning [63.53407136812255]
  Offline Reinforcement Learning promises to learn effective policies from previously-collected, static datasets without the need for exploration. Existing Q-learning and actor-critic based off-policy RL algorithms fail when bootstrapping from out-of-distribution (OOD) actions or states. We propose Uncertainty Weighted Actor-Critic (UWAC), an algorithm that detects OOD state-action pairs and down-weights their contribution in the training objectives accordingly.
  arXiv  Detail & Related papers  (2021-05-17T20:16:46Z)
• Reinforcement Learning through Active Inference [62.997667081978825]
  We show how ideas from active inference can augment traditional reinforcement learning approaches. We develop and implement a novel objective for decision making, which we term the free energy of the expected future. We demonstrate that the resulting algorithm successfully exploration and exploitation, simultaneously achieving robust performance on several challenging RL benchmarks with sparse, well-shaped, and no rewards.
  arXiv  Detail & Related papers  (2020-02-28T10:28:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.