Characterization of Human Balance through a Reinforcement Learning-based Muscle Controller

• URL: http://arxiv.org/abs/2308.04462v1
• Date: Tue, 8 Aug 2023 01:53:26 GMT
• Title: Characterization of Human Balance through a Reinforcement Learning-based Muscle Controller
• Authors: K\"ubra Akba\c{s}, Carlotta Mummolo, Xianlian Zhou
• Abstract summary: Balance assessment during physical rehabilitation often relies on rubric-oriented battery tests to score a patient's physical capabilities, leading to subjectivity. This study explores the use of the center of mass (COM) state space and presents a promising avenue for monitoring the balance capabilities in humans.
• Score: 1.5469452301122177
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Balance assessment during physical rehabilitation often relies on rubric-oriented battery tests to score a patient's physical capabilities, leading to subjectivity. While some objective balance assessments exist, they are often limited to tracking the center of pressure (COP), which does not fully capture the whole-body postural stability. This study explores the use of the center of mass (COM) state space and presents a promising avenue for monitoring the balance capabilities in humans. We employ a musculoskeletal model integrated with a balance controller, trained through reinforcement learning (RL), to investigate balancing capabilities. The RL framework consists of two interconnected neural networks governing balance recovery and muscle coordination respectively, trained using Proximal Policy Optimization (PPO) with reference state initialization, early termination, and multiple training strategies. By exploring recovery from random initial COM states (position and velocity) space for a trained controller, we obtain the final BR enclosing successful balance recovery trajectories. Comparing the BRs with analytical postural stability limits from a linear inverted pendulum model, we observe a similar trend in successful COM states but more limited ranges in the recoverable areas. We further investigate the effect of muscle weakness and neural excitation delay on the BRs, revealing reduced balancing capability in different regions. Overall, our approach of learning muscular balance controllers presents a promising new method for establishing balance recovery limits and objectively assessing balance capability in bipedal systems, particularly in humans.

Related papers

• Noradrenergic-inspired gain modulation attenuates the stability gap in joint training [44.99833362998488]
  Studies in continual learning have identified a transient drop in performance on mastered tasks when assimilating new ones, known as the stability gap.<n>We argue that it reflects an imbalance between rapid adaptation and robust retention at task boundaries.<n>Inspired by locus coeruleus mediated noradrenergic bursts, we propose uncertainty-modulated gain dynamics.
  arXiv  Detail & Related papers  (2025-07-18T16:34:06Z)
• Bipedal Balance Control with Whole-body Musculoskeletal Standing and Falling Simulations [11.689074741652163]
  Balance control is important for human and bipedal robotic systems.<n>This work offers unique muscle-level insights into human balance dynamics.<n>It could provide a foundation for developing targeted interventions for individuals with balance impairments.
  arXiv  Detail & Related papers  (2025-06-11T04:23:49Z)
• Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learning [47.023972617451044]
  We propose Neuron-level Balance between Stability and Plasticity (NBSP) method. N BSP takes inspiration from the observation that specific neurons are strongly relevant to task-relevant skills. N BSP significantly outperforms existing approaches in balancing stability and plasticity.
  arXiv  Detail & Related papers  (2025-04-09T05:43:30Z)
• Learning Control Policies of Hodgkin-Huxley Neuronal Dynamics [1.629803445577911]
  We approximate the value function offline using a neural network to enable generating controls (stimuli) in real time via the feedback form. Our numerical experiments illustrate the accuracy of our approach for out-of-distribution samples and the robustness to moderate shocks and disturbances in the system.
  arXiv  Detail & Related papers  (2023-11-13T18:53:50Z)
• Scalable kernel balancing weights in a nationwide observational study of hospital profit status and heart attack outcomes [1.9950682531209158]
  We describe a scalable and flexible approach to weighting that integrates a basis expansion in a reproducing kernel Hilbert space with state-of-the-art convex optimization techniques. Specifically, we use the rank-restricted Nystr"om method to efficiently compute a kernel basis for balancing in nearly linear time and space, and then use the specialized first-order alternating direction method of multipliers to rapidly find the optimal weights. We also use this weighting approach to conduct a national study of the relationship between hospital profit status and heart attack outcomes in a comprehensive dataset of 1.27 million patients.
  arXiv  Detail & Related papers  (2023-11-01T15:08:52Z)
• Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective [80.51463286812314]
  Adversarial Training (AT) has become arguably the state-of-the-art algorithm for extracting robust features. AT suffers from severe robust overfitting problems, particularly after learning rate (LR) decay. We show how LR decay breaks the balance between the minimax game by empowering the trainer with a stronger memorization ability.
  arXiv  Detail & Related papers  (2023-10-30T09:00:11Z)
• A Population-Level Analysis of Neural Dynamics in Robust Legged Robots [6.107812768939554]
  We investigate population-level activity of robust robot locomotion controllers. We find that fragile controllers have a higher number of fixed points with unstable directions, resulting in poorer balance when instructed to stand in place. We find evidence that recurrent state dynamics are structured and low-dimensional during walking, which aligns with primate studies.
  arXiv  Detail & Related papers  (2023-06-27T20:41:59Z)
• Towards AI-controlled FES-restoration of arm movements: Controlling for progressive muscular fatigue with Gaussian state-space models [6.320141734801679]
  Reinforcement Learning (RL) emerges as a promising approach to govern customised control rules for different settings. Yet, one remaining challenge of controlling FES systems for RL is unobservable muscle fatigue. We present a method to address the unobservable muscle fatigue issue, allowing our RL controller to achieve higher control performances.
  arXiv  Detail & Related papers  (2023-01-10T14:51:55Z)
• Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing [53.096237570992294]
  Strategy training is a rehabilitation approach that teaches skills to reduce disability among those with cognitive impairments following a stroke. Standardized fidelity assessment is used to measure adherence to treatment principles. We developed a rule-based NLP algorithm, a long-short term memory (LSTM) model, and a bidirectional encoder representation from transformers (BERT) model for this task.
  arXiv  Detail & Related papers  (2022-09-14T15:33:30Z)
• Minimizing Control for Credit Assignment with Strong Feedback [65.59995261310529]
  Current methods for gradient-based credit assignment in deep neural networks need infinitesimally small feedback signals. We combine strong feedback influences on neural activity with gradient-based learning and show that this naturally leads to a novel view on neural network optimization. We show that the use of strong feedback in DFC allows learning forward and feedback connections simultaneously, using a learning rule fully local in space and time.
  arXiv  Detail & Related papers  (2022-04-14T22:06:21Z)
• Towards Balanced Learning for Instance Recognition [149.76724446376977]
  We propose Libra R-CNN, a framework towards balanced learning for instance recognition. It integrates IoU-balanced sampling, balanced feature pyramid, and objective re-weighting, respectively for reducing the imbalance at sample, feature, and objective level.
  arXiv  Detail & Related papers  (2021-08-23T13:40:45Z)
• Persistent Reinforcement Learning via Subgoal Curricula [114.83989499740193]
  Value-accelerated Persistent Reinforcement Learning (VaPRL) generates a curriculum of initial states. VaPRL reduces the interventions required by three orders of magnitude compared to episodic reinforcement learning.
  arXiv  Detail & Related papers  (2021-07-27T16:39:45Z)
• Equilibrium Propagation with Continual Weight Updates [69.87491240509485]
  We propose a learning algorithm that bridges Machine Learning and Neuroscience, by computing gradients closely matching those of Backpropagation Through Time (BPTT) We prove theoretically that, provided the learning rates are sufficiently small, at each time step of the second phase the dynamics of neurons and synapses follow the gradients of the loss given by BPTT. These results bring EP a step closer to biology by better complying with hardware constraints while maintaining its intimate link with backpropagation.
  arXiv  Detail & Related papers  (2020-04-29T14:54:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.