Embedding Classical Balance Control Principles in Reinforcement Learning for Humanoid Recovery

• URL: http://arxiv.org/abs/2603.08619v1
• Date: Mon, 09 Mar 2026 17:02:30 GMT
• Title: Embedding Classical Balance Control Principles in Reinforcement Learning for Humanoid Recovery
• Abstract summary: Humans are vulnerable to falls and unrecoverable failure states, limiting their practical utility in unstructured environments.<n>We present a unified RL policy that addresses this limitation by embedding classical balance metrics.<n>The policy achieves a 93.4% recovery rate across randomized initial poses and unscripted fall configurations.
• Score: 1.0883174135300417
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humanoid robots remain vulnerable to falls and unrecoverable failure states, limiting their practical utility in unstructured environments. While reinforcement learning has demonstrated stand-up behaviors, existing approaches treat recovery as a pure task-reward problem without an explicit representation of the balance state. We present a unified RL policy that addresses this limitation by embedding classical balance metrics: capture point, center-of-mass state, and centroidal momentum, as privileged critic inputs and shaping rewards directly around these quantities during training, while the actor relies solely on proprioception for zero-shot hardware transfer. Without reference trajectories or scripted contacts, a single policy spans the full recovery spectrum: ankle and hip strategies for small disturbances, corrective stepping under large pushes, and compliant falling with multi-contact stand-up using the hands, elbows, and knees. Trained on the Unitree H1-2 in Isaac Lab, the policy achieves a 93.4% recovery rate across randomized initial poses and unscripted fall configurations. An ablation study shows that removing the balance-informed structure causes stand-up learning to fail entirely, confirming that these metrics provide a meaningful learning signal rather than incidental structure. Sim-to-sim transfer to MuJoCo and preliminary hardware experiments further demonstrate cross-environment generalization. These results show that embedding interpretable balance structure into the learning framework substantially reduces time spent in failure states and broadens the envelope of autonomous recovery.

Related papers

• Calibrated Partial Resets: Preventing Policy Collapse in Continual Reinforcement Learning [3.1681974483553663]
  We propose Calibrated Partial Resets (CPR) to preserve plasticity without destabilizing training.<n>CPR avoids policy collapse over 400M training steps in SlipperyAnt.<n>CPR highlights utility-scaled reinitialization as a promising direction for continual learning.
  arXiv  Detail & Related papers  (2026-07-27T18:53:00Z)
• Stubborn: A Streamlined and Unified Reinforcement Learning Framework for Robust Motion Tracking and Fall Recovery for Humanoids [9.395015640853973]
  We propose Stubborn, a streamlined and unified reinforcement learning framework to achieve robust humanoid motion tracking and fall recovery.<n>Specifically, Stubborn uses an asymmetric Actor-Critic architecture and consists of three major components.<n>For real-world demonstrations, please refer to https://aislab-sustech.io/Stubborn/.
  arXiv  Detail & Related papers  (2026-06-11T02:13:53Z)
• Predictive Style Matching: Natural and Robust Humanoid Locomotion [0.2348805691644085]
  Predictive Style Matching maps the robot's lower-body state history and velocity commands to interpretable upper-body joint and gait targets.<n>On the Unitree G1, in both simulation and hardware, PSM reduces upper-body style error by roughly an order of magnitude over task-only RL.
  arXiv  Detail & Related papers  (2026-06-05T09:21:20Z)
• Adaptive state-action abstractions via rate-distortion [52.25810164390199]
  Reinforcement learning offers multiple techniques for building simple versions of complex tasks.<n>This paper proposes one such principle: refine the abstraction as soon as the learning error within it becomes comparable to the error induced by the abstraction itself.
  arXiv  Detail & Related papers  (2026-06-04T13:10:18Z)
• Delayed Homomorphic Reinforcement Learning for Environments with Delayed Feedback [11.866061471514582]
  Delayed feedback breaks the Markov assumption and impedes learning and control.<n>We propose a framework grounded in MDP homomorphisms that collapses belief-equivalent augmented states.<n> Experiments on continuous control tasks in MuJoCo benchmark confirm that our algorithm outperforms strong augmentation-based baselines.
  arXiv  Detail & Related papers  (2026-04-04T08:38:52Z)
• What Makes Value Learning Efficient in Residual Reinforcement Learning? [57.635661297706065]
  Residual reinforcement learning (RL) enables stable online refinement of expressive pretrained policies by freezing the base and learning only bounded corrections.<n>In this work, we identify two key bottlenecks: cold start pathology, where the critic lacks knowledge of the value landscape around the base policy, and structural scale mismatch.<n>We propose DAWN, a minimal approach targeting efficient value learning in residual RL.
  arXiv  Detail & Related papers  (2026-02-11T05:25:39Z)
• Cerebellar-Inspired Residual Control for Fault Recovery: From Inference-Time Adaptation to Structural Consolidation [2.0421986354783437]
  We introduce an inference-time, cerebellar-inspired residual control framework that augments a frozen reinforcement learning policy with online corrective actions.<n>The framework instantiates core cerebellar principles, including high-dimensional pattern separation via fixed feature expansion.<n>Experiments on MuJoCo benchmarks show improvements of up to $+66%$ on textttHalfCheetah-v5 and $+53%$ on textttHumanoid-v5 under moderate faults.
  arXiv  Detail & Related papers  (2026-02-06T22:16:00Z)
• Stabilizing Reinforcement Learning with LLMs: Formulation and Practices [61.361819972410046]
  We show why and under what conditions the true sequence-level reward can be optimized via a surrogate token-level objective in policy gradient methods such as REINFORCE.<n>This insight provides a principled explanation for the crucial role of several widely adopted techniques in stabilizing RL training.
  arXiv  Detail & Related papers  (2025-12-01T07:45:39Z)
• Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Training [76.12556589212666]
  We show that curriculum post-training avoids the exponential complexity bottleneck.<n>Under outcome-only reward signals, reinforcement learning finetuning achieves high accuracy with sample complexity.<n>We establish guarantees for test-time scaling, where curriculum-aware querying reduces both reward oracle calls and sampling cost from exponential to order.
  arXiv  Detail & Related papers  (2025-11-10T18:29:54Z)
• Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails [103.05296856071931]
  We identify the Alignment Tipping Process (ATP), a critical post-deployment risk unique to self-evolving Large Language Model (LLM) agents.<n>ATP arises when continual interaction drives agents to abandon alignment constraints established during training in favor of reinforced, self-interested strategies.<n>Our experiments show that alignment benefits erode rapidly under self-evolution, with initially aligned models converging toward unaligned states.
  arXiv  Detail & Related papers  (2025-10-06T14:48:39Z)
• Towards Control-Centric Representations in Reinforcement Learning from Images [43.21376020617722]
  ReBis aims to capture control-centric information by integrating reward-free control information alongside reward-specific knowledge. Empirical studies on two large benchmarks, including Atari games and DeepMind Control Suit, demonstrate that ReBis has superior performance compared to existing methods.
  arXiv  Detail & Related papers  (2023-10-25T14:09:53Z)
• Characterization of Human Balance through a Reinforcement Learning-based Muscle Controller [1.5469452301122177]
  Balance assessment during physical rehabilitation often relies on rubric-oriented battery tests to score a patient's physical capabilities, leading to subjectivity. This study explores the use of the center of mass (COM) state space and presents a promising avenue for monitoring the balance capabilities in humans.
  arXiv  Detail & Related papers  (2023-08-08T01:53:26Z)
• Hindsight-DICE: Stable Credit Assignment for Deep Reinforcement Learning [11.084321518414226]
  We adapt existing importance-sampling ratio estimation techniques for off-policy evaluation to drastically improve the stability and efficiency of so-called hindsight policy methods. Our hindsight distribution correction facilitates stable, efficient learning across a broad range of environments where credit assignment plagues baseline methods.
  arXiv  Detail & Related papers  (2023-07-21T20:54:52Z)
• Persistent Reinforcement Learning via Subgoal Curricula [114.83989499740193]
  Value-accelerated Persistent Reinforcement Learning (VaPRL) generates a curriculum of initial states. VaPRL reduces the interventions required by three orders of magnitude compared to episodic reinforcement learning.
  arXiv  Detail & Related papers  (2021-07-27T16:39:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.