Heracles: Bridging Precise Tracking and Generative Synthesis for General Humanoid Control

• URL: http://arxiv.org/abs/2603.27756v2
• Date: Tue, 31 Mar 2026 07:05:07 GMT
• Title: Heracles: Bridging Precise Tracking and Generative Synthesis for General Humanoid Control
• Abstract summary: Current general controllers predominantly formulate motion control as a rigid reference-tracking problem.<n>Heracles is a novel state-conditioned diffusion that bridges precise motion tracking and generative synthesis.<n>Our framework demonstrates that integrating generative priors into the control loop significantly enhances robustness against extreme perturbations.
• Score: 35.756113194514974
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Achieving general-purpose humanoid control requires a delicate balance between the precise execution of commanded motions and the flexible, anthropomorphic adaptability needed to recover from unpredictable environmental perturbations. Current general controllers predominantly formulate motion control as a rigid reference-tracking problem. While effective in nominal conditions, these trackers often exhibit brittle, non-anthropomorphic failure modes under severe disturbances, lacking the generative adaptability inherent to human motor control. To overcome this limitation, we propose Heracles, a novel state-conditioned diffusion middleware that bridges precise motion tracking and generative synthesis. Rather than relying on rigid tracking paradigms or complex explicit mode-switching, Heracles operates as an intermediary layer between high-level reference motions and low-level physics trackers. By conditioning on the robot's real-time state, the diffusion model implicitly adapts its behavior: it approximates an identity map when the state closely aligns with the reference, preserving zero-shot tracking fidelity. Conversely, when encountering significant state deviations, it seamlessly transitions into a generative synthesizer to produce natural, anthropomorphic recovery trajectories. Our framework demonstrates that integrating generative priors into the control loop not only significantly enhances robustness against extreme perturbations but also elevates humanoid control from a rigid tracking paradigm to an open-ended, generative general-purpose architecture.

Related papers

• CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements [63.40365657068892]
  Transferring human hand demonstrations to robotic grippers has recently emerged as a cost-effective solution for robot learning.<n>Motivated by this gap, we adopt an implicit, data-driven approach guided by fine-grained hand-pose motions.<n>In both simulation and real-robot experiments, our framework enables stable and precise hand-to-gripper transfer of complex spatial manipulations.
  arXiv  Detail & Related papers  (2026-09-07T13:47:22Z)
• Schrödinger Bridges over Kinetic Swarming Models [16.998652690384596]
  We consider finite-horizon minimum-energy collective steering for swarms subject to disturbances.<n>Our objective is to steer the swarm between prescribed endpoint distributions using a state-feedback control.<n>We derive necessary optimality systems with a time-symmetric structure reminiscent of classical Schrdinger bridges.
  arXiv  Detail & Related papers  (2026-08-26T01:34:36Z)
• Cycle-World: Mitigating Error Accumulation in Long-term Video World Models via Reverse-Prediction Cycle Consistency [123.12549538776109]
  Cycle-World is a framework designed for stable and temporally consistent long-video generation.<n>Our approach tackles error drift by enforcing strict temporal reversibility across both the training and inference phases.<n>Experiments on the VBench benchmark demonstrate that Cycle-World's dual-phase synergy significantly mitigates error drift.
  arXiv  Detail & Related papers  (2026-07-13T17:27:13Z)
• ReactiveBFM: Reactive Closed-Loop Motion Planning Towards Universal Humanoid Whole-Body Control [54.267632784131415]
  ReactiveBFM is a real-time-loop planning-control framework for humanoids.<n>It mitigates fatal cumulative exposure bias via a scheduled prefix sampling curriculum.<n>It demonstrates unprecedented physical agility across a vast repertoire of text-conditioned closed-loop motions.
  arXiv  Detail & Related papers  (2026-06-29T14:27:27Z)
• OMG: Omni-Modal Motion Generation for Generalist Humanoid Control [30.236954254902614]
  We argue that the key to general-purpose humanoid control is to build a scalable brain, atop a reactive motion tracking system.<n>We present OMG, which addresses two challenges: acquiring a vast amount of high-quality data to achieve general purpose control, and equipping the generator with the capability to condition on compositional, multi-modal inputs.
  arXiv  Detail & Related papers  (2026-06-09T02:46:14Z)
• RoboForge: Physically Optimized Text-guided Whole-Body Locomotion for Humanoids [20.796118584632904]
  We present a unified latent-driven framework that bridges natural language and whole-body locomotion.<n>Our framework provides a practical path toward deployable text-guided humanoid intelligence.
  arXiv  Detail & Related papers  (2026-03-18T17:02:56Z)
• ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation [55.467742403416175]
  We introduce a physics-driven neural algorithm that translates large-scale motion capture to humanoid embodiments.<n>We learn a unified multimodal controller that supports both dense references and sparse task specifications.<n>Results show that ULTRA generalizes to autonomous, goal-conditioned whole-body loco-manipulation from egocentric perception.
  arXiv  Detail & Related papers  (2026-03-03T18:59:29Z)
• OmniTrack: General Motion Tracking via Physics-Consistent Reference [54.256312586493415]
  We introduce OmniTrack, a general tracking framework that explicitly decouples physical feasibility from general motion tracking.<n>Experiments show that OmniTrack improves tracking accuracy and demonstrates strong generalization to unseen motions.<n>In real-world tests, OmniTrack achieves hour-long, consistent, and stable tracking, including complex acrobatic motions such as flips and cartwheels.
  arXiv  Detail & Related papers  (2026-02-27T09:12:15Z)
• Robust and Generalized Humanoid Motion Tracking [17.58241987932198]
  Learning a general humanoid whole-body controller is challenging because practical reference motions can exhibit noise and inconsistencies after being transferred to the robot domain.<n>We propose a dynamics-conditioned command aggregation framework that uses a causal temporal encoder to summarize recent proprioception and a multi-head cross-attention command encoder to selectively aggregate a context window.<n>The proposed method is evaluated under diverse reference inputs and challenging motion regimes, demonstrating zero-shot transfer to unseen motions as well as robust sim-to-real transfer on a physical humanoid robot.
  arXiv  Detail & Related papers  (2026-01-30T15:27:43Z)
• AdaMorph: Unified Motion Retargeting via Embodiment-Aware Adaptive Transformers [49.796014934503184]
  We present a unified neural framework that enables a single model to adapt human motion to diverse robot morphologies.<n> Experimental results on 12 distinct humanoid robots demonstrate that AdaMorph effectively control across heterogeneous topologies.
  arXiv  Detail & Related papers  (2026-01-12T07:39:38Z)
• AERMANI-Diffusion: Regime-Conditioned Diffusion for Dynamics Learning in Aerial Manipulators [2.376629619475859]
  Aerial manipulators undergo rapid, configuration-dependent changes in inertial coupling forces and aerodynamic forces.<n>We propose a regime-conditioned diffusion framework that models the full distribution of residual forces using a conditional diffusion process and a lightweight temporal encoder.
  arXiv  Detail & Related papers  (2025-12-11T16:10:32Z)
• From Language to Locomotion: Retargeting-free Humanoid Control via Motion Latent Guidance [55.31807046722006]
  Existing language-guided humanoid pipelines are cumbersome and untrustworthy.<n>We present RoboGhost, a language-free framework that conditions humanoid policies on language-grounded motion latents.<n>We show that RoboGhost substantially reduces deployment latency, improves success rates and tracking precision, and produces smooth, semantically aligned humanoids.
  arXiv  Detail & Related papers  (2025-10-16T17:57:47Z)
• BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis [14.331548412833513]
  Mixed reality applications require tracking the user's full-body motion to enable an immersive experience. We propose BoDiffusion -- a generative diffusion model for motion synthesis to tackle this under-constrained reconstruction problem. We present a time and space conditioning scheme that allows BoDiffusion to leverage sparse tracking inputs while generating smooth and realistic full-body motion sequences.
  arXiv  Detail & Related papers  (2023-04-21T16:39:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.