Perceptive Behavior Foundation Model: Adapting Human Motion Priors to Robot-Centric Terrain

• URL: http://arxiv.org/abs/2606.08059v2
• Date: Mon, 15 Jun 2026 15:19:43 GMT
• Title: Perceptive Behavior Foundation Model: Adapting Human Motion Priors to Robot-Centric Terrain
• Abstract summary: We develop a terrain-aware humanoid control framework that grounds human motion priors in robot-centric perception.<n>We train a blind adaptedreference teacher and transfer its terrain-conformal behavior to a deployed raw-reference student.<n>The student is an identity-gated Transformer tracker whose terrain features enter through residual pathways to preserve the motion-tracking prior and trained to produce local corrections only when needed.
• Score: 25.92545023668694
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humanoid behavior foundation models aim to acquire reusable whole-body control policies from broad human motion priors, enabling a single controller to produce diverse and expressive behaviors. However, existing motion-centric foundation policies largely assume that the reference motion is already physically compatible with the robot's surroundings. This assumption breaks when the demonstrator, operator, and robot inhabit different environments: a human motion may specify the intended behavior, but not the footholds, clearance, body height, or contact timing required by the robot's local terrain. We introduce \emph{Perceptive Behavior Foundation Model} (Perceptive BFM), a terrain-aware humanoid control framework that grounds human motion priors in robot-centric perception. The model preserves raw kinematic motion references as the behavioral interface, while using local terrain observations to adapt contacts, posture, and timing. To provide scalable terrain supervision, we develop \emph{terrain-conformal reference synthesis} (TCRS), which converts locomotion-oriented human motion clips into terrain-consistent references through contact-aware foothold construction, foot-geometry-aware swing optimization, support-aware root reconstruction, collision repair, and multi-point inverse kinematics. We then train a blind adapted-reference teacher and transfer its terrain-conformal behavior to a deployed raw-reference student through target-frame action alignment. The student is an identity-gated Transformer tracker whose terrain features enter through residual pathways initialized to preserve the motion-tracking prior and trained to produce local corrections only when needed.

Related papers

• PGMT: Perceptive General Motion Tracking for Humanoid Robots [12.745867492642523]
  PGMT is a pipeline for humanoid robots that learns terrain adaptation from independently selected motion references and terrains.<n>Zero-shot deployment on a Unitree G1 demonstrates robust terrain-adaptive locomotion and whole-body motion execution over real-world terrain.
  arXiv  Detail & Related papers  (2026-09-08T09:57:32Z)
• Unifying Physics-Based Humanoid Interaction with a Context-Conditioned Interaction Prior [20.58647951068921]
  CHIP is a physics-grounded framework for learning reusable humanoid interaction skills from heterogeneous motion data.<n>We first learn physics-based motion-imitation policies that acquire grounded teacher behaviors from heterogeneous interaction data.<n>We then distill these behaviors into a context-conditioned interaction prior that captures reusable motion structure across locomotion and manipulation.
  arXiv  Detail & Related papers  (2026-09-06T13:16:48Z)
• EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal [63.906929035961134]
  We open-source a reconstruction pipeline capturing 55 scene-aligned 4D human motion sequences in diverse, complex environments.<n>The resulting dataset comprises over 150k frames, which we evaluate against motion-capture ground truth.<n>Our pipeline enables community-driven dataset extensions and factors the problem to help researchers build context-aware robots.
  arXiv  Detail & Related papers  (2026-07-15T06:02:41Z)
• GRAIL: Generating Humanoid Loco-Manipulation from 3D Assets and Video Priors [113.71148915419246]
  We present GRAIL, a digital generation pipeline that composes 3D assets, simulator-ready scenes, and priors from video foundation models (VFMs) to synthesize interactions without rebuilding physical environments or teleoperating the robot.<n>GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction.<n>We retarget the recovered motions to a humanoid robot and train complementary task-general morphology trackers.<n>GRAIL produces over 20,000 sequences spanning pick-up, object manipulation, sitting
  arXiv  Detail & Related papers  (2026-06-03T17:57:45Z)
• Learning Whole-Body Humanoid Locomotion via Motion Generation and Motion Tracking [22.8041193953823]
  Whole-body humanoid locomotion is challenging due to high-dimensional control, morphological instability, and the need for real-time adaptation to various terrains using onboard perception.<n>We propose a whole-body humanoid locomotion framework that combines skills learned from reference motions with terrain-aware adaptation.
  arXiv  Detail & Related papers  (2026-04-19T09:02:55Z)
• PhyGile: Physics-Prefix Guided Motion Generation for Agile General Humanoid Motion Tracking [23.96905373874686]
  We present PhyGile, a unified framework that closes the loop between robot-native motion generation and inference.<n>We train the GMT controller with a curriculum-based mixture-of-experts scheme, followed by post-training on unlabeled motion data to improve robustness over large-scale robot motions.<n>PhyGile expands the frontier of text-driven humanoid control, enabling stable tracking of agile, highly difficult whole-body motions.
  arXiv  Detail & Related papers  (2026-03-13T06:43:44Z)
• Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning [29.898955971414154]
  We formulate the imitation of closely interacting, force-exchanging human-human motion sequences as a multi-agent reinforcement learning problem.<n>We show that AssistMimic is the first method capable of successfully tracking assistive interaction motions on established benchmarks.
  arXiv  Detail & Related papers  (2026-03-11T22:25:44Z)
• OmniTrack: General Motion Tracking via Physics-Consistent Reference [54.256312586493415]
  We introduce OmniTrack, a general tracking framework that explicitly decouples physical feasibility from general motion tracking.<n>Experiments show that OmniTrack improves tracking accuracy and demonstrates strong generalization to unseen motions.<n>In real-world tests, OmniTrack achieves hour-long, consistent, and stable tracking, including complex acrobatic motions such as flips and cartwheels.
  arXiv  Detail & Related papers  (2026-02-27T09:12:15Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• From Language to Locomotion: Retargeting-free Humanoid Control via Motion Latent Guidance [55.31807046722006]
  Existing language-guided humanoid pipelines are cumbersome and untrustworthy.<n>We present RoboGhost, a language-free framework that conditions humanoid policies on language-grounded motion latents.<n>We show that RoboGhost substantially reduces deployment latency, improves success rates and tracking precision, and produces smooth, semantically aligned humanoids.
  arXiv  Detail & Related papers  (2025-10-16T17:57:47Z)
• ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learning [59.64325421657381]
  Humanoid whole-body loco-manipulation promises transformative capabilities for daily service and warehouse tasks.<n>We introduce ResMimic, a two-stage residual learning framework for precise and expressive humanoid control from human motion data.<n>Results show substantial gains in task success, training efficiency, and robustness over strong baselines.
  arXiv  Detail & Related papers  (2025-10-06T17:47:02Z)
• OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction [76.44108003274955]
  A dominant paradigm for teaching humanoid robots complex skills is to retarget human motions as kinematic references to train reinforcement learning policies.<n>We introduce OmniRetarget, an interaction-preserving data generation engine based on an interaction mesh.<n>By minimizing the Laplacian deformation between the human and robot meshes, OmniRetarget generates kinematically feasible trajectories.
  arXiv  Detail & Related papers  (2025-09-30T17:59:02Z)
• Learning Humanoid Standing-up Control across Diverse Postures [27.79222176982376]
  Standing-up control is crucial for humanoid robots, with the potential for integration into current locomotion and loco-manipulation systems.<n>We present HoST (Humanoid Standing-up Control), a reinforcement learning framework that learns standing-up control from scratch.<n>Our experimental results demonstrate that the controllers achieve smooth, stable, and robust standing-up motions across a wide range of laboratory and outdoor environments.
  arXiv  Detail & Related papers  (2025-02-12T13:10:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.