Global-Local Attention Decomposition for Terrain Encoding in Humanoid Perceptive Locomotion

• URL: http://arxiv.org/abs/2606.00637v1
• Date: Sat, 30 May 2026 09:23:07 GMT
• Title: Global-Local Attention Decomposition for Terrain Encoding in Humanoid Perceptive Locomotion
• Abstract summary: We propose Global-Local Attention Decomposition (GLAD) for terrain encoding in humanoid locomotion.<n> Experiments demonstrate that GLAD enables reliable locomotion over challenging gaps, stepping stones, and stairs.<n>In real-world deployment on a Unitree G1 humanoid robot using onboard LiDAR, the proposed method achieves robust zero-shot sim-to-real transfer.
• Score: 7.888727249239356
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although reinforcement learning has significantly advanced humanoid locomotion, perceptive policies still struggle on sparse-foothold terrain and constrained environments. Success in these scenarios requires both broad terrain awareness and precise foothold selection, two perceptual roles that conventional encoders often entangle. To address this challenge, we propose Global-Local Attention Decomposition (GLAD) for terrain encoding in humanoid locomotion. Realized by a coarse-to-fine encoder over a robot-centric elevation map, GLAD explicitly separates these objectives: a global attention branch utilizes attention pooling to summarize the surrounding terrain context, while a state-conditioned local attention branch sparsifies and encodes precise foothold-relevant geometry. This explicit attention decomposition prevents the dilution of fine-grained spatial cues while reducing training overhead. Experiments demonstrate that GLAD enables reliable locomotion over challenging gaps, stepping stones, and stairs. Furthermore, the learned policy exhibits emergent terrain-responsive behaviors, autonomously following narrow paths and avoiding obstacles under simple velocity commands without explicit navigation planners. In real-world deployment on a Unitree G1 humanoid robot using onboard LiDAR, the proposed method achieves robust zero-shot sim-to-real transfer across diverse sparse-foothold and obstacle-rich domains.

Related papers

• TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model [49.03909082056646]
  We study the problem of navigating cluttered indoor environments with a humanoid robot.<n>We introduce TANGO, the first whole-body vision-language navigation framework.<n>In simulation experiments, TANGO demonstrates state-of-the-art performance in vision-language navigation.
  arXiv  Detail & Related papers  (2026-09-08T17:59:55Z)
• Mind Your Steps: A General Learning Framework for Accurate Humanoid Foothold Tracking [78.30338456206984]
  We introduce a novel framework for training general-purpose 3D foothold-tracking policies.<n>By dynamically providing footstep support through a goal sampler, this method enables the learned policy to be agnostic to specific terrains.<n>Our new target representation effectively mitigates challenges arising in the real world, such as noisy and inaccurate pose estimation and foot contact estimation.
  arXiv  Detail & Related papers  (2026-06-06T16:44:56Z)
• Meridian: Metric-Semantic Primitive Matching for Cross-View Geo-Localization Beyond Urban Environments [47.53030013880309]
  Successful robot automation requires accurate global localization to support repeatability, task planning, goal specification, and safe operation.<n>We present Meridian, a method for matching high-level metric-semantic primitives across aerial images and ground robot RGB-D camera data.<n>We demonstrate that our algorithm can localize a ground robot across a wide variety of environments, including an autonomous driving dataset, a park and campus area, and a wilderness camp.
  arXiv  Detail & Related papers  (2026-06-04T15:52:40Z)
• TAGA: Terrain-aware Active Gaze Learning for Generalizable Agile Humanoid Locomotion [13.927597120628064]
  We introduce TAGA, a Terrain-aware Active Gaze learning framework for Attention-based humanoid control.<n>Our framework guides the model to learn anticipatory cues and actively attend to specific areas of the height scan.<n>We find that such gaze behaviors can naturally emerge through reinforcement learning alone, without requiring additional supervision or explicit guidance.
  arXiv  Detail & Related papers  (2026-06-04T08:52:56Z)
• ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation [55.467742403416175]
  We introduce a physics-driven neural algorithm that translates large-scale motion capture to humanoid embodiments.<n>We learn a unified multimodal controller that supports both dense references and sparse task specifications.<n>Results show that ULTRA generalizes to autonomous, goal-conditioned whole-body loco-manipulation from egocentric perception.
  arXiv  Detail & Related papers  (2026-03-03T18:59:29Z)
• Pro-HOI: Perceptive Root-guided Humanoid-Object Interaction [57.252599851031505]
  We introduce Perceptive Root-guided Humanoid-Object Interaction, Pro-HOI.<n>Pro-HOI is a generalizable framework for robust humanoid loco-manipulation.<n> Empirical validation on a Unitree G1 robot demonstrates that Pro-HOI significantly outperforms baselines in generalization and robustness.
  arXiv  Detail & Related papers  (2026-03-01T14:19:59Z)
• AME-2: Agile and Generalized Legged Locomotion via Attention-Based Neural Map Encoding [13.367858405827926]
  AME-2 is a unified reinforcement learning framework for agile and generalized locomotion.<n>It incorporates a novel attention-based map encoder in the control policy.<n>The pipeline also integrates with parallel simulation so that we can train controllers with online mapping.
  arXiv  Detail & Related papers  (2026-01-13T12:18:50Z)
• START: Traversing Sparse Footholds with Terrain Reconstruction [11.632418196552324]
  We propose START, a single-stage learning framework that enables agile, stable locomotion on highly sparse footholds.<n> START leverages only low-cost onboard vision and proprioception to accurately reconstruct local terrain heightmap.<n> Experimental results demonstrate that START achieves zero-shot transfer across diverse real-world scenarios.
  arXiv  Detail & Related papers  (2025-12-15T10:02:41Z)
• Mastering Diverse, Unknown, and Cluttered Tracks for Robust Vision-Based Drone Racing [22.63301233637327]
  Most reinforcement learning methods for drone racing target fixed, obstacle-free tracks, leaving the generalization to unknown, cluttered environments largely unaddressed.<n>This challenge stems from the need to balance racing speed and collision avoidance, limited feasible space causing policy exploration trapped in local optima during training.<n>We propose a two-phase learning framework: an initial soft-collision training phase that preserves policy exploration for high-speed flight, followed by a hard-collision refinement phase that enforces robust obstacle avoidance.
  arXiv  Detail & Related papers  (2025-12-10T12:02:48Z)
• Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction [86.53248703859718]
  We introduce a perceptive locomotion framework that merges terrain sensing, gait regulation, and whole-body control into a single reinforcement learning policy.<n>Experiments conducted on a 31-DoF, 1.65 m humanoid robot demonstrate robust locomotion in both simulation and real-world settings.
  arXiv  Detail & Related papers  (2025-12-08T11:42:41Z)
• NoTVLA: Narrowing of Dense Action Trajectories for Generalizable Robot Manipulation [54.87964060934928]
  Vision-Language-Action (VLA) models confront critical barriers to real-world deployment, most notably catastrophic forgetting.<n>We propose the Narrowing of Trajectory VLA framework: a novel approach that narrows its focus to sparse trajectories.<n>NoTVLA achieves superior performance and generalization compared to pi0 while operating under two critical constraints.
  arXiv  Detail & Related papers  (2025-10-04T18:26:55Z)
• Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration [58.4036440289082]
  Hand-object motion-capture (MoCap) offer large-scale, contact-rich demonstrations and hold promise for dexterous robotic scopes.<n>We introduce Dexplore, a unified single-loop optimization that performs repositories and tracking to learn robot control policies directly from MoCap at scale.
  arXiv  Detail & Related papers  (2025-09-11T17:59:07Z)
• Cascaded Diffusion Models for Neural Motion Planning [36.53334347874921]
  We propose an approach for learning global motion planning using diffusion policies.<n>Our approach uses cascaded hierarchical models which unify global prediction and local refinement.<n>Our method outperforms (by 5%) on challenging tasks in multiple domains including navigation and manipulation.
  arXiv  Detail & Related papers  (2025-05-21T06:21:50Z)
• Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation [87.03299519917019]
  We propose a dual-scale graph transformer (DUET) for joint long-term action planning and fine-grained cross-modal understanding. We build a topological map on-the-fly to enable efficient exploration in global action space. The proposed approach, DUET, significantly outperforms state-of-the-art methods on goal-oriented vision-and-language navigation benchmarks.
  arXiv  Detail & Related papers  (2022-02-23T19:06:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.