GeoLoco: Leveraging 3D Geometric Priors from Visual Foundation Model for Robust RGB-Only Humanoid Locomotion

• URL: http://arxiv.org/abs/2603.07624v1
• Date: Sun, 08 Mar 2026 13:18:27 GMT
• Title: GeoLoco: Leveraging 3D Geometric Priors from Visual Foundation Model for Robust RGB-Only Humanoid Locomotion
• Abstract summary: GeoLoco conceptualizes monocular images as high-dimensional 3D latent representations by harnessing the powerful geometric priors of a frozen, scale-aware Visual Foundation Model (VFM)<n>GeoLoco achieves robust zero-shot transfer to the Unitree G1 humanoid and successfully negotiates challenging terrains.
• Score: 17.719701533574412
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The prevailing paradigm of perceptive humanoid locomotion relies heavily on active depth sensors. However, this depth-centric approach fundamentally discards the rich semantic and dense appearance cues of the visual world, severing low-level control from the high-level reasoning essential for general embodied intelligence. While monocular RGB offers a ubiquitous, information-dense alternative, end-to-end reinforcement learning from raw 2D pixels suffers from extreme sample inefficiency and catastrophic sim-to-real collapse due to the inherent loss of geometric scale. To break this deadlock, we propose GeoLoco, a purely RGB-driven locomotion framework that conceptualizes monocular images as high-dimensional 3D latent representations by harnessing the powerful geometric priors of a frozen, scale-aware Visual Foundation Model (VFM). Rather than naive feature concatenation, we design a proprioceptive-query multi-head cross-attention mechanism that dynamically attends to task-critical topological features conditioned on the robot's real-time gait phase. Crucially, to prevent the policy from overfitting to superficial textures, we introduce a dual-head auxiliary learning scheme. This explicit regularization forces the high-dimensional latent space to strictly align with the physical terrain geometry, ensuring robust zero-shot sim-to-real transfer. Trained exclusively in simulation, GeoLoco achieves robust zero-shot transfer to the Unitree G1 humanoid and successfully negotiates challenging terrains.

Related papers

• Reliability-Aware 3D Geometric Injection for Universal Person Re-identification [47.874819213762926]
  We propose a Universal Monocular 3D-Enhanced ReID framework driven by a Consistency-Aware Reliability Gate and Dual-Stream Residual Fusion.<n>Our method improves challenging, structure-sensitive scenarios while preserving competitive performance on clean domains.
  arXiv  Detail & Related papers  (2026-07-21T08:54:25Z)
• GeoHAT: Geometry-Adaptive Hybrid Action Transformer for Mobile Manipulation [6.488530751190965]
  Whole-body mobile manipulation requires coordinating mobile base and manipulator under shifting viewpoints.<n>We present GeoHAT, an end-to-end diffusion-based framework built on a simple principle.<n>Experiments on the ManiSkill-HAB simulation benchmark demonstrate that GeoHAT achieves a 79.3% mean success rate.
  arXiv  Detail & Related papers  (2026-06-11T14:25:09Z)
• GARDEN: Gravity-Aligned Reconstruction of Disentangled ENvironments from RGB images [61.2170105836525]
  We propose an RGB-only framework that reformulates reconstruction as physically-grounded scene factorization and outputs a structured hybrid scene representation.<n>The resulting representation combines explicit rigid bodies with a decoupled background, enabling direct physics simulation while preserving visual realism.
  arXiv  Detail & Related papers  (2026-06-02T17:13:01Z)
• MaMi-HOI: Harmonizing Global Kinematics and Local Geometry for Human-Object Interaction Generation [14.056276184323321]
  MaMi-HOI reconciles textbfMacro-level fluidity with textbfMicro-level spatial precision.<n>Experiments validate that MaMi-HOI simultaneously achieves natural motion and precise contact.
  arXiv  Detail & Related papers  (2026-05-07T06:52:14Z)
• Beyond Gaussian Bottlenecks: Topologically Aligned Encoding of Vision-Transformer Feature Spaces [18.19675060489249]
  We propose S$2$VAE, a geometry-first latent learning framework that focuses on compressing and representing the latent 3D state of a scene.<n>In depth estimation, camera pose recovery, and point cloud reconstruction, we show that geometry-aligned hyperspherical latents consistently outperform conventional Gaussian bottlenecks.
  arXiv  Detail & Related papers  (2026-04-30T17:12:31Z)
• R-PGA: Robust Physical Adversarial Camouflage Generation via Relightable 3D Gaussian Splatting [59.29134814041703]
  Physical adversarial camouflage poses a severe security threat to autonomous driving systems.<n>Current methods fail to generalize across diverse geometric (e.g., viewing configurations) and radiometric variations.<n>We propose the Relightable Physical 3D Gaussian Splatting framework (R-PGA)
  arXiv  Detail & Related papers  (2026-03-27T04:38:04Z)
• SGR-OCC: Evolving Monocular Priors for Embodied 3D Occupancy Prediction via Soft-Gating Lifting and Semantic-Adaptive Geometric Refinement [9.891265334631889]
  3D semantic occupancy prediction is a cornerstone for embodied AI.<n>We propose SGR-OCC (Soft-Gating and Ray-refinement Occupancy), a unified framework driven by the philosophy of "Inheritance and Evolution"<n>In local prediction tasks, SGR-OCC achieves a completion IoU of 58.55$%$ and a semantic mIoU of 49.89$%$, surpassing the previous best method, EmbodiedOcc++, by 3.65$%$ and 3.69$%$ respectively.
  arXiv  Detail & Related papers  (2026-03-14T18:45:03Z)
• Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Joints [87.13154261503168]
  Motion-controllable video generation is crucial for egocentric applications in virtual reality and embodied AI.<n>Existing methods often struggle to achieve 3D-consistent fine-grained hand articulation.<n>We propose a novel framework that generates egocentric videos from a single reference frame.
  arXiv  Detail & Related papers  (2026-03-12T10:02:23Z)
• Inverse Rendering for High-Genus 3D Surface Meshes from Multi-view Images with Persistent Homology Priors [11.227213428407673]
  Reconstructing 3D objects from images is inherently an ill-posed problem due to ambiguities in geometry, appearance, and topology.<n>This paper introduces collaborative rendering with persistent homology priors, a novel strategy that leverages topological constraints to resolve these ambiguities.
  arXiv  Detail & Related papers  (2026-01-17T20:06:19Z)
• Robust Mesh Saliency GT Acquisition in VR via View Cone Sampling and Geometric Smoothing [59.12032628787018]
  3D mesh saliency ground truth is essential for human-centric visual modeling in virtual reality (VR)<n>Current VR eye-tracking pipelines rely on single ray sampling and Euclidean smoothing, triggering texture attention and signal leakage across gaps.<n>This paper proposes a robust framework to address these limitations.
  arXiv  Detail & Related papers  (2026-01-06T05:20:12Z)
• From Spatial to Actions: Grounding Vision-Language-Action Model in Spatial Foundation Priors [54.84863164684646]
  Existing vision-language-action (VLA) models act in 3D real-world but are typically built on 2D encoders.<n>In this work, we introduce FALCON, a novel paradigm that injects rich 3D spatial tokens into the action head.
  arXiv  Detail & Related papers  (2025-10-20T11:26:45Z)
• Cross-Modal Geometric Hierarchy Fusion: An Implicit-Submap Driven Framework for Resilient 3D Place Recognition [9.411542547451193]
  We propose a novel framework that redefines 3D place recognition through density-agnostic geometric reasoning.<n>Specifically, we introduce an implicit 3D representation based on elastic points, which is immune to the interference of original scene point cloud density.<n>With the aid of these two types of information, we obtain descriptors that fuse geometric information from both bird's-eye view and 3D segment perspectives.
  arXiv  Detail & Related papers  (2025-06-17T07:04:07Z)
• GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision [49.839374549646884]
  This paper presents GEOcc, a Geometric-Enhanced Occupancy network tailored for vision-only surround-view perception.<n>Our approach achieves State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image resolution needed and the most weightless image backbone.
  arXiv  Detail & Related papers  (2024-05-17T07:31:20Z)
• Aligning Silhouette Topology for Self-Adaptive 3D Human Pose Recovery [70.66865453410958]
  Articulation-centric 2D/3D pose supervision forms the core training objective in most existing 3D human pose estimation techniques. We propose a novel framework that relies only on silhouette supervision to adapt a source-trained model-based regressor. We develop a series of convolution-friendly spatial transformations in order to disentangle a topological-skeleton representation from the raw silhouette.
  arXiv  Detail & Related papers  (2022-04-04T06:58:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.