GN0: Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language Navigation

• URL: http://arxiv.org/abs/2606.03682v1
• Date: Tue, 02 Jun 2026 14:05:47 GMT
• Title: GN0: Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language Navigation
• Abstract summary: Embodied navigation connects intelligent agents with the physical world.<n>We curate diverse 3D scenes and develop an automated pipeline for large-scale navigation data.<n>We introduce a high-fidelity simulation platform supporting interactive roaming and collision-aware navigation.
• Score: 44.636961177849145
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied navigation connects intelligent agents with the physical world and is fundamental for general robotic intelligence. Limited availability and quality of navigation data have constrained Vision-and-Language Navigation (VLN) systems' generalization and long-horizon capabilities. To address this, we curate diverse 3D scenes and develop an automated pipeline for large-scale navigation data, resulting in the GN-Matrix dataset. Building on a 3D Gaussian Splatting (3DGS) engine, we introduce a high-fidelity simulation platform supporting interactive roaming and collision-aware navigation. We further propose GN-Bench, the first BEV-based benchmark incorporating dynamic 3DGS avatars for human-robot interaction evaluation. To leverage the simulator, we develop an RL-driven navigation foundation model, Break and Establish (BAE). After supervised learning, DAgger exposes the model to rollout-induced states, breaking narrow expert-centric distributions and enabling downstream RL exploration. This unified VLN paradigm integrates map-based and map-free tasks, including instruction following, human following, and goal navigation. GN-BAE formalizes high-fidelity 3DGS-rendered Bird's Eye View representations as compact memory, unlocking latent spatial reasoning in VLMs. Extensive evaluations on GN-Bench and VLN-CE show that GN0 outperforms state-of-the-art VLN methods. Overall, GN-Matrix offers a unified framework spanning data, simulation, and learning, advancing embodied navigation in research and industrial applications.

Related papers

• GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation [75.85672467847631]
  We introduce the Geometry-Aware BEV (GA-BEV) - a compact, 3D-grounded feature representation.<n>We construct BEV spatial maps from RGB-D inputs by projecting visual features into 3D space.<n>We incorporate features from a pretrained 3D foundation model into the BEV space, injecting structural priors learned from large-scale 3D reconstruction tasks.
  arXiv  Detail & Related papers  (2026-05-21T06:20:17Z)
• LookasideVLN: Direction-Aware Aerial Vision-and-Language Navigation [96.09246387639006]
  LookasideVLN exploits directional cues in natural language to achieve both more accurate spatial reasoning and greater computational efficiency.<n>LookasideVLN significantly outperforms the state-of-the-art CityNavAgent, even with a single-level lookahead.
  arXiv  Detail & Related papers  (2026-04-19T01:36:53Z)
• AgentVLN: Towards Agentic Vision-and-Language Navigation [78.739525400071]
  Vision-and-Language Navigation (VLN) requires an embodied agent to ground complex natural-language instructions into long-horizon navigation in unseen environments.<n>In this paper, we propose AgentVLN, a novel and efficient embodied navigation framework that can be deployed on edge computing platforms.
  arXiv  Detail & Related papers  (2026-03-18T12:43:47Z)
• NavGSim: High-Fidelity Gaussian Splatting Simulator for Large-Scale Navigation [14.045751242776332]
  NavGSim is a simulator designed to generate high-fidelity, large-scale navigation environments.<n>Built upon a hierarchical 3D Gaussian Splatting framework, NavGSim enables rendering in expansive scenes spanning hundreds of square meters.
  arXiv  Detail & Related papers  (2026-03-16T12:23:20Z)
• 3DGSNav: Enhancing Vision-Language Model Reasoning for Object Navigation via Active 3D Gaussian Splatting [12.057873540714098]
  3DGSNav is a novel framework that embeds 3D Gaussian Splatting (3DGS) as persistent memory for vision-language models (VLMs) to enhance spatial reasoning.<n>3DGSNav incrementally constructs a 3DGS representation of the environment, enabling trajectory-guided free-viewpoint rendering of frontier-aware first-person views.<n>During navigation, a real-time object detector filters potential targets, while VLM-driven active viewpoint switching performs target re-verification.
  arXiv  Detail & Related papers  (2026-02-12T16:41:26Z)
• NavDreamer: Video Models as Zero-Shot 3D Navigators [10.105345998118915]
  We propose a video-based framework for 3D navigation that leverages generative video models as a universal interface between language instructions and trajectories.<n>Our main hypothesis is that video's ability to encode information and physical dynamics, combined with internet-scale availability, enables strong zero-shot generalization in navigation.
  arXiv  Detail & Related papers  (2026-02-10T13:24:12Z)
• OpenFly: A Comprehensive Platform for Aerial Vision-Language Navigation [49.697035403548966]
  Vision-Language Navigation (VLN) aims to guide agents by leveraging language instructions and visual cues, playing a pivotal role in embodied AI.<n>We propose OpenFly, a platform comprising various rendering engines, a versatile toolchain, and a large-scale benchmark for aerial VLN.<n>We construct a large-scale aerial VLN dataset with 100k trajectories, covering diverse heights and lengths across 18 scenes.
  arXiv  Detail & Related papers  (2025-02-25T09:57:18Z)
• UnitedVLN: Generalizable Gaussian Splatting for Continuous Vision-Language Navigation [71.97405667493477]
  We introduce a novel, generalizable 3DGS-based pre-training paradigm, called UnitedVLN.<n>It enables agents to better explore future environments by unitedly rendering high-fidelity 360 visual images and semantic features.<n>UnitedVLN outperforms state-of-the-art methods on existing VLN-CE benchmarks.
  arXiv  Detail & Related papers  (2024-11-25T02:44:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.