NavDreamer: Video Models as Zero-Shot 3D Navigators

• URL: http://arxiv.org/abs/2602.09765v1
• Date: Tue, 10 Feb 2026 13:24:12 GMT
• Title: NavDreamer: Video Models as Zero-Shot 3D Navigators
• Abstract summary: We propose a video-based framework for 3D navigation that leverages generative video models as a universal interface between language instructions and trajectories.<n>Our main hypothesis is that video's ability to encode information and physical dynamics, combined with internet-scale availability, enables strong zero-shot generalization in navigation.
• Score: 10.105345998118915
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Previous Vision-Language-Action models face critical limitations in navigation: scarce, diverse data from labor-intensive collection and static representations that fail to capture temporal dynamics and physical laws. We propose NavDreamer, a video-based framework for 3D navigation that leverages generative video models as a universal interface between language instructions and navigation trajectories. Our main hypothesis is that video's ability to encode spatiotemporal information and physical dynamics, combined with internet-scale availability, enables strong zero-shot generalization in navigation. To mitigate the stochasticity of generative predictions, we introduce a sampling-based optimization method that utilizes a VLM for trajectory scoring and selection. An inverse dynamics model is employed to decode executable waypoints from generated video plans for navigation. To systematically evaluate this paradigm in several video model backbones, we introduce a comprehensive benchmark covering object navigation, precise navigation, spatial grounding, language control, and scene reasoning. Extensive experiments demonstrate robust generalization across novel objects and unseen environments, with ablation studies revealing that navigation's high-level decision-making nature makes it particularly suited for video-based planning.

Related papers

• LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation [52.91080323083409]
  Embodied navigation requires agents to translate heterogeneous goals and visual observations into actions across tasks, environments, and robot embodiments.<n>Here we present LightNav-0, a compact generalist embodied navigation model that elicits the spatial intelligence of a pretrained VLM and aligns it with navigation, without task-specific prediction heads.
  arXiv  Detail & Related papers  (2026-08-31T15:08:53Z)
• GN0: Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language Navigation [44.636961177849145]
  Embodied navigation connects intelligent agents with the physical world.<n>We curate diverse 3D scenes and develop an automated pipeline for large-scale navigation data.<n>We introduce a high-fidelity simulation platform supporting interactive roaming and collision-aware navigation.
  arXiv  Detail & Related papers  (2026-06-02T14:05:47Z)
• ImagiNav: Scalable Embodied Navigation via Generative Visual Prediction and Inverse Dynamics [13.382453086651019]
  Vision-Language Navigation has relied on end-to-end policies trained on expensive, embodiment-specific robot data.<n>We propose ImagiNav, a novel modular paradigm that decouples visual planning from robot actuation.<n>ImagiNav demonstrates strong zero-shot transfer to robot navigation without requiring robot demonstrations.
  arXiv  Detail & Related papers  (2026-03-14T08:34:02Z)
• Implicit Geometry Representations for Vision-and-Language Navigation from Web Videos [87.15961946494629]
  We introduce a large-scale video-instruction framework derived from web-based room tour videos.<n>Unlike existing datasets, our framework integrates open-ended description-enriched trajectories and action-enriched trajectories reconstructed in 3D.<n>A key extension in this work is the incorporation of implicit geometry representations, which extract spatial cues directly from RGB frames without requiring fragile 3D reconstruction.
  arXiv  Detail & Related papers  (2026-03-10T06:47:38Z)
• OpenFrontier: General Navigation with Visual-Language Grounded Frontiers [54.661157616245966]
  Open-world navigation requires robots to make decisions in complex everyday environments.<n>Recent advances in vision--language navigation (VLN) and vision--language--action (VLA) models enable end-to-end policies conditioned on natural language.<n>We propose OpenFrontier, a training-free navigation framework that seamlessly integrates diverse vision--language prior models.
  arXiv  Detail & Related papers  (2026-03-05T17:02:22Z)
• 3DGSNav: Enhancing Vision-Language Model Reasoning for Object Navigation via Active 3D Gaussian Splatting [12.057873540714098]
  3DGSNav is a novel framework that embeds 3D Gaussian Splatting (3DGS) as persistent memory for vision-language models (VLMs) to enhance spatial reasoning.<n>3DGSNav incrementally constructs a 3DGS representation of the environment, enabling trajectory-guided free-viewpoint rendering of frontier-aware first-person views.<n>During navigation, a real-time object detector filters potential targets, while VLM-driven active viewpoint switching performs target re-verification.
  arXiv  Detail & Related papers  (2026-02-12T16:41:26Z)
• LookOut: Real-World Humanoid Egocentric Navigation [61.14016011125957]
  We introduce the challenging problem of predicting a sequence of future 6D head poses from an egocentric video.<n>To solve this task, we propose a framework that reasons over temporally aggregated 3D latent features.<n>Motivated by the lack of training data in this space, we present a dataset collected through this approach.
  arXiv  Detail & Related papers  (2025-08-20T06:43:36Z)
• History-Augmented Vision-Language Models for Frontier-Based Zero-Shot Object Navigation [5.343932820859596]
  This paper introduces a novel zero-shot ObjectNav framework that pioneers the use of dynamic, history-aware prompting.<n>Our core innovation lies in providing the VLM with action history context, enabling it to generate semantic guidance scores for navigation actions.<n>We also introduce a VLM-assisted waypoint generation mechanism for refining the final approach to detected objects.
  arXiv  Detail & Related papers  (2025-06-19T21:50:16Z)
• Navigation World Models [68.58459393846461]
  We introduce a controllable video generation model that predicts future visual observations based on past observations and navigation actions.<n>In familiar environments, NWM can plan navigation trajectories by simulating them and evaluating whether they achieve the desired goal.<n>Experiments demonstrate its effectiveness in planning trajectories from scratch or by ranking trajectories sampled from an external policy.
  arXiv  Detail & Related papers  (2024-12-04T18:59:45Z)
• LangNav: Language as a Perceptual Representation for Navigation [63.90602960822604]
  We explore the use of language as a perceptual representation for vision-and-language navigation (VLN) Our approach uses off-the-shelf vision systems for image captioning and object detection to convert an agent's egocentric panoramic view at each time step into natural language descriptions.
  arXiv  Detail & Related papers  (2023-10-11T20:52:30Z)
• Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigation [58.3480730643517]
  We present LGX, a novel algorithm for Language-Driven Zero-Shot Object Goal Navigation (L-ZSON) Our approach makes use of Large Language Models (LLMs) for this task. We achieve state-of-the-art zero-shot object navigation results on RoboTHOR with a success rate (SR) improvement of over 27% over the current baseline.
  arXiv  Detail & Related papers  (2023-03-06T20:19:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.