WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation

• URL: http://arxiv.org/abs/2605.15964v1
• Date: Fri, 15 May 2026 13:55:39 GMT
• Title: WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation
• Abstract summary: We propose WorldVLN, the first autoregressive world action model for aerial VLN.<n>WorldVLN adapts a latent autoregressive video backbone to predict short-horizon world-state transitions.<n>WorldVLN consistently outperforms existing Vision-Language-Action baselines.
• Score: 31.224842983083803
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Aerial vision-language navigation (VLN) requires agents to follow natural-language instructions through closed-loop perception and action in 3D environments. We argue that aerial VLN can be formulated as a prediction-driven world-action problem: the agent should anticipate latent world evolution and act according to the predicted consequences. To this end, we propose WorldVLN, the first autoregressive world action model for aerial VLN. Unlike full-sequence video-generation world models that generate an entire visual clip, WorldVLN adapts a latent autoregressive video backbone to predict short-horizon world-state transitions and directly decodes them into executable waypoint actions. After each action segment is executed, newly received observations are encoded back into the autoregressive context, enabling closed-loop world-action prediction. We further introduce a two-stage training framework that first grounds the video prior in instruction-conditioned navigation dynamics and then develops Action-aware GRPO, the first reinforcement learning method tailored to autoregressive WAMs, to optimize waypoint decisions through their downstream rollout consequences. On public outdoor and indoor benchmarks, WorldVLN consistently outperforms existing Vision-Language-Action baselines with 12\%+ success-rate gains and larger advantages on challenging cases. It further transfers zero-shot to real drone deployment, suggesting that the proposed WorldVLN offers a promising route for spatial action tasks. Demos and code are available at https://embodiedcity.github.io/WorldVLN/.

Related papers

• World Tokens: Enhancing Embodied Policies with Training-Time World Modeling [29.442423651761107]
  We introduce World Tokens, an embodied policy architecture built around a World Adapter.<n>World Tokens bridges visual-language understanding, world-dynamics modeling, and action generation.<n>We attains the best reported averages on SIMPLE, substantially improves real-world R1 Pro success over a matched action-only baseline, and generates each action chunk at VLA-level latency.
  arXiv  Detail & Related papers  (2026-08-10T15:30:38Z)
• WNM-3D: A World Navigation Model with 3D Scene Conditioning for Closed-Loop VLN [53.419929910272735]
  We present WNM-3D, a generative World Navigation Model with 3D scene conditioning for continuous VLN.<n>To consolidate past observations into persistent scene context, a frozen feed-forward geometry encoder extracts geometry-aware representations from the monocular egocentric RGB history.<n>Through block-causal attention, this prefix conditions every future video-action block, providing a shared geometric context for both future-view and action generation.
  arXiv  Detail & Related papers  (2026-08-07T14:29:00Z)
• FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation [22.882723669402026]
  FutureNav is a VLM-based unified world-action modeling framework for vision-and-language navigation.<n>We show that FutureNav achieves state-of-the-art performance on multiple VLN benchmarks and substantially outperforms prior VLN methods.
  arXiv  Detail & Related papers  (2026-06-29T14:33:03Z)
• World Pilot: Steering Vision-Language-Action Models with World-Action Priors [21.867210989881016]
  Vision-Language-Action (VLA) models inherit semantic grounding from large-scale pretraining.<n>World Pilot augments the policy with priors from a World-Action Model (WAM)<n>World Pilot attains a state-of-the-art Total success rate of 84.7% on the LIBERO-Plus zero-shot OOD benchmark.
  arXiv  Detail & Related papers  (2026-06-10T17:59:08Z)
• ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning [8.298446181889675]
  Vision-language navigation (VLN) for UAVs demands grounding free-form instructions into 6-DoF flight under partial observability.<n>We propose ImagineUAV, an imagination-driven framework leveraging cascaded world-action modeling.<n>ImagineUAV outperforms prior VLN and VLA baselines on benchmarks and real-world flights.
  arXiv  Detail & Related papers  (2026-05-31T12:39:44Z)
• Learning Vision-Language-Action World Models for Autonomous Driving [15.103497388527943]
  Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving.<n>VLA models often lack explicit modeling of temporal dynamics and global world consistency.<n>We present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning.
  arXiv  Detail & Related papers  (2026-04-10T07:38:05Z)
• LMGenDrive: Bridging Multimodal Understanding and Generative World Modeling for End-to-End Driving [60.31765454895336]
  We present LMGenDrive, the first framework that combines multimodal understanding with generative world models for end-to-end closed-loop driving.<n>We propose a progressive three-stage training strategy, from vision pretraining to multi-step long-horizon driving, to improve stability and performance.
  arXiv  Detail & Related papers  (2026-04-09T19:13:14Z)
• DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA [72.9197085473598]
  DIAL is a framework bridging high-level decision making and low-level motor execution through a differentiable latent intent bottleneck.<n>A VLM-based System-2 performs latent world modeling by latent visual foresight within the VLM's native feature space.<n>A lightweight System-1 policy then decodes this predicted intent together with the current observation into precise robot actions.
  arXiv  Detail & Related papers  (2026-03-31T15:02:27Z)
• Chain of World: World Model Thinking in Latent Motion [24.24061036481793]
  Vision-Language-Action (VLA) models often overlook the predictive and temporal-causal structure underlying visual dynamics.<n>We introduce CoWVLA, a new "Chain of World" paradigm that unifies world-model temporal reasoning with a disentangled latent motion representation.<n>CoWVLA outperforms existing world-model and latent-action approaches and achieves moderate computational efficiency.
  arXiv  Detail & Related papers  (2026-03-03T17:52:06Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)
• World-VLA-Loop: Closed-Loop Learning of Video World Model and VLA Policy [55.03832008486675]
  World-VLA-Loop is a closed-loop framework for the joint refinement of world models and Vision-Language-Action (VLA) policies.<n>We propose a state-aware video world model that functions as a high-fidelity interactive simulator by jointly predicting future observations and reward signals.
  arXiv  Detail & Related papers  (2026-02-06T08:57:55Z)
• DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge [41.030494146004806]
  We propose DreamVLA, a novel VLA framework that integrates comprehensive world knowledge forecasting to enable inverse dynamics modeling.<n>DreamVLA introduces a dynamic-region-guided world knowledge prediction, integrated with the spatial and semantic cues, which provide compact yet comprehensive representations for action planning.<n>Experiments on both real-world and simulation environments demonstrate that DreamVLA achieves 76.7% success rate on real robot tasks.
  arXiv  Detail & Related papers  (2025-07-06T16:14:29Z)
• VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning [77.34267241692706]
  Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions.<n>We propose VLN-R1, an end-to-end framework that leverages Large Vision-Language Models (LVLM) to directly translate egocentric video streams into continuous navigation actions.
  arXiv  Detail & Related papers  (2025-06-20T17:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.