WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigation

• URL: http://arxiv.org/abs/2606.06147v1
• Date: Thu, 04 Jun 2026 13:23:05 GMT
• Title: WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigation
• Abstract summary: We argue that the ability to "imagine" future states is critical for robust decision-making under such partial observability.<n>We propose WorldFly, a novel world-model-based VLA framework that employs a dual-branch coupled flow matching mechanism to jointly generate future video predictions and navigation actions.
• Score: 42.8714305455232
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: End-to-end Vision-Language-Action (VLA) models have shown promise in UAV navigation. However, existing approaches typically rely on historical observations to directly predict actions, often struggling in dense urban environments where severe occlusions and sharp turns result in drastic viewpoint transitions. We argue that the ability to "imagine" future states -- inherent in World Models -- is critical for robust decision-making under such partial observability. To address this, we construct a challenging Urban Canyon Traversal Benchmark, specifically designed to evaluate spatial understanding in scenarios characterized by severe occlusions and drastic viewpoint transitions. To this end, we propose WorldFly, a novel world-model-based VLA framework that employs a dual-branch coupled flow matching mechanism to jointly generate future video predictions and navigation actions, thereby explicitly guiding the agent's policy via spatial imagination. Extensive evaluations on our benchmark demonstrate that WorldFly outperforms other baselines, particularly in unseen environments, validating the effectiveness of integrating world models into embodied aerial agents.

Related papers

• SC$^{2}$-WM: A Self-Correcting World Model with Closed-Loop Feedback for Vision-and-Language Navigation in Continuous Environments [54.76467884696539]
  We propose SC$2$-WM, a self-correcting world model framework that introduces internal feedback for closed-loop decision making in VLN-CE.<n>Our method derives feedback from world-model foresight to perform state-level plan refinement before action execution.<n>Experiments on standard VLN-CE benchmarks demonstrate improved navigation and generalization.
  arXiv  Detail & Related papers  (2026-08-01T03:17:48Z)
• FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation [22.882723669402026]
  FutureNav is a VLM-based unified world-action modeling framework for vision-and-language navigation.<n>We show that FutureNav achieves state-of-the-art performance on multiple VLN benchmarks and substantially outperforms prior VLN methods.
  arXiv  Detail & Related papers  (2026-06-29T14:33:03Z)
• NavWM: A Unified Navigation World Model for Foresight-Driven Planning [15.801647085785826]
  NavWM is a unified navigation world model that seamlessly integrates latent world reasoning, multimodal action prediction, and controllable visual generation.<n>To overcome the limitations of deterministic policies, we introduce an anchor-based multimodal trajectory forecasting framework.<n>Experiments across diverse datasets demonstrate that NavWM significantly advances the state-the-art.
  arXiv  Detail & Related papers  (2026-06-23T03:30:20Z)
• World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems [32.4918638412746]
  Vision-Language-Action (VLA) models have emerged as a promising paradigm for building embodied agents that ground perception and language into action.<n>We introduce World-Value-Action (WAV) model, a unified framework that enables implicit planning in VLA systems.
  arXiv  Detail & Related papers  (2026-04-16T07:46:05Z)
• Learning Vision-Language-Action World Models for Autonomous Driving [15.103497388527943]
  Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving.<n>VLA models often lack explicit modeling of temporal dynamics and global world consistency.<n>We present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning.
  arXiv  Detail & Related papers  (2026-04-10T07:38:05Z)
• LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning [51.969318585152116]
  LatentPilot exploits future observations during training as a valuable data source to learn action-conditioned visual dynamics.<n>We propose a flywheel-style training mechanism that iteratively collects on-policy trajectories and retrains the model to better match the agent's behavior distribution.<n> Experiments on R2R-CE, RxR-CE, and R2R-PE benchmarks achieve new SOTA results, and real-robot tests across diverse environments demonstrate LatentPilot's superior understanding of environment-action dynamics in scene.
  arXiv  Detail & Related papers  (2026-03-31T02:21:59Z)
• Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving [52.04950569530877]
  We present Uni-World VLA, a unified vision-language-action model that tightly interleaves future frame prediction and trajectory planning.<n>Our approach achieves competitive closed-loop planning performance while producing high-fidelity future frame predictions.
  arXiv  Detail & Related papers  (2026-03-28T14:39:51Z)
• ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model [53.15040805435013]
  Vision-language models (VLMs) provide strong semantic grounding and general knowledge by reasoning over uniformly sampled frames.<n>We propose a VLM-guided JEPA-style latent world modeling framework that combines dense-frame dynamics modeling with long-horizon semantic guidance.
  arXiv  Detail & Related papers  (2026-03-23T17:59:42Z)
• AstraNav-World: World Model for Foresight Control and Consistency [40.07910402326578]
  Embodied navigation in dynamic environments demands accurate foresight of how the world will evolve and how actions will unfold over time.<n>We propose AstraNav-World, an end-to-end world model that jointly reasons about future visual states and action sequences.<n>Our framework integrates a diffusion-based video generator with a vision-language policy, enabling synchronized rollouts.
  arXiv  Detail & Related papers  (2025-12-25T15:31:24Z)
• DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge [41.030494146004806]
  We propose DreamVLA, a novel VLA framework that integrates comprehensive world knowledge forecasting to enable inverse dynamics modeling.<n>DreamVLA introduces a dynamic-region-guided world knowledge prediction, integrated with the spatial and semantic cues, which provide compact yet comprehensive representations for action planning.<n>Experiments on both real-world and simulation environments demonstrate that DreamVLA achieves 76.7% success rate on real robot tasks.
  arXiv  Detail & Related papers  (2025-07-06T16:14:29Z)
• Ground-level Viewpoint Vision-and-Language Navigation in Continuous Environments [10.953629652228024]
  Vision-and-Language Navigation (VLN) agents associate time-sequenced visual observations with corresponding instructions to make decisions.<n>In this paper, we address the mismatch between human-centric instructions and quadruped robots with a low-height field of view.<n>We propose a Ground-level Viewpoint Navigation (GVNav) approach to mitigate this issue.
  arXiv  Detail & Related papers  (2025-02-26T10:30:40Z)
• DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation [107.5934592892763]
  We propose DREAMWALKER -- a world model based VLN-CE agent. The world model is built to summarize the visual, topological, and dynamic properties of the complicated continuous environment. It can simulate and evaluate possible plans entirely in such internal abstract world, before executing costly actions.
  arXiv  Detail & Related papers  (2023-08-14T23:45:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.