ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning

• URL: http://arxiv.org/abs/2606.01205v2
• Date: Mon, 08 Jun 2026 02:19:04 GMT
• Title: ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning
• Abstract summary: Vision-language navigation (VLN) for UAVs demands grounding free-form instructions into 6-DoF flight under partial observability.<n>We propose ImagineUAV, an imagination-driven framework leveraging cascaded world-action modeling.<n>ImagineUAV outperforms prior VLN and VLA baselines on benchmarks and real-world flights.
• Score: 8.298446181889675
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language navigation (VLN) for UAVs demands grounding free-form instructions into 6-DoF flight under partial observability. While Vision-Language-Action (VLA) models excel at semantic reasoning, they suffer from brittleness due to geometric inconsistency and dynamics mismatch. To address this, we propose ImagineUAV, an imagination-driven framework leveraging cascaded world-action modeling. Instead of direct regression, ImagineUAV employs a latent video diffusion model to generate instruction-conditioned future observations, explicitly imagining environmental evolution, from which 6-DoF motions are inferred via an action extractor. A kinodynamic planner then refines these estimates into collision-free trajectories. Additionally, a step-distilled inference pipeline ensures real-time execution. With only 1.3B parameters, ImagineUAV outperforms prior VLN and VLA baselines on benchmarks and real-world flights, validating the practicality of imagination-driven aerial navigation.

Related papers

• AeroAct: Action-Centered World-Action Models for Language-Conditioned Quadrotor Flight [42.75555298269899]
  Language-conditioned quadrotor flight requires a policy to ground semantic goals.<n>We present AeroAct, an action-centered world-action model (WAM) for quadrotor navigation.
  arXiv  Detail & Related papers  (2026-07-16T13:46:00Z)
• DynFly: Dynamic-Aware Continuous Trajectory Generation for UAV Vision-Language Navigation in Urban Environments [55.73595346639892]
  We propose DynFly to bridge high-level navigation reasoning and executable UAV motion.<n>DynFly represents expert trajectories in B-spline control-point space and employs a Spline-DiT generator to learn conditional trajectory generation.<n>Experiments on the OpenUAV UAV-VLN benchmark show that DynFly improves both navigation performance and trajectory quality.
  arXiv  Detail & Related papers  (2026-06-30T13:33:48Z)
• WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigation [42.8714305455232]
  We argue that the ability to "imagine" future states is critical for robust decision-making under such partial observability.<n>We propose WorldFly, a novel world-model-based VLA framework that employs a dual-branch coupled flow matching mechanism to jointly generate future video predictions and navigation actions.
  arXiv  Detail & Related papers  (2026-06-04T13:23:05Z)
• FlyMirage: A Fully Automated Generation Pipeline for Diverse and Scalable UAV Flight Data via Generative World Model [13.125579202955384]
  FlyMirage is a highly scalable and fully automated data generation pipeline for aerial VLN.<n>We generate a large-scale, diverse, and photorealistic aerial VLN dataset, with dynamically feasible flying trajectories.
  arXiv  Detail & Related papers  (2026-05-19T09:41:04Z)
• WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation [31.224842983083803]
  We propose WorldVLN, the first autoregressive world action model for aerial VLN.<n>WorldVLN adapts a latent autoregressive video backbone to predict short-horizon world-state transitions.<n>WorldVLN consistently outperforms existing Vision-Language-Action baselines.
  arXiv  Detail & Related papers  (2026-05-15T13:55:39Z)
• MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving [54.57163800903507]
  We present MindVLA-U1, the first unified streaming VLA architecture for autonomous driving.<n>A unified VLM backbone produces AR language tokens and flow-matching continuous action trajectories in a single forward pass over one shared representation.<n>On the long-tail WOD-E2E benchmark, MindVLA-U1 surpasses experienced human drivers for the first time.
  arXiv  Detail & Related papers  (2026-05-12T18:09:42Z)
• MotionScape: A Large-Scale Real-World Highly Dynamic UAV Video Dataset for World Models [6.680381992984212]
  We present MotionScape, a large-scale real-world UAVview video dataset with highly dynamic motion for world modeling.<n>MotionScape contains over 30 hours of 4K UAV-view videos, totaling more than 4.5M frames.<n>This novel dataset features semantically and geometrically aligned training samples, where diverse real-world UAV videos are tightly coupled with accurate 6-DoF camera trajectories and fine-grained natural language descriptions.
  arXiv  Detail & Related papers  (2026-04-09T08:59:40Z)
• AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild [62.47761809929869]
  Vision-language navigation (VLN) requires intelligent agents to navigate environments by interpreting linguistic instructions alongside visual observations.<n>Current VLN research for unmanned aerial vehicles (UAVs) relies on detailed, pre-specified instructions to guide the UAV along predetermined routes.<n>We propose AutoFly, an end-to-end Vision-Language-Action model for autonomous UAV navigation.
  arXiv  Detail & Related papers  (2026-02-10T11:08:07Z)
• Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Space [48.19308247102762]
  We propose ANWM, an aerial navigation world model that predicts future visual observations conditioned on past frames and actions.<n> ANWM is trained on 4-DoF UAV trajectories and introduces a physics-inspired module: Future Frame Projection.<n> Empirical results demonstrate that ANWM significantly outperforms existing world models in long-distance visual forecasting and improves UAV navigation success rates in large-scale environments.
  arXiv  Detail & Related papers  (2025-12-26T06:22:39Z)
• VISTAv2: World Imagination for Indoor Vision-and-Language Navigation [15.33980337718478]
  Vision-and-Language Navigation (VLN) requires agents to follow language instructions while acting in real-world spaces.<n>Prior image imagination based VLN work shows benefits for discrete panoramas but lacks online, action-conditioned predictions.<n>We propose VISTAv2, a generative world model that rolls out egocentric future views conditioned on past observations.
  arXiv  Detail & Related papers  (2025-11-14T10:20:22Z)
• VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning [77.34267241692706]
  Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions.<n>We propose VLN-R1, an end-to-end framework that leverages Large Vision-Language Models (LVLM) to directly translate egocentric video streams into continuous navigation actions.
  arXiv  Detail & Related papers  (2025-06-20T17:59:59Z)
• CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models [89.44024245194315]
  We introduce a method that incorporates explicit visual chain-of-thought (CoT) reasoning into vision-language-action models (VLAs)<n>We introduce CoT-VLA, a state-of-the-art 7B VLA that can understand and generate visual and action tokens.<n>Our experimental results demonstrate that CoT-VLA achieves strong performance, outperforming the state-of-the-art VLA model by 17% in real-world manipulation tasks and 6% in simulation benchmarks.
  arXiv  Detail & Related papers  (2025-03-27T22:23:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.