Think Like a Pilot: Fine-Grained Long-Horizon UAV Navigation

• URL: http://arxiv.org/abs/2606.06836v1
• Date: Fri, 05 Jun 2026 02:23:05 GMT
• Title: Think Like a Pilot: Fine-Grained Long-Horizon UAV Navigation
• Abstract summary: Language-guided UAV agents must execute long-horizon semantic instructions while producing smooth, physically feasible continuous flight commands.<n>We introduce textbfFLIGHT, a textbfFine-grained textbfLong-horizon textbfInstruction-textbfGuided benchmark for textbfHybrid UAV navigation and reasoning textbfTasks.<n>To endow the UAV agent with the capability of real-time in-flight reasoning over task execution status and mission planning, we
• Score: 22.062997228195815
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language-guided UAV agents must execute long-horizon semantic instructions while producing smooth, physically feasible continuous flight commands, yet existing Vision-Language Navigation (VLN) benchmarks typically use discrete or coarse actions and existing UAV Vision-Language-Action (VLA) tasks focus on short, atomic maneuvers. To address this gap in UAV task settings, we introduce \textbf{FLIGHT}, a \textbf{F}ine-grained \textbf{L}ong-horizon \textbf{I}nstruction-\textbf{G}uided benchmark for \textbf{H}ybrid UAV navigation and reasoning \textbf{T}asks, which combines multi-stage instructions with dense 6-DoF trajectory annotations across two dataset splits: Fine-grained VLN and Long-horizon Flow. To endow the UAV agent with the capability of real-time in-flight reasoning over task execution status and mission planning, while simultaneously accommodating high-frequency, real-time precise control, we further propose \textbf{FLIGHT VLA}, an asynchronous architecture that decouples a low-frequency Streaming Pilot Vision-Language Model (VLM) for task-state reasoning from a high-frequency diffusion action model for continuous control, supervised by explicit \textbf{Pilot Reasoning} texts that summarize the current flight state and anticipate the next subgoal. In closed-loop evaluation, FLIGHT VLA consistently surpasses representative VLN and VLA baselines on our FLIGHT benchmarks, achieving stronger multi-stage completion, subgoal adherence, and terminal control. Its trained Streaming Pilot Reasoning VLM further improves UAV video reasoning, validating the effectiveness of our design.

Related papers

• FSD-VLN: Fast-Slow Dual-System Modeling for Aerial Long-Horizon Vision-Language Navigation [33.92221208368759]
  Vision-Language Navigation (VLN) is an autonomous navigation in unknown environments by mapping language instructions to real-time visual inputs.<n>Existing methods suffer from structural misalignment between global multimodal understanding and sequential action generation.<n>We propose FSD-VLN, a fast-slow dual-system architecture disentangling semantic reasoning and low-altitude flight command generation.
  arXiv  Detail & Related papers  (2026-07-09T11:17:04Z)
• CoFL-S: Spatially Queryable Sector Flow Fields for Local Language-Conditioned Navigation [13.76195083398865]
  CoFL-S is a low-level vision-language-action framework that predicts a language-conditioned flow field over the robot's local visible sector.<n>To train this low-level representation, we convert each VLN-CE episode, originally a whole-episode instruction paired with an action sequence, into frame-level local supervision.<n>For evaluation, we introduce a continuous-time Habitat benchmark that isolates low-level action interfaces from instruction decomposition.
  arXiv  Detail & Related papers  (2026-07-02T14:26:55Z)
• DynFly: Dynamic-Aware Continuous Trajectory Generation for UAV Vision-Language Navigation in Urban Environments [55.73595346639892]
  We propose DynFly to bridge high-level navigation reasoning and executable UAV motion.<n>DynFly represents expert trajectories in B-spline control-point space and employs a Spline-DiT generator to learn conditional trajectory generation.<n>Experiments on the OpenUAV UAV-VLN benchmark show that DynFly improves both navigation performance and trajectory quality.
  arXiv  Detail & Related papers  (2026-06-30T13:33:48Z)
• MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving [54.57163800903507]
  We present MindVLA-U1, the first unified streaming VLA architecture for autonomous driving.<n>A unified VLM backbone produces AR language tokens and flow-matching continuous action trajectories in a single forward pass over one shared representation.<n>On the long-tail WOD-E2E benchmark, MindVLA-U1 surpasses experienced human drivers for the first time.
  arXiv  Detail & Related papers  (2026-05-12T18:09:42Z)
• LiteVLA-H: Dual-Rate Vision-Language-Action Inference for Onboard Aerial Guidance and Semantic Perception [0.6119773373677944]
  We present LiteVLA-H, a compact 256M- parameter VLA system for dual-rate operation on an Jets NVIDIAon AGX Orin.<n>The central empirical observation is that, in this compact edge regime, end-to-end latency is dominated by multimodal pre-fill.<n>To specialize the model without collapsing its descriptive competence, we use a knowledge-preserving fine-tuning recipe that mixes reactive flight data, aerial semantic data, and generic caption/VQA supervision.
  arXiv  Detail & Related papers  (2026-04-27T06:42:50Z)
• VP-VLA: Visual Prompting as an Interface for Vision-Language-Action Models [80.1055544841585]
  Vision-Language-Action models typically map visual observations and linguistic instructions directly to robotic control signals.<n>We propose VP-VLA, a dual-system framework that decouples high-level reasoning and low-level execution via a structured visual prompting interface.<n> Experiments on the Robocasa-GR1-Tabletop benchmark and SimplerEnv simulation demonstrate that VP-VLA improves success rates by 5% and 8.3%.
  arXiv  Detail & Related papers  (2026-03-23T14:08:58Z)
• LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integration [49.676794850174325]
  LongFly is a history-awaretemporal modeling framework for long-horizon UAV VLN.<n>It transforms fragmented and redundant historical data into structured, compact, and expressive representations.<n>It outperforms state-of-the-art UAV VLN baselines by 7.89% in success rate and 6.33% in success weighted by path length.
  arXiv  Detail & Related papers  (2025-12-26T12:09:40Z)
• IndoorUAV: Benchmarking Vision-Language UAV Navigation in Continuous Indoor Environments [21.821075450697027]
  Vision-IndoorLanguage Navigation (VLN) enables agents to navigate in complex environments by following natural language instructions grounded in visual observations.<n> indoor UAV-based VLN remains underexplored, despite its relevance to real-world applications such as inspection, delivery, and search-and-rescue in confined spaces.<n>We introduce textbfIndoorUAV, a novel benchmark and method specifically tailored for VLN with indoor UAVs.
  arXiv  Detail & Related papers  (2025-12-22T04:42:35Z)
• NinA: Normalizing Flows in Action. Training VLA Models with Normalizing Flows [75.70583906344815]
  Diffusion models have been widely adopted as action decoders due to their ability to model complex, multimodal action distributions.<n>We present NinA, a fast and expressive alternative to diffusion-based decoders for Vision-Language-Action (VLA) models.
  arXiv  Detail & Related papers  (2025-08-23T00:02:15Z)
• VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning [77.34267241692706]
  Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions.<n>We propose VLN-R1, an end-to-end framework that leverages Large Vision-Language Models (LVLM) to directly translate egocentric video streams into continuous navigation actions.
  arXiv  Detail & Related papers  (2025-06-20T17:59:59Z)
• Grounded Vision-Language Navigation for UAVs with Open-Vocabulary Goal Understanding [1.280979348722635]
  Vision-and-language navigation (VLN) is a long-standing challenge in autonomous robotics, aiming to empower agents with the ability to follow human instructions while navigating complex environments.<n>We propose Vision-Language Fly (VLFly), a framework tailored for Unmanned Aerial Vehicles (UAVs) to execute language-guided flight.
  arXiv  Detail & Related papers  (2025-06-12T14:40:50Z)
• UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning [39.07541452390107]
  Unmanned Aerial Vehicles (UAVs) are evolving into language-interactive platforms, enabling more intuitive forms of human-drone interaction.<n>We formalize this problem as the Flying-on-a-Word (Flow) task and introduce UAV imitation learning as an effective approach.<n>We present UAV-Flow, the first real-world benchmark for language-conditioned, fine-grained UAV control.
  arXiv  Detail & Related papers  (2025-05-21T16:31:28Z)
• Federated Learning in the Sky: Joint Power Allocation and Scheduling with UAV Swarms [98.78553146823829]
  Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks. In this paper, a novel framework is proposed to implement distributed learning (FL) algorithms within a UAV swarm.
  arXiv  Detail & Related papers  (2020-02-19T14:04:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.