OnFly: Onboard Zero-Shot Aerial Vision-Language Navigation toward Safety and Efficiency

• URL: http://arxiv.org/abs/2603.10682v1
• Date: Wed, 11 Mar 2026 11:48:22 GMT
• Title: OnFly: Onboard Zero-Shot Aerial Vision-Language Navigation toward Safety and Efficiency
• Abstract summary: We propose OnFly, a fully onboard, real-time framework for zero-shot AVLN.<n>OnFly adopts a shared-perception dual-agent architecture that decouples high-frequency target generation from low-frequency progress monitoring.<n>In simulation, OnFly improves task success from 26.4% to 67.8%, compared with the strongest state-of-the-art baseline.
• Score: 9.769964979769204
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Aerial vision-language navigation (AVLN) enables UAVs to follow natural-language instructions in complex 3D environments. However, existing zero-shot AVLN methods often suffer from unstable single-stream Vision-Language Model decision-making, unreliable long-horizon progress monitoring, and a trade-off between safety and efficiency. We propose OnFly, a fully onboard, real-time framework for zero-shot AVLN. OnFly adopts a shared-perception dual-agent architecture that decouples high-frequency target generation from low-frequency progress monitoring, thereby stabilizing decision-making. It further employs a hybrid keyframe-recent-frame memory to preserve global trajectory context while maintaining KV-cache prefix stability, enabling reliable long-horizon monitoring with termination and recovery signals. In addition, a semantic-geometric verifier refines VLM-predicted targets for instruction consistency and geometric safety using VLM features and depth cues, while a receding-horizon planner generates optimized collision-free trajectories under geometric safety constraints, improving both safety and efficiency. In simulation, OnFly improves task success from 26.4% to 67.8%, compared with the strongest state-of-the-art baseline, while fully onboard real-world flights validate its feasibility for real-time deployment. The code will be released at https://github.com/Robotics-STAR-Lab/OnFly

Related papers

• Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation [16.850454229093135]
  We propose a safety-constrained perception-control integrated framework for UAV navigation.<n>A lightweight network encodes sparse observations into collision-risk-aware features using asymmetric and depthwise separable convolutions.<n> Experiments with varying obstacle densities and flight speeds demonstrate higher success rates, improved safety, and better efficiency.
  arXiv  Detail & Related papers  (2026-07-02T07:12:24Z)
• Dynamic Control Barrier Function Regulation with Vision-Language Models for Safe, Adaptive, and Realtime Visual Navigation [5.268770735910558]
  We present AlphaAdj, a vision-to-control navigation framework that uses egocentric RGB input to adapt the conservativeness of a CBF safety filter in real time.<n>Results show that AlphaAdj maintains collision-free navigation while improving efficiency (in terms of path length and time to goal) by up to 18.5% relative to fixed settings.
  arXiv  Detail & Related papers  (2026-03-22T09:35:18Z)
• HUGE-Bench: A Benchmark for High-Level UAV Vision-Language-Action Tasks [73.62053624331227]
  Huge-Bench is a benchmark for High-Level UAV Vision-Language-Action tasks.<n>It comprises 4 real-world digital twin scenes, 8 high-level tasks, and 2.56M meters of trajectories.<n>We introduce process-oriented and collision-aware metrics to assess process fidelity, terminal accuracy, and safety.
  arXiv  Detail & Related papers  (2026-03-20T10:08:42Z)
• FC-Vision: Real-Time Visibility-Aware Replanning for Occlusion-Free Aerial Target Structure Scanning in Unknown Environments [27.67128708714023]
  FC-Vision is an on-the-fly visibility-aware replanning framework for autonomous aerial scanning of target structures.<n>FC-Vision consistently improves scanning quality under unexpected occluders.
  arXiv  Detail & Related papers  (2026-02-14T11:01:40Z)
• AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge [49.66156306240961]
  High latency breaks the control loop, rendering powerful models unsafe for real-time deployment.<n>We propose AsyncVLA, an asynchronous control framework that decouples semantic reasoning from reactive execution.<n>AsyncVLA achieves a 40% higher success rate than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-02-13T21:31:19Z)
• VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environments [12.689250855332569]
  VLA-AN is a framework dedicated to autonomous drone navigation in complex environments.<n>It addresses four major limitations of existing large aerial navigation models.<n>It achieves a maximum single-task success rate of 98.1%.
  arXiv  Detail & Related papers  (2025-12-17T10:02:55Z)
• Trajectory Design for UAV-Based Low-Altitude Wireless Networks in Unknown Environments: A Digital Twin-Assisted TD3 Approach [62.11847362756054]
  Unmanned aerial vehicles (UAVs) are emerging as key enablers for low-altitude wireless network (LAWN)<n>We propose a digital twin (DT)-assisted training and deployment framework.<n>In this framework, the UAV transmits integrated sensing and communication signals to provide communication services to ground users, while simultaneously collecting echoes that are uploaded to the DT server to progressively construct virtual environments (VEs)<n>These VEs accelerate model training and are continuously updated with real-time UAV sensing data during deployment, supporting decision-making and enhancing flight safety.
  arXiv  Detail & Related papers  (2025-10-28T10:05:53Z)
• When UAV Swarm Meets IRS: Collaborative Secure Communications in Low-altitude Wireless Networks [68.45202147860537]
  Low-altitude wireless networks (LAWNs) provide enhanced coverage, reliability, and throughput for diverse applications.<n>These networks face significant security vulnerabilities from both known and potential unknown eavesdroppers.<n>We propose a novel secure communication framework for LAWNs where the selected UAVs within a swarm function as a virtual antenna array.
  arXiv  Detail & Related papers  (2025-10-25T02:02:14Z)
• LLM Meets the Sky: Heuristic Multi-Agent Reinforcement Learning for Secure Heterogeneous UAV Networks [57.27815890269697]
  This work focuses on maximizing the secrecy rate in heterogeneous UAV networks (HetUAVNs) under energy constraints.<n>We introduce a Large Language Model (LLM)-guided multi-agent learning approach.<n>Results show that our method outperforms existing baselines in secrecy and energy efficiency.
  arXiv  Detail & Related papers  (2025-07-23T04:22:57Z)
• NOVA: Navigation via Object-Centric Visual Autonomy for High-Speed Target Tracking in Unstructured GPS-Denied Environments [56.35569661650558]
  We introduce NOVA, a fully onboard, object-centric framework that enables robust target tracking and collision-aware navigation.<n>Rather than constructing a global map, NOVA formulates perception, estimation, and control entirely in the target's reference frame.<n>We validate NOVA across challenging real-world scenarios, including urban mazes, forest trails, and repeated transitions through buildings with intermittent GPS loss.
  arXiv  Detail & Related papers  (2025-06-23T14:28:30Z)
• ASMA: An Adaptive Safety Margin Algorithm for Vision-Language Drone Navigation via Scene-Aware Control Barrier Functions [9.645098673995317]
  We consider a VLN-operated drone platform and enhance its safety by formulating a novel scene-aware CBF.<n>A CBF-less baseline system uses a Vision-Language with cross-modal attention to convert commands into an ordered sequence of landmarks.<n>ASMA tracks moving objects and performs scene-aware CBF evaluation on-the-fly, which serves as an additional constraint.
  arXiv  Detail & Related papers  (2024-09-16T13:44:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.