Air-Ground Collaborative Vision-and-Language Navigation via Shared Bird's-Eye Maps

• URL: http://arxiv.org/abs/2609.03483v3
• Date: Tue, 08 Sep 2026 08:06:30 GMT
• Title: Air-Ground Collaborative Vision-and-Language Navigation via Shared Bird's-Eye Maps
• Abstract summary: Air-ground collaborative Vision-and-Language Navigation (VLN) pairs an unmanned aerial vehicle (UAV) with a global bird's-eye view and an unmanned ground vehicle (UGV) with a local first-person view.<n>We establish AGC-VLN (Air-Ground Collaborative VLN), the first training-free baseline for air-ground collaborative VLN.<n>On 100 closed-loop episodes in CARLA-Air's Town10HD scene, AGC-VLN reaches a 77.0% joint success rate, a collaboration gain of +27.0% over the weaker individual agent
• Score: 23.62701600758693
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Air-ground collaborative Vision-and-Language Navigation (VLN) pairs an unmanned aerial vehicle (UAV) with a global bird's-eye view and an unmanned ground vehicle (UGV) with a local first-person view, yet the setting remains largely unexplored: existing training-free methods solve single-agent tasks but offer no collaboration mechanism, and a recent CARLA-Air evaluation found no stable cooperative behavior across five state-of-the-art VLA models; naive semantic communication or bidirectional coupling even degrades performance. We establish AGC-VLN (Air-Ground Collaborative VLN), the first training-free baseline for air-ground collaborative VLN. The key insight is that training-free methods decompose navigation into VLM-based semantic reasoning and deterministic geometric execution, exposing a collaboration interface: the UAV's global view, over which it renders the UGV's reported pose and the VLM-anchored target as CAR/GOAL markers with distance labels, yielding a shared bird's-eye map. From this map, the UGV acquires global spatial context its first-person view cannot provide, plans a road-following path with a frozen VLM, and executes it under closed-loop control; in parallel, the UAV runs 3D-SPF, a spatial-search upgrade of SPF that localizes the target in the downward view and flies toward it. On 100 closed-loop episodes in CARLA-Air's Town10HD scene, AGC-VLN reaches a 77.0% joint success rate, a collaboration gain of +27.0% over the weaker individual agent (the UAV, 50.0%), and exceeds the strongest published single-agent baseline (Travel UAV, 53.0%) by 24.0 points, stemming from the complementarity of the UAV's global view and the UGV's road-following execution. Project page: https://github.com/ZSN2024/AGC-VLN.

Related papers

• AeroGround: A Comprehensive Benchmark for Aerial-Ground Collaborative Reasoning [28.18995637655191]
  We introduce AeroGround, a benchmark for evaluating vision-language models (VLMs) in aerial-ground collaborative reasoning.<n>AeroGround is built upon a simulated aerial-ground dataset containing approximately 29,000 multimodal observation groups from diverse open environments.<n>Experiments reveal a substantial gap between current models and human performance: the best model achieves an average accuracy of 54.4%, whereas humans reach 93.3%.
  arXiv  Detail & Related papers  (2026-08-12T07:31:15Z)
• CoNav-UAV: Cooperative Dual-Altitude Aerial Navigation via Stackelberg Learning [36.051483242181014]
  Target-oriented vision-and-language navigation (VLN) on aerial platforms is attracting growing attention for missions such as disaster rescue, infrastructure inspection, and security patrol.<n>Here we propose CoNav-UAV, which explicitly models the task as a Stackelberg game between a high-altitude leader and a low-altitude follower.
  arXiv  Detail & Related papers  (2026-08-03T07:10:38Z)
• DynFly: Dynamic-Aware Continuous Trajectory Generation for UAV Vision-Language Navigation in Urban Environments [55.73595346639892]
  We propose DynFly to bridge high-level navigation reasoning and executable UAV motion.<n>DynFly represents expert trajectories in B-spline control-point space and employs a Spline-DiT generator to learn conditional trajectory generation.<n>Experiments on the OpenUAV UAV-VLN benchmark show that DynFly improves both navigation performance and trajectory quality.
  arXiv  Detail & Related papers  (2026-06-30T13:33:48Z)
• Can Aerial VLA Models Cooperate? Evaluating Closed-Loop Air-Ground Coordination with CARLA-Air [2.061094726550731]
  We study whether single-UAV capabilities can transfer to air-ground cooperation.<n>We use CARLA-Air, a single-process evaluation environment that unifies CARLA and AirSim inside one Unreal Engine runtime.<n>Results show that current aerial VLA models can often track or follow a ground partner, but struggle to convert this single-agent competence into stable cooperative behavior.
  arXiv  Detail & Related papers  (2026-05-29T09:36:08Z)
• Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigation [60.07205156194741]
  Embodied navigation requires an agent to map language and visual observations to a stream of spatial actions that drive a real robot through environments it has never seen.<n>We present Uni-LaViRA, a unified agentic architecture that extends the same insight to four task families and to four heterogeneous real robots.
  arXiv  Detail & Related papers  (2026-05-26T18:52:04Z)
• Beyond Matching to Tiles: Bridging Unaligned Aerial and Satellite Views for Vision-Only UAV Navigation [51.286599397552756]
  We present Bearing-UAV, a vision-driven cross-view navigation method that jointly predicts UAV absolute location and heading from neighboring features.<n>We also present Bearing-UAV-90k, a benchmark for evaluating cross-view localization and navigation.
  arXiv  Detail & Related papers  (2026-03-23T16:17:39Z)
• AeroDuo: Aerial Duo for UAV-based Vision and Language Navigation [34.63571674882289]
  Aerial Vision-and-Language Navigation (VLN) is an emerging task that enables Unmanned Aerial Vehicles (UAVs) to navigate outdoor environments using natural language instructions and visual cues.<n>We introduce a novel task called Dual-Altitude UAV Collaborative VLN (DuAl-VLN)<n>In this task, two UAVs operate at distinct altitudes: a high-altitude UAV responsible for broad environmental reasoning, and a low-altitude UAV tasked with precise navigation.
  arXiv  Detail & Related papers  (2025-08-21T04:43:35Z)
• AGC-Drive: A Large-Scale Dataset for Real-World Aerial-Ground Collaboration in Driving Scenarios [68.84774511206797]
  We present AGC-Drive, the first large-scale real-world dataset for Aerial-Ground Cooperative 3D perception.<n>AGC-Drive contains 350 scenes, each with approximately 100 frames and fully annotated 3D bounding boxes covering 13 object categories.<n>We provide benchmarks for two 3D perception tasks: vehicle-to-vehicle collaborative perception and vehicle-to-Ground collaborative perception.
  arXiv  Detail & Related papers  (2025-06-19T14:48:43Z)
• Exploring the best way for UAV visual localization under Low-altitude Multi-view Observation Condition: a Benchmark [6.693781685584959]
  Low-altitude, Multi-view UAV AVL presents greater challenges due to extreme viewpoint changes.<n>This benchmark revealed the challenges of Low-altitude, Multi-view UAV AVL and provided valuable guidance for future research.
  arXiv  Detail & Related papers  (2025-03-12T03:29:27Z)
• A Multi-UAV System for Exploration and Target Finding in Cluttered and GPS-Denied Environments [68.31522961125589]
  We propose a framework for a team of UAVs to cooperatively explore and find a target in complex GPS-denied environments with obstacles. The team of UAVs autonomously navigates, explores, detects, and finds the target in a cluttered environment with a known map. Results indicate that the proposed multi-UAV system has improvements in terms of time-cost, the proportion of search area surveyed, as well as successful rates for search and rescue missions.
  arXiv  Detail & Related papers  (2021-07-19T12:54:04Z)
• Attention-based Reinforcement Learning for Real-Time UAV Semantic Communication [53.46235596543596]
  We study the problem of air-to-ground ultra-reliable and low-latency communication (URLLC) for a moving ground user. We propose a novel multi-agent deep reinforcement learning framework, coined a graph attention exchange network (GAXNet) GAXNet achieves 6.5x lower latency with the target 0.0000001 error rate, compared to a state-of-the-art baseline framework.
  arXiv  Detail & Related papers  (2021-05-22T12:43:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.