From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation

• URL: http://arxiv.org/abs/2608.09564v1
• Date: Mon, 10 Aug 2026 12:59:50 GMT
• Title: From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation
• Abstract summary: UAV vision-language navigation (UAV-VLN) focuses on enabling an aerial agent to follow natural-language instructions in open 3D environments.<n>Current approaches suffer from three coupled issues: weak grounding of instruction-relevant landmarks in visual observations, insufficient exploitation of long-horizon history, and unstable decisions under local traps or repeated exploration.<n>We propose a unified semantic-to-decision framework to address these issues.
• Score: 40.51712647240084
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: UAV vision-language navigation (UAV-VLN) focuses on enabling an aerial agent to follow natural-language instructions in open 3D environments from egocentric visual observations. Current approaches suffer from three coupled issues: weak grounding of instruction-relevant landmarks in visual observations, insufficient exploitation of long-horizon history, and unstable decisions under local traps or repeated exploration. To address these issues, we propose a unified semantic-to-decision framework. First, we present an instruction-grounded semantic enhancement module that injects object-level semantics and relative spatial cues into the current observation state. Subsequently, we develop a relevance-aware dynamic temporal aggregation strategy that reweights the full history buffer while converting a few high-relevance frames into structured landmark prompts for the decoder. Finally, we devise a topology-aware decision method that combines local-optimum cognition with group-relative policy optimization under progress, goal, semantic, and path-compliance rewards. Experiments on the widely used AerialVLN and OpenFly benchmarks clearly demonstrate that our method achieves state-of-the-art performance.

Related papers

• Spatial-Semantic Reasoning using Large Language Models for Efficient UAV Search Operations [2.012425476229879]
  We present a real-time semantic navigation framework for Unmanned Aerial Vehicles (UAVs) focused on improving time efficiency in the Object Goal Navigation (ObjectNav) task.<n>A Large Language Model (LLM) interprets user-provided natural language instructions and performs semantic reasoning over detected objects.<n>Experiments in both simulated and real-world settings demonstrate reductions in mission duration while maintaining high search accuracy.
  arXiv  Detail & Related papers  (2026-08-28T12:33:49Z)
• EgoHieraLoc: A Cortically Inspired Hierarchical Segmentation-Guided Framework for Egocentric Visual Query Localization [9.547789251222838]
  Visual query localization (VQL) aims to retrieve and re-localize a queried object in egocentric videos.<n>Human vision handles ambiguity through a hierarchical process.<n>We propose textbfEgoHieraLoc, a unified framework for VQL-2D and VQL-3D.
  arXiv  Detail & Related papers  (2026-08-10T14:31:01Z)
• dinov3.seg: Open-Vocabulary Semantic Segmentation with DINOv3 [36.6036728217708]
  Open-Vocabulary Semantics (OVSS) assigns pixel-level labels from an open set of text-defined categories, demanding reliable generalization to unseen classes at inference.<n>We introduce dinov3.seg, extending dinov3.txt into a dedicated framework for OVSS.<n>Our contributions are four-fold. First, we design a task-specific architecture tailored to this backbone, systematically adapting established design principles from prior open-vocabulary segmentation work.<n>Second, we jointly leverage text embeddings aligned with both the global [] token and local patch-level visual features from ViT-based encoder
  arXiv  Detail & Related papers  (2026-03-19T23:57:28Z)
• AgentVLN: Towards Agentic Vision-and-Language Navigation [78.739525400071]
  Vision-and-Language Navigation (VLN) requires an embodied agent to ground complex natural-language instructions into long-horizon navigation in unseen environments.<n>In this paper, we propose AgentVLN, a novel and efficient embodied navigation framework that can be deployed on edge computing platforms.
  arXiv  Detail & Related papers  (2026-03-18T12:43:47Z)
• DecoVLN: Decoupling Observation, Reasoning, and Correction for Vision-and-Language Navigation [49.233063630722334]
  We propose DecoVLN, a framework for robust streaming perception and closed-loop control in long-horizon navigation.<n>We introduce adaptive refinement mechanism that selects frames from a historical candidate pool by iteratively optimizing a unified scoring function.<n>We also introduce a state-action pair-level corrective finetuning strategy to alleviate compounding errors.
  arXiv  Detail & Related papers  (2026-03-13T16:24:37Z)
• RSGround-R1: Rethinking Remote Sensing Visual Grounding through Spatial Reasoning [61.84363374647606]
  Remote Sensing Visual Grounding (RSVG) aims to localize target objects in large-scale aerial imagery based on natural language descriptions.<n>These descriptions often rely heavily on positional cues, posing unique challenges for Multimodal Large Language Models (MLLMs) in spatial reasoning.<n>We propose a reasoning-guided, position-aware post-training framework, dubbed textbfRSGround-R1, to progressively enhance spatial understanding.
  arXiv  Detail & Related papers  (2026-01-29T12:35:57Z)
• ST-Booster: An Iterative SpatioTemporal Perception Booster for Vision-and-Language Navigation in Continuous Environments [1.9566515100805284]
  VLN-CE requires agents to navigate continuous spaces based on natural language instructions.<n>This paper introduces ST-Booster, a navigation booster that enhances performance through multi-granularity perception and instruction-aware reasoning.<n>Extensive experiments and performance analyses are conducted, demonstrating that ST-Booster outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-04-14T03:29:08Z)
• Without Paired Labeled Data: End-to-End Self-Supervised Learning for Drone-view Geo-Localization [20.603433987118837]
  Drone-view Geo-Localization (DVGL) aims to achieve accurate localization of drones by retrieving the most relevant GPS-tagged satellite images.<n>Existing methods heavily rely on strictly pre-paired drone-satellite images for supervised learning.<n>We propose a novel end-to-end self-supervised learning method with a shallow backbone network.
  arXiv  Detail & Related papers  (2025-02-17T02:53:08Z)
• UnitedVLN: Generalizable Gaussian Splatting for Continuous Vision-Language Navigation [71.97405667493477]
  We introduce a novel, generalizable 3DGS-based pre-training paradigm, called UnitedVLN.<n>It enables agents to better explore future environments by unitedly rendering high-fidelity 360 visual images and semantic features.<n>UnitedVLN outperforms state-of-the-art methods on existing VLN-CE benchmarks.
  arXiv  Detail & Related papers  (2024-11-25T02:44:59Z)
• Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completion [57.232688209606515]
  We present HTCL, a novel Temporal Temporal Context Learning paradigm for improving camera-based semantic scene completion. Our method ranks $1st$ on the Semantic KITTI benchmark and even surpasses LiDAR-based methods in terms of mIoU.
  arXiv  Detail & Related papers  (2024-07-02T09:11:17Z)
• SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environments [7.5606260987453116]
  This paper presents a novel approach for the Vision-and-Language Navigation (VLN) task in continuous 3D environments. Existing end-to-end learning-based methods struggle at this task as they focus mostly on raw visual observations. We present a hybrid transformer-recurrence model which focuses on combining classical semantic mapping techniques with a learning-based method.
  arXiv  Detail & Related papers  (2021-08-26T17:57:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.