GRASP: Granularity-Aware Region Alignment and Semantic Prototype Learning for Fine-Grained Cross-Modal Understanding in Drone Views

• URL: http://arxiv.org/abs/2608.09270v1
• Date: Mon, 10 Aug 2026 08:26:47 GMT
• Title: GRASP: Granularity-Aware Region Alignment and Semantic Prototype Learning for Fine-Grained Cross-Modal Understanding in Drone Views
• Abstract summary: Fine-grained cross-modal understanding is essential for aerial vision-language navigation.<n>At the macro level, overwhelming background clutter in visual representations leads to Cross-Modal Focus Misalignment.<n>Visual Isomorphism creates ambiguity, where candidates share similar geometric structures yet differ only in subtle attributes.
• Score: 12.957721462609491
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fine-grained cross-modal understanding in drone views is essential for aerial vision-language navigation. However, the inherent wide field of view and overhead perspective of drone scenarios impose dual challenges on vision-language understanding. At the macro level, overwhelming background clutter in visual representations leads to Cross-Modal Focus Misalignment, where the model prioritizes global environmental similarities over specific object details. At the micro level, Visual Isomorphism creates ambiguity, where candidates share similar geometric structures yet differ only in subtle attributes. To address these challenges, we propose the Granularity-Aware Region Alignment and Semantic Prototype (GRASP) learning framework, enhancing discriminative capability through two synergistic strategies. Specifically, we introduce Region-Focused Alignment (RFA) to promote object-centric cross-modal alignment while suppressing background interference. Concurrently, to tackle visual isomorphism, we propose Semantic Perturbation Enhanced Matching (SPEM), which leverages a foreground-purified Semantic Prototype Codebook (SPC) to construct semantically perturbed negatives for fine-grained semantic discrimination. Extensive experiments on the GeoText-1652 benchmark and the unseen ERA dataset demonstrate that GRASP achieves competitive performance in drone-view fine-grained image-text retrieval, validating its effectiveness for cross-modal understanding in aerial scenarios. Our code implementation is available at https://github.com/UCAS-JC/GRASP.

Related papers

• OmniGF: A Dual-Branch Vision-Language Framework for Unified Gaze Following [59.53720386342017]
  We propose a unified vision-language framework that adapts foundational Vision-Language Models for scalable multi-person gaze reasoning.<n>By modeling all individuals, OmniGF seamlessly integrates precise spatial gaze target estimation, semantic gaze prediction, and complex social gaze reasoning.
  arXiv  Detail & Related papers  (2026-05-26T00:08:06Z)
• LaCoVL-FER: Landmark-Guided Contrastive Learning Network with Vision-Language Enhancement for Facial Expression Recognition [51.70817823155725]
  We propose a novel landmark-guided contrastive learning network with vision-language enhancement for Facial Expression Recognition (FER)<n>LaCoVL-FER integrates geometric priors from facial landmarks and semantic priors from a vision-language model.<n>Experiments demonstrate that LaCoVL-FER outperforms state-of-the-art methods on three representative real-world FER datasets.
  arXiv  Detail & Related papers  (2026-05-19T13:15:41Z)
• Geospatial-Reasoning-Driven Vocabulary-Agnostic Remote Sensing Semantic Segmentation [13.743073097114461]
  Open-vocabulary semantic segmentation has emerged as a promising research direction in remote sensing.<n>We propose a Geospatial Reasoning Chain-of-Thought (GR-CoT) framework to guide open-vocabulary segmentation models toward precise mapping.
  arXiv  Detail & Related papers  (2026-02-09T02:09:21Z)
• Revisiting Multi-Task Visual Representation Learning [52.93947931352643]
  We introduce MTV, a principled multi-task visual pretraining framework.<n>We leverage high-capacity "expert" models to synthesize dense, structured pseudo-labels at scale.<n>Our results demonstrate that MTV achieves "best-of-both-worlds" performance.
  arXiv  Detail & Related papers  (2026-01-20T11:59:19Z)
• Background Matters Too: A Language-Enhanced Adversarial Framework for Person Re-Identification [1.409283414986451]
  We argue that background semantics are as important as the foreground semantics in ReID.<n>This paper proposes an end-to-end framework that jointly models foreground and background information.
  arXiv  Detail & Related papers  (2025-09-03T05:38:22Z)
• Constrained Prompt Enhancement for Improving Zero-Shot Generalization of Vision-Language Models [57.357091028792325]
  Vision-language models (VLMs) pre-trained on web-scale data exhibit promising zero-shot generalization but often suffer from semantic misalignment.<n>We propose a novel constrained prompt enhancement (CPE) method to improve visual-textual alignment.<n>Our approach consists of two key components: Topology-Guided Synonymous Semantic Generation (TGSSG) and Category-Agnostic Discriminative Region Selection (CADRS)
  arXiv  Detail & Related papers  (2025-08-24T15:45:22Z)
• Scale-wise Bidirectional Alignment Network for Referring Remote Sensing Image Segmentation [12.893224628061516]
  The goal of referring remote sensing image segmentation (RRSIS) is to extract specific pixel-level regions within an aerial image via a natural language expression.<n>We propose an innovative framework called Scale-wise Bidirectional Alignment Network (SBANet) to address these challenges.<n>Our proposed method achieves superior performance in comparison to previous state-of-the-art methods on the RRSIS-D and RefSegRS datasets.
  arXiv  Detail & Related papers  (2025-01-01T14:24:04Z)
• Aligning and Prompting Everything All at Once for Universal Visual Perception [79.96124061108728]
  APE is a universal visual perception model for aligning and prompting everything all at once in an image to perform diverse tasks. APE advances the convergence of detection and grounding by reformulating language-guided grounding as open-vocabulary detection. Experiments on over 160 datasets demonstrate that APE outperforms state-of-the-art models.
  arXiv  Detail & Related papers  (2023-12-04T18:59:50Z)
• Fine-Grained Semantically Aligned Vision-Language Pre-Training [151.7372197904064]
  Large-scale vision-language pre-training has shown impressive advances in a wide range of downstream tasks. Existing methods mainly model the cross-modal alignment by the similarity of the global representations of images and texts. We introduce LO, a fine-grained semantically aLigned visiOn-langUage PrE-training framework, which learns fine-grained semantic alignment from the novel perspective of game-theoretic interactions.
  arXiv  Detail & Related papers  (2022-08-04T07:51:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.