GeoThreat: Transferable Targeted Adversarial Attacks on Large Vision-Language Models for Remote Sensing Image Interpretation

• URL: http://arxiv.org/abs/2607.21036v1
• Date: Thu, 23 Jul 2026 08:16:56 GMT
• Title: GeoThreat: Transferable Targeted Adversarial Attacks on Large Vision-Language Models for Remote Sensing Image Interpretation
• Abstract summary: Adversarial attacks against large vision-language models (LVLMs) serve as an effective means of assessing their robustness in cross-modal semantic understanding.<n>We propose GeoThreat, a transferable targeted adversarial attack method against LVLMs for remote sensing image interpretation.
• Score: 25.134254498593137
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adversarial attacks against large vision-language models (LVLMs) serve as an effective means of assessing their robustness in cross-modal semantic understanding. Existing studies mainly focus on corrupting visual inputs to induce predefined erroneous responses in general vision-language tasks, whereas corresponding investigations in remote sensing fields remain largely underexplored. Compared with natural image understanding, remote sensing image interpretation requires joint reasoning over local discriminative cues and global scene context. This poses additional challenges to achieving transferable semantic manipulation toward specified responses under black-box settings. To tackle these challenges, we propose GeoThreat, a transferable targeted adversarial attack method against LVLMs for remote sensing image interpretation. Specifically, GeoThreat modulates adversarial representations in accordance with the target content at both conceptual and perceptual levels. The class tokens from surrogate image encoders are employed as conceptual representations, while perceptual representations are distilled from patch tokens of the adversarial example through collaborative importance estimation. Beyond merely rolling out attention scores across layers, we incorporate adversarial-target similarity gradients to more faithfully characterize the relevance of local visual cues to the intended semantic manipulation. The perceptual representations are then dynamically aligned with target patch tokens in a cross-attentive manner, facilitating the adaptation of local cues toward designated semantic details. Finally, adversarial perturbations are iteratively updated via ensemble-based joint optimization of conceptual calibration and perceptual adaptation. Extensive experiments across diverse LVLMs demonstrate the superiority of GeoThreat in both transferability and controllability.

Related papers

• Semantic-Spatial Discriminability Enhancement for Generalized Visual Grounding [28.39401600080181]
  Generalized Visual Grounding (GVG) task aims to localize targets in an image based on referring expressions.<n>This paper proposes a novel Semantic-Spatial Discriminability Enhancement (SSDE) framework for generalized visual grounding.<n>SSDE aims to enhance the discriminative ability on fine-grained semantics and spatial localization.
  arXiv  Detail & Related papers  (2026-08-31T04:37:36Z)
• Primitive-Driven Compositional Forensic Visual Prompting for Open-World Face Anti-Spoofing [59.765442274924595]
  We propose a compositional visual prompt learning framework that operates entirely in the visual feature space.<n>The framework employs patch-aware attention to refine a shared set of learnable micro-forensic primitives into localized forensic evidence units.<n>Experiments on nine open-world protocols demonstrate state-of-the-art performance, strong cross-domain generalization, and robust adaptation to unseen attacks.
  arXiv  Detail & Related papers  (2026-08-18T04:18:08Z)
• Warp-free Cross-view Geo-localization via Feature-space Consensus Mining [49.843311525678565]
  Cross-view geo-localization is challenging due to drastic viewpoint changes and large appearance discrepancies between street-level and satellite imagery.<n>We propose a novel joint-view consensus-guided learning framework that entirely bypasses explicit geometric warping.
  arXiv  Detail & Related papers  (2026-08-10T09:02:06Z)
• Attention Hijacking: Response Manipulation Across Queries in Vision-Language Models [57.870323273127234]
  Motivated by the observation, we propose textbfAttention Hijacking, a novel adversarial attack that steers internal attention toward a persistent image-dominant pattern.<n>Our method reduces the dependence of the manipulated output on the specific wording of the query.
  arXiv  Detail & Related papers  (2026-05-17T08:02:27Z)
• Seeking Consensus: Geometric-Semantic On-the-Fly Recalibration for Open-Vocabulary Remote Sensing Semantic Segmentation [27.476148859056114]
  SeeCo is a plug-and-play framework to boost the performance of training-free OVSS models in remote sensing images.<n>It recalibrates arbitrary OVSS models on-the-fly by seeking dual consensus.<n>Experiments on eight remote sensing OVSS benchmarks show consistent gains.
  arXiv  Detail & Related papers  (2026-04-29T01:57:21Z)
• Perception-Aware Multimodal Spatial Reasoning from Monocular Images [57.42071289037214]
  spatial reasoning from monocular images is essential for autonomous driving.<n>Current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception.<n>We propose a perception-aware multimodal reasoning framework that equipsVLMs with explicit object-centric grounding ability.
  arXiv  Detail & Related papers  (2026-03-07T02:05:12Z)
• SGHA-Attack: Semantic-Guided Hierarchical Alignment for Transferable Targeted Attacks on Vision-Language Models [73.19044613922911]
  Large vision-language models (VLMs) are vulnerable to transfer-based adversarial perturbations.<n>We propose SGHA-Attack, a framework that adopts multiple target references and enforces intermediate-layer consistency.<n>Experiments on open-source and commercial black-box VLMs show that SGHA-Attack achieves stronger targeted transferability than prior methods.
  arXiv  Detail & Related papers  (2026-02-02T03:10:41Z)
• TRUST: Leveraging Text Robustness for Unsupervised Domain Adaptation [9.906359339999039]
  We introduce a novel UDA approach that exploits the robustness of the language modality to guide the adaptation of a vision model.<n>We propose a multimodal soft-contrastive learning loss that aligns the vision and language feature spaces.<n>Our approach outperforms previous methods, setting the new state-of-the-art on classical (DomainNet) and complex (GeoNet) domain shifts.
  arXiv  Detail & Related papers  (2025-08-08T16:51:44Z)
• Weakly-Supervised Image Forgery Localization via Vision-Language Collaborative Reasoning Framework [16.961220047066792]
  ViLaCo is a vision-language collaborative reasoning framework that introduces auxiliary semantic supervision distilled from pre-trained vision-language models.<n>ViLaCo substantially outperforms existing WSIFL methods, achieving state-of-the-art performance in both detection and localization accuracy.
  arXiv  Detail & Related papers  (2025-08-02T12:14:29Z)
• Referring Remote Sensing Image Segmentation with Cross-view Semantics Interaction Network [65.01521002836611]
  We propose a paralleled yet unified segmentation framework Cross-view Semantics Interaction Network (CSINet) to solve the limitations.<n>Motivated by human behavior in observing targets of interest, the network orchestrates visual cues from remote and close distances to conduct synergistic prediction.<n>In its every encoding stage, a Cross-View Window-attention module (CVWin) is utilized to supplement global and local semantics into close-view and remote-view branch features.
  arXiv  Detail & Related papers  (2025-08-02T11:57:56Z)
• IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasis [46.502962768034166]
  Zero-shot Referring Image identifies the instance mask that best aligns with a referring expression without training and fine-tuning.<n>Previous CLIP-based models exhibit a notable reduction in their capacity to discern relative spatial relationships of objects.<n>IteRPrimE outperforms previous state-of-the-art zero-shot methods, particularly excelling in out-of-domain scenarios.
  arXiv  Detail & Related papers  (2025-03-02T15:19:37Z)
• Enhancing Cross-Prompt Transferability in Vision-Language Models through Contextual Injection of Target Tokens [28.356269620160937]
  We propose a Contextual-Injection Attack (CIA) that employs gradient-based perturbation to inject target tokens into both visual and textual contexts. CIA enhances the cross-prompt transferability of adversarial images.
  arXiv  Detail & Related papers  (2024-06-19T07:32:55Z)
• ZoomNeXt: A Unified Collaborative Pyramid Network for Camouflaged Object Detection [70.11264880907652]
  Recent object (COD) attempts to segment objects visually blended into their surroundings, which is extremely complex and difficult in real-world scenarios. We propose an effective unified collaborative pyramid network that mimics human behavior when observing vague images and camouflaged zooming in and out. Our framework consistently outperforms existing state-of-the-art methods in image and video COD benchmarks.
  arXiv  Detail & Related papers  (2023-10-31T06:11:23Z)
• Enhancing the Self-Universality for Transferable Targeted Attacks [88.6081640779354]
  Our new attack method is proposed based on the observation that highly universal adversarial perturbations tend to be more transferable for targeted attacks. Instead of optimizing the perturbations on different images, optimizing on different regions to achieve self-universality can get rid of using extra data. With the feature similarity loss, our method makes the features from adversarial perturbations to be more dominant than that of benign images.
  arXiv  Detail & Related papers  (2022-09-08T11:21:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.