Exposing Vulnerabilities in Visible-Infrared VLMs: A Unified Geometric Adversarial Framework with Cross-Task Transferability

• URL: http://arxiv.org/abs/2605.22273v1
• Date: Thu, 21 May 2026 10:15:58 GMT
• Title: Exposing Vulnerabilities in Visible-Infrared VLMs: A Unified Geometric Adversarial Framework with Cross-Task Transferability
• Abstract summary: Vision-language models (VLMs) have achieved strong performance across diverse multimodal tasks, but their adversarial robustness in visible-infrared (VIS-IR) scenarios remains underexplored.<n>We propose CFGPatch, a curved-edge fractal geometric adversarial patch framework for attacking VIS-IR VLMs.
• Score: 11.461885964494199
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models (VLMs) have achieved strong performance across diverse multimodal tasks, but their adversarial robustness in visible-infrared (VIS-IR) scenarios remains underexplored. This gap is critical because VIS-IR sensing is widely used in real-world perception systems to support reliable understanding under challenging imaging conditions. To address this cross-modal threat setting, we propose CFGPatch, a curved-edge fractal geometric adversarial patch framework for attacking VIS-IR VLMs. CFGPatch builds on triangular fractal geometry and replaces rigid straight-edged primitives with Bezier-curved elements, preserving multi-scale fractal self-similarity while introducing smoother contours, richer directional variation, and more flexible shape deformation. In addition, we design a modality-specific Fraser-spiral rendering mechanism to inject fine-grained texture distortions and misleading perceptual cues into visible and infrared images. By coupling global curved-fractal geometry with local spiral-based appearance interference, CFGPatch disrupts both shape perception and texture interpretation. We further adopt expectation over transformation (EOT) to improve robustness against common image-level transformations. Extensive experiments show that CFGPatch effectively fools VIS-IR VLMs and consistently outperforms standard patch baselines in attack effectiveness and robustness. Moreover, adversarial samples optimized for zero-shot classification transfer well to image captioning and visual question answering, demonstrating strong cross-task transferability and generalizability across downstream tasks.

Related papers

• Partial Skeleton Visibility for Action Recognition: A Constrained Field-of-View Approach [60.596944437968524]
  PartialVisGraph is a novel framework for robust skeleton-based action recognition under constrained field-of-view.<n>It consistently achieves state-of-the-art accuracy under partial visibility, with gains of up to 68.8% on subsets with severe FoV restrictions.<n>Our approach offers a principled and practical pathway toward deployable skeleton-based action understanding in unconstrained environments.
  arXiv  Detail & Related papers  (2026-07-01T10:03:11Z)
• CogSENet: Blind Image Deblurring with Blur-Conditioned Semantic Routing and Explicit Frequency Fusion [3.7521959864518593]
  CogSENet is a dynamic, semantic-aligned reconstruction framework inspired by the eagle's visual system.<n>CogSENetoutperforms state-of-the-art deblurring methods in both visual quality and structural fidelity with fewer parameters.
  arXiv  Detail & Related papers  (2026-06-29T09:29:21Z)
• ExACT: Exemplar-Driven Calibrated Refinement for Training-Free Visual Grounding in Remote Sensing Images [51.53960096934913]
  Remote sensing visual grounding (RSVG) aims to locate specific objects in high-resolution RS imagery using free-form natural language descriptions.<n>Recent advances in multimodal large language models (MLLMs) show great potential for such open-vocabulary RSVG.<n>We propose Exemplar-driven Calibrated Refinement (ExACT), a training-free framework driven by a one-shot visual prompting mechanism.
  arXiv  Detail & Related papers  (2026-06-27T13:50:39Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• A Systematic Failure Analysis of Vision Foundation Models for Open Set Iris Presentation Attack Detection [6.472008340680056]
  This work presents a systematic failure analysis of general-purpose vision foundation models for open-set iris PAD using periocular imagery.<n>Results indicate that foundation models can transfer across datasets with similar sensing characteristics, but fail to generalise reliably to unseen attack instruments.<n>These findings show that strong closed-set or cross-dataset performance should not be treated as evidence of robust open-set security.
  arXiv  Detail & Related papers  (2026-05-18T18:41:26Z)
• BGG: Bridging the Geometric Gap between Cross-View images by Vision Foundation Model Adaptation for Geo-Localization [50.74663742490919]
  Cross-View Geo-Localization (CVGL) aims to acquire the geolocation of images by image retrieval.<n>This paper proposes a parameter-efficient framework for bridging the geometric gap across images based on the vision foundation model (VFM)<n>It mainly contains a Multi-granularity Feature Enhancement Adapter (MFEA) and a Frequency-Aware Structural Aggregation (FASA) module.
  arXiv  Detail & Related papers  (2026-05-11T10:46:33Z)
• Physically-Induced Atmospheric Adversarial Perturbations: Enhancing Transferability and Robustness in Remote Sensing Image Classification [54.952203312050564]
  Adversarial attacks pose a severe threat to the reliability of deep learning models in remote sensing (RS) image classification.<n>We propose FogFool, a physically plausible adversarial framework that generates fog-based perturbations.<n>Experiments on two benchmark RS datasets demonstrate that FogFool achieves superior performance.
  arXiv  Detail & Related papers  (2026-04-16T05:47:57Z)
• Revealing Physical-World Semantic Vulnerabilities: Universal Adversarial Patches for Infrared Vision-Language Models [21.429674567539607]
  Infrared vision-language models (IR-VLMs) have emerged as a promising paradigm for multimodal perception in low-visibility environments.<n>Existing adversarial patch methods are mainly designed for RGB-based models in closed-set settings.<n>We propose Universal Curved-Grid Patch (UCGP), a universal physical adversarial patch framework for IR-VLMs.
  arXiv  Detail & Related papers  (2026-04-03T15:42:55Z)
• Perception-Aware Multimodal Spatial Reasoning from Monocular Images [57.42071289037214]
  spatial reasoning from monocular images is essential for autonomous driving.<n>Current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception.<n>We propose a perception-aware multimodal reasoning framework that equipsVLMs with explicit object-centric grounding ability.
  arXiv  Detail & Related papers  (2026-03-07T02:05:12Z)
• Adversarial Patch Generation for Visual-Infrared Dense Prediction Tasks via Joint Position-Color Optimization [14.358458317718174]
  We propose a joint position-color optimization framework (AP-PCO) for generating adversarial patches in visual-infrared settings.<n>We introduce a crossmodal color adaptation strategy that constrains patch appearance according to infrared grayscale characteristics.<n> experiments on visual-infrared dense prediction tasks demonstrate that the proposed AP-PCO achieves consistently strong attack performance.
  arXiv  Detail & Related papers  (2026-02-27T19:26:17Z)
• Multi-Sensor Matching with HyperNetworks [14.911092205861822]
  We leverage hypernetworks to improve multimodal patch matching.<n>We introduce a lightweight descriptor-learning architecture that augments a Siamese CNN.<n>We also release GAP-VIR, a cross-platform (ground/aerial) VIS-IR patch dataset with 500K pairs.
  arXiv  Detail & Related papers  (2026-01-18T09:19:33Z)
• SIFT-Graph: Benchmarking Multimodal Defense Against Image Adversarial Attacks With Robust Feature Graph [0.6999740786886536]
  Adversarial attacks expose a fundamental vulnerability in modern deep vision models.<n>We introduce SIFT-Graph, a multimodal defense framework that enhances the robustness of traditional vision models.<n>Preliminary results demonstrate that our method effectively improves the visual model robustness against gradient-based white box adversarial attacks.
  arXiv  Detail & Related papers  (2025-11-11T22:23:27Z)
• Adaptive Transformers for Robust Few-shot Cross-domain Face Anti-spoofing [71.06718651013965]
  We present adaptive vision transformers (ViT) for robust cross-domain face antispoofing. We adopt ViT as a backbone to exploit its strength to account for long-range dependencies among pixels. Experiments on several benchmark datasets show that the proposed models achieve both robust and competitive performance.
  arXiv  Detail & Related papers  (2022-03-23T03:37:44Z)
• Intriguing Properties of Vision Transformers [114.28522466830374]
  Vision transformers (ViT) have demonstrated impressive performance across various machine vision problems. We systematically study this question via an extensive set of experiments and comparisons with a high-performing convolutional neural network (CNN) We show effective features of ViTs are due to flexible receptive and dynamic fields possible via the self-attention mechanism.
  arXiv  Detail & Related papers  (2021-05-21T17:59:18Z)
• Deep Semantic Matching with Foreground Detection and Cycle-Consistency [103.22976097225457]
  We address weakly supervised semantic matching based on a deep network. We explicitly estimate the foreground regions to suppress the effect of background clutter. We develop cycle-consistent losses to enforce the predicted transformations across multiple images to be geometrically plausible and consistent.
  arXiv  Detail & Related papers  (2020-03-31T22:38:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.