Adversarial Attacks Against MLLMs via Progressive Resolution Processing and Adaptive Feature Alignment

• URL: http://arxiv.org/abs/2605.09902v1
• Date: Mon, 11 May 2026 02:45:48 GMT
• Title: Adversarial Attacks Against MLLMs via Progressive Resolution Processing and Adaptive Feature Alignment
• Abstract summary: Adversarial perturbations can mislead Multimodal Large Language Models (MLLMs) recognize a benign image as a specific target object.<n>This makes transfer-based targeted attacks crucial for understanding and improving black-box MLLM robustness.<n>We propose PRAF-Attack, a targeted transfer-based attack framework that integrates multi-scale global semantic guidance with robust intermediate-layer local alignment.
• Score: 56.085182951399496
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adversarial perturbations can mislead Multimodal Large Language Models (MLLMs) recognize a benign image as a specific target object, posing serious risks in safety-critical scenarios such as autonomous driving and medical diagnosis. This makes transfer-based targeted attacks crucial for understanding and improving black-box MLLM robustness. Existing transfer-based targeted attack methods typically rely on the final global features of the surrogate encoder and anchor optimization to original-resolution target crops, leading to their limited transferability and robustness. To address these challenges, we propose Progressive Resolution Processing and Adaptive Feature Alignment (PRAF-Attack), a targeted transfer-based attack framework that integrates multi-scale global semantic guidance with robust intermediate-layer local alignment. Unlike prior methods that align only the surrogate encoder's final layer, we design an adaptive feature alignment strategy that leverages intermediate representations to enhance transferability. Specifically, we introduce an adaptive intermediate layer selection mechanism to identify transferable hierarchical features across surrogate ensembles via gradient consistency, along with an adaptive patch-level optimization strategy that preserves highly correlated local regions through efficient patch filtering. To overcome the reliance on fixed original-resolution target crops, we propose a progressive resolution processing strategy that gradually refines optimization from coarse to fine, enabling the attack to better exploit target information at multiple scales and achieve stronger transferability. We evaluate PRAF-Attack on a diverse suite of black-box MLLMs, including six open-source models and six closed-source commercial APIs. Compared with seven state-of-the-art targeted attack baselines, the proposed PRAF-Attack consistently achieves superior transferability.

Related papers

• Frequency-Domain Regularized Adversarial Alignment for Transferable Attacks against Closed-Source MLLMs [59.973833105841685]
  A key challenge for improving adversarial transferability is to effectively capture the intrinsic visual focus shared across different models.<n>We propose FRA-Attack, which addresses both challenges from a unified frequency-domain regularization perspective.<n>For feature alignment, a high-pass DCT objective on patch features suppresses redundant global structures and concentrates the loss on the high-frequency band that carries the MLLMs' intrinsic visual focus.<n>For gradient optimization, we introduce Frequency-domain Gradient Regularization (FGR), a textitmodel-agnostic low-pass regularizer that modulates the surrogate gradient using
  arXiv  Detail & Related papers  (2026-05-20T08:15:56Z)
• AnchorRefine: Synergy-Manipulation Based on Trajectory Anchor and Residual Refinement for Vision-Language-Action Models [60.04879435087352]
  Vision-language-action (VLA) policies generate actions within a single unified space.<n>We propose AnchorRefine, a hierarchical framework that factorizes VLA action modeling into trajectory anchor and residual refinement.<n> Experiments on LIBERO, CALVIN, and real-robot tasks demonstrate that AnchorRefine consistently improves both regression-based and diffusion-based VLA backbones.
  arXiv  Detail & Related papers  (2026-04-20T04:25:24Z)
• SGHA-Attack: Semantic-Guided Hierarchical Alignment for Transferable Targeted Attacks on Vision-Language Models [73.19044613922911]
  Large vision-language models (VLMs) are vulnerable to transfer-based adversarial perturbations.<n>We propose SGHA-Attack, a framework that adopts multiple target references and enforces intermediate-layer consistency.<n>Experiments on open-source and commercial black-box VLMs show that SGHA-Attack achieves stronger targeted transferability than prior methods.
  arXiv  Detail & Related papers  (2026-02-02T03:10:41Z)
• Make Anything Match Your Target: Universal Adversarial Perturbations against Closed-Source MLLMs via Multi-Crop Routed Meta Optimization [49.30177419529011]
  We study a more stringent setting, Universal Targeted Transferable Adversarial Attacks (UTTAA)<n>A single perturbation must consistently steer arbitrary inputs toward a specified target across unknown commercial MLLMs.<n>We propose M CRMO-Attack, which stabilizes supervision via Multi-Crop Aggregation with an Attention-Guided Crop.
  arXiv  Detail & Related papers  (2026-01-30T17:03:24Z)
• MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization [56.074760766965085]
  Group-Relative Policy Optimization has emerged as an efficient paradigm for aligning Large Language Models (LLMs)<n>We propose MAESTRO, which treats reward scalarization as a dynamic latent policy, leveraging the model's terminal hidden states as a semantic bottleneck.<n>We formulate this as a contextual bandit problem within a bi-level optimization framework, where a lightweight Conductor network co-evolves with the policy by utilizing group-relative advantages as a meta-reward signal.
  arXiv  Detail & Related papers  (2026-01-12T05:02:48Z)
• Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignment [35.77916460821855]
  Multimodal large language models (MLLMs) remain vulnerable to transferable adversarial attacks.<n>We propose a targeted transferable adversarial attack method based on feature optimal alignment.<n>Experiments demonstrate the superiority of the proposed method, outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-05-27T17:56:57Z)
• HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-tuned LLM with Quantization [55.972018549438964]
  Federated fine-tuning of pre-trained Large Language Models (LLMs) enables task-specific adaptation across diverse datasets while preserving privacy.<n>We propose HAFLQ (Heterogeneous Adaptive Federated Low-Rank Adaptation Fine-tuned LLM with Quantization), a novel framework for efficient and scalable fine-tuning of LLMs in heterogeneous environments.<n> Experimental results on the text classification task demonstrate that HAFLQ reduces memory usage by 31%, lowers communication cost by 49%, improves accuracy by 50%, and achieves faster convergence compared to the baseline method.
  arXiv  Detail & Related papers  (2024-11-10T19:59:54Z)
• Pareto Domain Adaptation [35.48609986914723]
  Domain adaptation (DA) attempts to transfer the knowledge from a labeled source domain to an unlabeled target domain. We propose a new approach to control the overall optimization direction, aiming to cooperatively optimize all training objectives.
  arXiv  Detail & Related papers  (2021-12-08T06:31:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.