Related papers: Adversarial Prompt Injection Attack on Multimodal Large Language Models

Adversarial Prompt Injection Attack on Multimodal Large Language Models

URL: http://arxiv.org/abs/2603.29418v1
Date: Tue, 31 Mar 2026 08:22:07 GMT
Title: Adversarial Prompt Injection Attack on Multimodal Large Language Models
Abstract summary: multimodal large language models (MLLMs) are increasingly deployed in real-world applications.<n>Existing prompt injection methods rely on textual prompts or perceptible visual prompts that are observable by human users.<n>In this work, we study imperceptible visual prompt injection against powerful closed-source MLLMs.
Score: 21.995939723909796
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Although multimodal large language models (MLLMs) are increasingly deployed in real-world applications, their instruction-following behavior leaves them vulnerable to prompt injection attacks. Existing prompt injection methods predominantly rely on textual prompts or perceptible visual prompts that are observable by human users. In this work, we study imperceptible visual prompt injection against powerful closed-source MLLMs, where adversarial instructions are embedded in the visual modality. Our method adaptively embeds the malicious prompt into the input image via a bounded text overlay to provide semantic guidance. Meanwhile, the imperceptible visual perturbation is iteratively optimized to align the feature representation of the attacked image with those of the malicious visual and textual targets at both coarse- and fine-grained levels. Specifically, the visual target is instantiated as a text-rendered image and progressively refined during optimization to more faithfully represent the desired semantics and improve transferability. Extensive experiments on two multimodal understanding tasks across multiple closed-source MLLMs demonstrate the superior performance of our approach compared to existing methods.

Related papers

Test-Time Computing for Referring Multimodal Large Language Models [143.49848714354698]
We propose ControlMLLM++, a novel test-time adaptation framework.<n>It injects learnable visual prompts into frozen multimodal large language models.
arXiv Detail & Related papers (2026-02-23T04:42:10Z)
Unleashing the Intrinsic Visual Representation Capability of Multimodal Large Language Models [58.91911788912665]
We propose Latent Visual Reconstruction (LaVer), a novel training framework that facilitates MLLMs in learning more discrimi visual representations.<n>Our method offers direct visual activation to MLLMs, which exhibit increased visual attention allocation, indicating enhanced utilization of visual information.
arXiv Detail & Related papers (2025-12-06T04:20:13Z)
An Image Is Worth Ten Thousand Words: Verbose-Text Induction Attacks on VLMs [48.05423013052023]
This paper proposes a novel verbose-text induction attack (VTIA) to inject imperceptible adversarial perturbations into benign images.<n>We first perform adversarial prompt search, employing reinforcement learning strategies to automatically identify adversarial prompts.<n>We then conduct vision-aligned perturbation optimization to craft adversarial examples on input images, maximizing the similarity between the perturbed image's visual embeddings and those of the adversarial prompt.
arXiv Detail & Related papers (2025-11-20T09:03:43Z)
Security Tensors as a Cross-Modal Bridge: Extending Text-Aligned Safety to Vision in LVLM [40.83149588857177]
Large visual-language models (LVLMs) integrate aligned large language models (LLMs) with visual modules to process multimodal inputs.<n>We introduce security tensors - trainable input vectors applied during inference through either the textual or visual modality.
arXiv Detail & Related papers (2025-07-28T16:59:53Z)
ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models [67.75439511654078]
Large Vision-Language Models (LVLMs) have introduced a new paradigm for understanding and reasoning about image input through textual responses.<n>They face the persistent challenge of hallucination, which introduces practical weaknesses and raises concerns about their reliable deployment in real-world applications.<n>We propose ONLY, a training-free decoding approach that requires only a single query and a one-layer intervention during decoding, enabling efficient real-time deployment.
arXiv Detail & Related papers (2025-07-01T16:01:08Z)
Multimodal Prompt Alignment for Facial Expression Recognition [24.470095812039286]
MPA-FER provides fine-grained semantic guidance to the learning process of prompted visual features.<n>Our framework outperforms state-of-the-art methods on three FER benchmark datasets.
arXiv Detail & Related papers (2025-06-26T05:28:57Z)
EAGLE: Towards Efficient Arbitrary Referring Visual Prompts Comprehension for Multimodal Large Language Models [80.00303150568696]
We propose a novel Multimodal Large Language Models (MLLM) that empowers comprehension of arbitrary referring visual prompts with less training efforts than existing approaches. Our approach embeds referring visual prompts as spatial concepts conveying specific spatial areas comprehensible to the MLLM. We also propose a Geometry-Agnostic Learning paradigm (GAL) to further disentangle the MLLM's region-level comprehension with the specific formats of referring visual prompts.
arXiv Detail & Related papers (2024-09-25T08:22:00Z)
NoteLLM-2: Multimodal Large Representation Models for Recommendation [71.87790090964734]
Large Language Models (LLMs) have demonstrated exceptional proficiency in text understanding and embedding tasks.<n>Their potential in multimodal representation, particularly for item-to-item (I2I) recommendations, remains underexplored.<n>We propose an end-to-end fine-tuning method that customizes the integration of any existing LLMs and vision encoders for efficient multimodal representation.
arXiv Detail & Related papers (2024-05-27T03:24:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.