Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructions

• URL: http://arxiv.org/abs/2603.03637v1
• Date: Wed, 04 Mar 2026 01:58:08 GMT
• Title: Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructions
• Abstract summary: We study Image-based Prompt Injection (IPI), a black-box attack in which adversarial instructions are embedded into natural images to override model behavior.<n>Our end-to-end IPI pipeline incorporates segmentation-based region selection, adaptive font scaling, and background-aware rendering.
• Score: 7.575333495550402
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Large Language Models (MLLMs) integrate vision and text to power applications, but this integration introduces new vulnerabilities. We study Image-based Prompt Injection (IPI), a black-box attack in which adversarial instructions are embedded into natural images to override model behavior. Our end-to-end IPI pipeline incorporates segmentation-based region selection, adaptive font scaling, and background-aware rendering to conceal prompts from human perception while preserving model interpretability. Using the COCO dataset and GPT-4-turbo, we evaluate 12 adversarial prompt strategies and multiple embedding configurations. The results show that IPI can reliably manipulate the output of the model, with the most effective configuration achieving up to 64\% attack success under stealth constraints. These findings highlight IPI as a practical threat in black-box settings and underscore the need for defenses against multimodal prompt injection.

Related papers

• DINO-VPT: Hierarchical Visual Prompt Tuning for Joint Physical-Digital Face Anti-Spoofing [2.099922236065961]
  We propose DINO-VPT, a lightweight, vision-only framework leveraging hierarchical visual prompt tuning.<n>Our method effectively disentangles diverse spoofing artifacts without requiring multimodal fusion.<n>Our results indicate that a properly structured vision-only architecture can achieve state-of-the-art performance in unified FAS without the need for multimodal supervision.
  arXiv  Detail & Related papers  (2026-07-23T03:51:22Z)
• VLMShield: Efficient and Robust Defense of Vision-Language Models against Malicious Prompts [63.54526809319117]
  Vision-Language Models (VLMs) face significant safety vulnerabilities from malicious prompt attacks due to weakened alignment during visual integration.<n>We propose the Multimodal Aggregated Feature Extraction framework that enables CLIP to handle long text and fuse multimodal information into unified representations.<n>We develop VLMShield, a lightweight safety detector that efficiently identifies multimodal malicious attacks as a plug-and-play solution.
  arXiv  Detail & Related papers  (2026-04-07T22:18:27Z)
• Adversarial Prompt Injection Attack on Multimodal Large Language Models [21.995939723909796]
  multimodal large language models (MLLMs) are increasingly deployed in real-world applications.<n>Existing prompt injection methods rely on textual prompts or perceptible visual prompts that are observable by human users.<n>In this work, we study imperceptible visual prompt injection against powerful closed-source MLLMs.
  arXiv  Detail & Related papers  (2026-03-31T08:22:07Z)
• Multi-Paradigm Collaborative Adversarial Attack Against Multi-Modal Large Language Models [67.45032003041399]
  We propose a novel Multi-Paradigm Collaborative Attack (MPCAttack) framework to boost the transferability of adversarial examples against MLLMs.<n>MPCO adaptively balances the importance of different paradigm representations and guides the global optimisation.<n>Our solution consistently outperforms state-of-the-art methods in both targeted and untargeted attacks on open-source and closed-source MLLMs.
  arXiv  Detail & Related papers  (2026-03-05T06:01:26Z)
• Test-Time Computing for Referring Multimodal Large Language Models [143.49848714354698]
  We propose ControlMLLM++, a novel test-time adaptation framework.<n>It injects learnable visual prompts into frozen multimodal large language models.
  arXiv  Detail & Related papers  (2026-02-23T04:42:10Z)
• LAMP: Learning Universal Adversarial Perturbations for Multi-Image Tasks via Pre-trained Models [6.127898072805579]
  This paper introduces LAMP, a black-box method for learning Universal Adversarial Perturbations (UAPs) targeting multi-image MLLMs.<n>LAMP applies an attention-based constraint that prevents the model from effectively aggregating information across images.<n>LAMP also introduces a novel cross-image contagious constraint that forces perturbed tokens to influence clean tokens, spreading adversarial effects without requiring all inputs to be modified.
  arXiv  Detail & Related papers  (2026-01-29T03:36:17Z)
• AgentTypo: Adaptive Typographic Prompt Injection Attacks against Black-box Multimodal Agents [22.88469633141419]
  We introduce AgentTypo, a framework that mounts adaptive typographic prompt injection by embedding optimized text into webpage images.<n>Our automatic typographic prompt injection (ATPI) algorithm maximizes prompt reconstruction by substituting captioners while minimizing human detectability via a stealth loss.<n>We also develop AgentTypo-pro, a multi-LLM system that iteratively refines injection prompts using evaluation feedback and retrieves successful past examples for continual learning.
  arXiv  Detail & Related papers  (2025-10-05T15:46:56Z)
• MADPromptS: Unlocking Zero-Shot Morphing Attack Detection with Multiple Prompt Aggregation [8.045296450065019]
  Face Morphing Attack Detection (MAD) is a critical challenge in face recognition security.<n>This work explores a pure zero-shot approach to MAD by leveraging CLIP without any additional training or fine-tuning.<n>By aggregating the embeddings of diverse prompts, we better align the model's internal representations with the MAD task.
  arXiv  Detail & Related papers  (2025-08-12T13:47:27Z)
• Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Models [20.99874786089634]
  Previous jailbreak attacks often inject malicious instructions from text into less aligned modalities, such as vision.<n>We propose a novel implicit jailbreak framework termed IJA that stealthily embeds malicious instructions into images via at least significant bit steganography.<n>On commercial models like GPT-4o and Gemini-1.5 Pro, our method achieves attack success rates of over 90% using an average of only 3 queries.
  arXiv  Detail & Related papers  (2025-05-22T09:34:47Z)
• Multimodal Unlearnable Examples: Protecting Data against Multimodal Contrastive Learning [53.766434746801366]
  Multimodal contrastive learning (MCL) has shown remarkable advances in zero-shot classification by learning from millions of image-caption pairs crawled from the Internet. Hackers may unauthorizedly exploit image-text data for model training, potentially including personal and privacy-sensitive information. Recent works propose generating unlearnable examples by adding imperceptible perturbations to training images to build shortcuts for protection. We propose Multi-step Error Minimization (MEM), a novel optimization process for generating multimodal unlearnable examples.
  arXiv  Detail & Related papers  (2024-07-23T09:00:52Z)
• White-box Multimodal Jailbreaks Against Large Vision-Language Models [61.97578116584653]
  We propose a more comprehensive strategy that jointly attacks both text and image modalities to exploit a broader spectrum of vulnerability within Large Vision-Language Models. Our attack method begins by optimizing an adversarial image prefix from random noise to generate diverse harmful responses in the absence of text input. An adversarial text suffix is integrated and co-optimized with the adversarial image prefix to maximize the probability of eliciting affirmative responses to various harmful instructions.
  arXiv  Detail & Related papers  (2024-05-28T07:13:30Z)
• VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models [65.23688155159398]
  Autoregressive Visual Language Models (VLMs) showcase impressive few-shot learning capabilities in a multimodal context. Recently, multimodal instruction tuning has been proposed to further enhance instruction-following abilities. Adversaries can implant a backdoor by injecting poisoned samples with triggers embedded in instructions or images. We propose a multimodal instruction backdoor attack, namely VL-Trojan.
  arXiv  Detail & Related papers  (2024-02-21T14:54:30Z)
• Adversarial Prompt Tuning for Vision-Language Models [86.5543597406173]
  Adversarial Prompt Tuning (AdvPT) is a technique to enhance the adversarial robustness of image encoders in Vision-Language Models (VLMs) We demonstrate that AdvPT improves resistance against white-box and black-box adversarial attacks and exhibits a synergistic effect when combined with existing image-processing-based defense techniques.
  arXiv  Detail & Related papers  (2023-11-19T07:47:43Z)
• Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions [126.3136109870403]
  We introduce a generic and lightweight Visual Prompt Generator Complete module (VPG-C) VPG-C infers and completes the missing details essential for comprehending demonstrative instructions. We build DEMON, a comprehensive benchmark for demonstrative instruction understanding.
  arXiv  Detail & Related papers  (2023-08-08T09:32:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.