Leave My Images Alone: Preventing Multi-Modal Large Language Models from Analyzing Images via Visual Prompt Injection

• URL: http://arxiv.org/abs/2604.09024v1
• Date: Fri, 10 Apr 2026 06:37:46 GMT
• Title: Leave My Images Alone: Preventing Multi-Modal Large Language Models from Analyzing Images via Visual Prompt Injection
• Abstract summary: Multi-modal large language models (MLLMs) have emerged as powerful tools for analyzing Internet-scale image data.<n>In particular, open-weight MLLMs may be misused to extract sensitive information from personal images at scale.<n>We propose ImageProtector, a user-side method that proactively protects images before sharing.
• Score: 37.48710514852417
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-modal large language models (MLLMs) have emerged as powerful tools for analyzing Internet-scale image data, offering significant benefits but also raising critical safety and societal concerns. In particular, open-weight MLLMs may be misused to extract sensitive information from personal images at scale, such as identities, locations, or other private details. In this work, we propose ImageProtector, a user-side method that proactively protects images before sharing by embedding a carefully crafted, nearly imperceptible perturbation that acts as a visual prompt injection attack on MLLMs. As a result, when an adversary analyzes a protected image with an MLLM, the MLLM is consistently induced to generate a refusal response such as "I'm sorry, I can't help with that request." We empirically demonstrate the effectiveness of ImageProtector across six MLLMs and four datasets. Additionally, we evaluate three potential countermeasures, Gaussian noise, DiffPure, and adversarial training, and show that while they partially mitigate the impact of ImageProtector, they simultaneously degrade model accuracy and/or efficiency. Our study focuses on the practically important setting of open-weight MLLMs and large-scale automated image analysis, and highlights both the promise and the limitations of perturbation-based privacy protection.

Related papers

• POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking [56.151482557515955]
  Multi-modality Machine Unlearning (MMU) was proposed as a mitigation that can effectively force MLLMs to forget private information.<n>In this paper, we propose a novel strategy, namely Prompt-d adversarial-POPS, aiming to recover the supposedly unlearned multi-modality knowledge from the MLLMs.<n>Our method elicits the victim MLLMs to generate potential private examples via promptsuffix optimization, and then exploits these synthesized outputs to fine-tune the models so they disclose the true private information.
  arXiv  Detail & Related papers  (2026-07-07T16:28:54Z)
• Image Prompt Reconstruction Attacks on Distributed MLLM Inference Frameworks [21.9083862179787]
  We investigate privacy risks to input images caused by intermediate embeddings in distributed MLLM frameworks.<n>We develop two passive black-box image reconstruction attacks, MPAA and IEDA, reflecting realistic threats from normal participants.<n>Results show consistently superior reconstruction performance in various settings.
  arXiv  Detail & Related papers  (2026-06-17T05:51:14Z)
• Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges [55.425628316813174]
  Multi-modal Large Language Models (MLLMs) process both text and images.<n>Compared to text-only models, MLLMs can extract and expose sensitive information embedded in images.<n>Some MLLMs are susceptible to privacy breaches, leaking sensitive data embedded in images or stored in memory.
  arXiv  Detail & Related papers  (2026-06-08T07:19:42Z)
• When Privacy Meets Recovery: The Overlooked Half of Surrogate-Driven Privacy Preservation for MLLM Editing [61.80513991207956]
  This work focuses on the challenge of how to restore surrogate-driven protected data in diverse MLLM scenarios.<n>We first bridge this research gap by contributing the SPPE (Surrogate Privacy Protected Editable) dataset.<n>We introduce a unified approach that reliably reconstructs private content while preserving the fidelity of MLLM-generated edits.
  arXiv  Detail & Related papers  (2025-12-08T04:59:03Z)
• SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attacks [17.77094760401298]
  We study the vulnerability of fine-tuned large language models to membership inference attacks (MIAs)<n>We propose SOFT, a novel defense technique that mitigates privacy leakage by leveraging influential data selection with an adjustable parameter to balance utility preservation and privacy protection.
  arXiv  Detail & Related papers  (2025-06-12T07:23:56Z)
• MLLM-as-a-Judge for Image Safety without Human Labeling [81.24707039432292]
  In the age of AI-generated content (AIGC), many image generation models are capable of producing harmful content.<n>It is crucial to identify such unsafe images based on established safety rules.<n>Existing approaches typically fine-tune MLLMs with human-labeled datasets.
  arXiv  Detail & Related papers  (2024-12-31T00:06:04Z)
• CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration [90.36429361299807]
  multimodal large language models (MLLMs) have demonstrated remarkable success in engaging in conversations involving visual inputs. The integration of visual modality has introduced a unique vulnerability: the MLLM becomes susceptible to malicious visual inputs. We introduce a technique termed CoCA, which amplifies the safety-awareness of the MLLM by calibrating its output distribution.
  arXiv  Detail & Related papers  (2024-09-17T17:14:41Z)
• MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance [36.03512474289962]
  This paper investigates the novel challenge of defending MLLMs against malicious attacks through visual inputs. Images act as a foreign language" that is not considered during safety alignment, making MLLMs more prone to producing harmful responses. We introduce MLLM-Protector, a plug-and-play strategy that solves two subtasks: 1) identifying harmful responses via a lightweight harm detector, and 2) transforming harmful responses into harmless ones via a detoxifier.
  arXiv  Detail & Related papers  (2024-01-05T17:05:42Z)
• MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models [41.708401515627784]
  We observe that Multimodal Large Language Models (MLLMs) can be easily compromised by query-relevant images. We introduce MM-SafetyBench, a framework designed for conducting safety-critical evaluations of MLLMs against such image-based manipulations. Our work underscores the need for a concerted effort to strengthen and enhance the safety measures of open-source MLLMs against potential malicious exploits.
  arXiv  Detail & Related papers  (2023-11-29T12:49:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.