Security Risk of Misalignment between Text and Image in Multi-modal Model

• URL: http://arxiv.org/abs/2510.26105v1
• Date: Thu, 30 Oct 2025 03:31:20 GMT
• Title: Security Risk of Misalignment between Text and Image in Multi-modal Model
• Abstract summary: We propose a novel attack to manipulate generated content by modifying the input image in conjunction with any specified prompt, without altering the prompt itself.<n>PReMA is the first attack that manipulates model outputs by solely creating adversarial images, distinguishing itself from prior methods that primarily generate adversarial prompts to produce NSFW content.
• Score: 10.368397924684063
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the notable advancements and versatility of multi-modal diffusion models, such as text-to-image models, their susceptibility to adversarial inputs remains underexplored. Contrary to expectations, our investigations reveal that the alignment between textual and Image modalities in existing diffusion models is inadequate. This misalignment presents significant risks, especially in the generation of inappropriate or Not-Safe-For-Work (NSFW) content. To this end, we propose a novel attack called Prompt-Restricted Multi-modal Attack (PReMA) to manipulate the generated content by modifying the input image in conjunction with any specified prompt, without altering the prompt itself. PReMA is the first attack that manipulates model outputs by solely creating adversarial images, distinguishing itself from prior methods that primarily generate adversarial prompts to produce NSFW content. Consequently, PReMA poses a novel threat to the integrity of multi-modal diffusion models, particularly in image-editing applications that operate with fixed prompts. Comprehensive evaluations conducted on image inpainting and style transfer tasks across various models confirm the potent efficacy of PReMA.

Related papers

• Failure-Informed Image Self-Augmentation for Multimodal Large Language Model Self-Improvement [50.97806700110183]
  Failure-informed Image Self-Augmentation (textbfFISA) is a framework for MLLM self-improvement that constructs augmented images from the model's own failure cases.<n>Our method generates visually challenging yet answer-preserving image complications, verifies their utility through self-examination, and applies dual fidelity filtering to avoid semantic distortion.
  arXiv  Detail & Related papers  (2026-08-04T14:28:42Z)
• Unveiling the Fragility of Vision-Language Models: Multi-Modal Adversarial Synergy via Texture-Constrained Perturbations and Cross-Modal Optimization [15.851694572297612]
  We introduce Multi-Modal Adrial Synergy, a framework that crafts universal, black-box multi-modal attacks against LVLMs.<n> MMAS simultaneously generates a universal adversarial perturbation for images and a learnable prompt perturbation for text.<n>Our experiments show the strong universal adversarial capabilities of our proposed attack with prevalent LVLMs.
  arXiv  Detail & Related papers  (2026-05-26T03:28:28Z)
• CoDefend: Cross-Modal Collaborative Defense via Diffusion Purification and Prompt Optimization [4.6467356929461925]
  Multimodal Large Language Models (MLLMs) have achieved remarkable success in tasks such as image captioning, visual question answering, and cross-modal reasoning.<n>Their multimodal nature exposes them to adversarial threats, where attackers can perturb either modality or both jointly to induce harmful, misleading, or policy violating outputs.<n>Existing defense strategies, such as adversarial training and input purification, face notable limitations.<n>We propose a supervised diffusion based denoising framework that leverages paired adversarial clean image datasets to fine-tune diffusion models.
  arXiv  Detail & Related papers  (2025-10-13T07:44:54Z)
• Image Can Bring Your Memory Back: A Novel Multi-Modal Guided Attack against Image Generation Model Unlearning [28.15997901023315]
  Recall is a novel adversarial framework designed to compromise the robustness of unlearned IGMs.<n>It consistently outperforms existing baselines in terms of adversarial effectiveness, computational efficiency, and semantic fidelity with the original prompt.<n>These findings reveal critical vulnerabilities in current unlearning mechanisms and underscore the need for more robust solutions.
  arXiv  Detail & Related papers  (2025-07-09T02:59:01Z)
• A Unified Framework for Stealthy Adversarial Generation via Latent Optimization and Transferability Enhancement [72.3054292908678]
  We propose a unified framework that seamlessly incorporates traditional transferability enhancement strategies into diffusion model-based adversarial example generation via image editing.<n>Our method won first place in the "1st Adversarial Attacks on Deepfake Detectors: A Challenge in the Era of AI-Generated Media" competition at ACM MM25.
  arXiv  Detail & Related papers  (2025-06-30T09:59:09Z)
• Proxy-Embedding as an Adversarial Teacher: An Embedding-Guided Bidirectional Attack for Referring Expression Segmentation Models [7.064823891326925]
  Referring Expression (RES) enables precise object segmentation in images based on natural language descriptions.<n>Despite its impressive performance, the robustness of RES models against adversarial examples remains largely unexplored.<n>We present PEAT, an Embedding-Guided Bidirectional Attack for RES models.
  arXiv  Detail & Related papers  (2025-06-19T09:14:04Z)
• MMAR: Towards Lossless Multi-Modal Auto-Regressive Probabilistic Modeling [64.09238330331195]
  We propose a novel Multi-Modal Auto-Regressive (MMAR) probabilistic modeling framework.<n>Unlike discretization line of method, MMAR takes in continuous-valued image tokens to avoid information loss in an efficient way.<n>We also propose a theoretically proven technique that addresses the numerical stability issue and a training strategy that balances the generation and understanding task goals.
  arXiv  Detail & Related papers  (2024-10-14T17:57:18Z)
• EMMA: Your Text-to-Image Diffusion Model Can Secretly Accept Multi-Modal Prompts [48.214475133206385]
  EMMA is a novel image generation model accepting multi-modal prompts built upon the state-of-the-art text-to-image (T2I) diffusion model, ELLA. By freezing all parameters in the original T2I diffusion model and only adjusting some additional layers, we reveal an interesting finding that the pre-trained T2I diffusion model can secretly accept multi-modal prompts.
  arXiv  Detail & Related papers  (2024-06-13T14:26:43Z)
• Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective [42.04728834962863]
  Pretrained vision-language models (VLMs) like CLIP exhibit exceptional generalization across diverse downstream tasks. Recent studies reveal their vulnerability to adversarial attacks, with defenses against text-based and multimodal attacks remaining largely unexplored. This work presents the first comprehensive study on improving the adversarial robustness of VLMs against attacks targeting image, text, and multimodal inputs.
  arXiv  Detail & Related papers  (2024-04-30T06:34:21Z)
• On the Multi-modal Vulnerability of Diffusion Models [56.08923332178462]
  We propose MMP-Attack to manipulate the generation results of diffusion models by appending a specific suffix to the original prompt.<n>Our goal is to induce diffusion models to generate a specific object while simultaneously eliminating the original object.
  arXiv  Detail & Related papers  (2024-02-02T12:39:49Z)
• Iterative Adversarial Attack on Image-guided Story Ending Generation [37.42908817585858]
  Multimodal learning involves developing models that can integrate information from various sources like images and texts. Deep neural networks, which are the backbone of recent IgSEG models, are vulnerable to adversarial samples. We propose an iterative adversarial attack method (Iterative-attack) that fuses image and text modality attacks.
  arXiv  Detail & Related papers  (2023-05-16T06:19:03Z)
• Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models [103.61066310897928]
  Recent text-to-image generative models have demonstrated an unparalleled ability to generate diverse and creative imagery guided by a target text prompt. While revolutionary, current state-of-the-art diffusion models may still fail in generating images that fully convey the semantics in the given text prompt. We analyze the publicly available Stable Diffusion model and assess the existence of catastrophic neglect, where the model fails to generate one or more of the subjects from the input prompt. We introduce the concept of Generative Semantic Nursing (GSN), where we seek to intervene in the generative process on the fly during inference time to improve the faithfulness
  arXiv  Detail & Related papers  (2023-01-31T18:10:38Z)
• Benchmarking Robustness of Multimodal Image-Text Models under Distribution Shift [50.64474103506595]
  We investigate the robustness of 12 popular open-sourced image-text models under common perturbations on five tasks. Character-level perturbations constitute the most severe distribution shift for text, and zoom blur is the most severe shift for image data.
  arXiv  Detail & Related papers  (2022-12-15T18:52:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.