Cross-Modal Backdoors in Multimodal Large Language Models

• URL: http://arxiv.org/abs/2605.07490v1
• Date: Fri, 08 May 2026 09:29:50 GMT
• Title: Cross-Modal Backdoors in Multimodal Large Language Models
• Abstract summary: A single compromised connector can establish a reusable latent-space backdoor pathway across modalities.<n>The attack achieves up to 99.9% attack success rate (ASR) in same-modality settings.
• Score: 21.893043087583667
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Developers increasingly construct multimodal large language models (MLLMs) by assembling pretrained components,introducing supply-chain attack surfaces.Existing security research primarily focuses on poisoning backbones such as encoders or large language models (LLMs),while the security risks of lightweight connectors remain unexplored.In this work,we propose a novel cross-modal backdoor attack that exploits this overlooked vulnerability.By poisoning only the connector using a single seed sample and several augmented variants from one modality,the adversary can subsequently activate the backdoor using inputs from other modalities.To achieve this,we first poison the connector to associate a compact latent region with a malicious target output.To activate the backdoor from other modalities,we further extract a malicious centroid from the poisoned latent representations and perform input-side optimization to steer inputs toward this latent anchor,without requiring repeated API queries or full-model access.Extensive evaluations on representative connector-based MLLM architectures,including PandaGPT and NExT-GPT,demonstrate both the effectiveness and cross-modal transferability of the proposed attack.The attack achieves up to 99.9% attack success rate (ASR) in same-modality settings,while most cross-modal settings exceed 95.0% ASR under bounded perturbations.Moreover,the attack remains highly stealthy,producing negligible leakage on clean inputs,and maintaining weight-cosine similarity above 0.97 relative to benign connectors.We further show that existing defense strategies fail to effectively mitigate this threat without incurring substantial utility degradation.These findings reveal a fundamental vulnerability in multimodal alignment: a single compromised connector can establish a reusable latent-space backdoor pathway across modalities,highlighting the need for safer modular MLLM design.

Related papers

• Securing Multimodal AI through Internal Information Decomposition [43.51030662124641]
  Multimodal large language models introduce attack surfaces absent in unimodal systems.<n>We propose FlowGuard, a lightweight inference-time framework that detects harmful inputs by monitoring internal multimodal consistency.<n>Our results demonstrate that monitoring cross-modal consistency offers an efficient and effective defense for multimodal reasoning.
  arXiv  Detail & Related papers  (2026-05-03T19:21:06Z)
• ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety [54.4092272526747]
  Multimodal Large Language Models (MLLMs) have achieved remarkable success in cross-modal understanding and generation, yet their deployment is threatened by critical safety vulnerabilities.<n>We propose ProjLens, an interpretability framework designed to demystify MLLMs backdoors.
  arXiv  Detail & Related papers  (2026-04-21T04:52:38Z)
• Robust Multimodal Safety via Conditional Decoding [52.92816441364308]
  Multimodal large-language models (MLLMs) often experience degraded safety alignment when harmful queries exploit cross-modal interactions.<n>We propose a simple conditional decoding strategy, CASA, that utilizes internal representations of MLLMs to predict a binary safety token before response generation.
  arXiv  Detail & Related papers  (2026-03-31T23:19:50Z)
• Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Models [74.1970982768771]
  We show that well-established data-poisoning pipelines can successfully implant backdoors into MDLMs.<n>We introduce a backdoor defense framework for MDLMs named DiSP (Diffusion Self-Purification)
  arXiv  Detail & Related papers  (2026-02-24T15:47:52Z)
• Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignment [18.100656799320777]
  We investigate a vulnerability in Omni-modal Large Language Models (OLLMs)<n>We propose OmniSteer, which modulates intervention intensity adaptively.<n>Experiments show that our method effectively preserves the general capabilities across all modalities.
  arXiv  Detail & Related papers  (2026-02-10T06:04:08Z)
• Beyond Text: Multimodal Jailbreaking of Vision-Language and Audio Models through Perceptually Simple Transformations [0.0]
  Multimodal large language models (MLLMs) have achieved remarkable progress, yet remain critically vulnerable to adversarial attacks.<n>We present a systematic study of multimodal jailbreaks targeting both vision-language and audio-language models.<n>Our evaluation spans 1,900 adversarial prompts across three high-risk safety categories.
  arXiv  Detail & Related papers  (2025-10-23T05:16:33Z)
• Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models [83.80177564873094]
  We propose a unified multimodal universal jailbreak attack framework.<n>We evaluate the undesirable context generation of MLLMs like LLaVA, Yi-VL, MiniGPT4, MiniGPT-v2, and InstructBLIP.<n>This study underscores the urgent need for robust safety measures in MLLMs.
  arXiv  Detail & Related papers  (2025-06-02T04:33:56Z)
• Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspace [15.457992715866995]
  We propose a novel Defense-Aware Merging (DAM) approach that simultaneously mitigates task interference and backdoor vulnerabilities.<n>Compared to existing merging methods, DAM achieves a more favorable balance between performance and security, reducing the attack success rate by 2-10 percentage points.
  arXiv  Detail & Related papers  (2024-10-17T00:13:31Z)
• BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models [57.5404308854535]
  Safety backdoor attacks in large language models (LLMs) enable the stealthy triggering of unsafe behaviors while evading detection during normal interactions. We present BEEAR, a mitigation approach leveraging the insight that backdoor triggers induce relatively uniform drifts in the model's embedding space. Our bi-level optimization method identifies universal embedding perturbations that elicit unwanted behaviors and adjusts the model parameters to reinforce safe behaviors against these perturbations.
  arXiv  Detail & Related papers  (2024-06-24T19:29:47Z)
• Model Supply Chain Poisoning: Backdooring Pre-trained Models via Embedding Indistinguishability [61.549465258257115]
  We propose a novel and severer backdoor attack, TransTroj, which enables the backdoors embedded in PTMs to efficiently transfer in the model supply chain.<n> Experimental results show that our method significantly outperforms SOTA task-agnostic backdoor attacks.
  arXiv  Detail & Related papers  (2024-01-29T04:35:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.