Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignment

• URL: http://arxiv.org/abs/2602.10161v1
• Date: Tue, 10 Feb 2026 06:04:08 GMT
• Title: Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignment
• Abstract summary: We investigate a vulnerability in Omni-modal Large Language Models (OLLMs)<n>We propose OmniSteer, which modulates intervention intensity adaptively.<n>Experiments show that our method effectively preserves the general capabilities across all modalities.
• Score: 18.100656799320777
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Omni-modal Large Language Models (OLLMs) greatly expand LLMs' multimodal capabilities but also introduce cross-modal safety risks. However, a systematic understanding of vulnerabilities in omni-modal interactions remains lacking. To bridge this gap, we establish a modality-semantics decoupling principle and construct the AdvBench-Omni dataset, which reveals a significant vulnerability in OLLMs. Mechanistic analysis uncovers a Mid-layer Dissolution phenomenon driven by refusal vector magnitude shrinkage, alongside the existence of a modal-invariant pure refusal direction. Inspired by these insights, we extract a golden refusal vector using Singular Value Decomposition and propose OmniSteer, which utilizes lightweight adapters to modulate intervention intensity adaptively. Extensive experiments show that our method not only increases the Refusal Success Rate against harmful inputs from 69.9% to 91.2%, but also effectively preserves the general capabilities across all modalities. Our code is available at: https://github.com/zhrli324/omni-safety-research.

Related papers

• SafeNexus: Discovering and Steering Modality-Universal Safety Neurons in MLLMs [69.51621286001262]
  We introduce SafeNexus, a cross-modal safety alignment framework that adopts a dedicated neuron-level intervention strategy.<n>We identify modality-bound safety neurons (BS-Neurons), and validate their role in regulating safety behavior within each modality.<n>We propose two safety alignment strategies: activation-level safety amplifier and safety neuron calibrator.
  arXiv  Detail & Related papers  (2026-07-31T02:45:56Z)
• Harnessing Textual Refusal Directions for Multimodal Safety [70.16764304975675]
  We introduce Modality-Agnostic Refusal Steering (MARS), a light-weight training-free approach that injects multimodal safety without the need for multimodal safety data.<n>MARS corrects modality misalignment via activation re-centering, adaptively scales steering strength within a geometrically defined trust region, and selects the optimal intervention layer.
  arXiv  Detail & Related papers  (2026-06-30T15:57:50Z)
• Furina: Fragmented Uncertainty-Driven Refusal Instability Attack [9.678677899786718]
  Safety alignment in large language models (LLMs) and multimodal large language models (MLLMs) is commonly assumed to operate as a near-binary threshold mechanism.<n>We challenge this assumption by revealing that safety behavior is governed by an instability region where small perturbations induce refusal decisions rather than deterministic outcomes.<n>We develop a multi-metric diagnostic framework combining external and internal signals to characterize this instability.<n>We introduce Furina, a jailbreak attack that deliberately induces this signature through fragmented, scene-anchored prompts without model-specific optimization.
  arXiv  Detail & Related papers  (2026-05-24T11:35:12Z)
• Safety Geometry Collapse in Multimodal LLMs and Adaptive Drift Correction [43.59114552026716]
  Multimodal large language models (MLLMs) often fail to transfer safety capabilities learned in the text modality to semantically equivalent non-text inputs.<n>We study this gap by analyzing a text-aligned refusal direction and a modality-induced drift direction.<n>We propose ReGap, a training-free inference-time method that adaptively corrects modality drift using self-rectification.
  arXiv  Detail & Related papers  (2026-05-18T09:16:55Z)
• Cross-Modal Backdoors in Multimodal Large Language Models [21.893043087583667]
  A single compromised connector can establish a reusable latent-space backdoor pathway across modalities.<n>The attack achieves up to 99.9% attack success rate (ASR) in same-modality settings.
  arXiv  Detail & Related papers  (2026-05-08T09:29:50Z)
• Beyond Surface Artifacts: Capturing Shared Latent Forgery Knowledge Across Modalities [52.189426539304065]
  This paper introduces a paradigm shift that redefines multimodal forensics from conventional "feature fusion" to "modality generalization"<n>We propose the first modality-agnostic forgery (MAF) detection framework. By explicitly decoupling modality-specific styles, MAF precisely extracts the essential, cross-modal latent forgery knowledge.
  arXiv  Detail & Related papers  (2026-04-09T03:35:21Z)
• When Safe Models Merge into Danger: Exploiting Latent Vulnerabilities in LLM Fusion [15.004295056225002]
  Model merging has emerged as a powerful technique for combining specialized capabilities from multiple fine-tuned LLMs without additional training costs.<n>We present TrojanMerge, a framework that embeds latent malicious components into source models that remain individually benign but produce severely misaligned models when merged.
  arXiv  Detail & Related papers  (2026-04-01T08:32:46Z)
• Robust Multimodal Safety via Conditional Decoding [52.92816441364308]
  Multimodal large-language models (MLLMs) often experience degraded safety alignment when harmful queries exploit cross-modal interactions.<n>We propose a simple conditional decoding strategy, CASA, that utilizes internal representations of MLLMs to predict a binary safety token before response generation.
  arXiv  Detail & Related papers  (2026-03-31T23:19:50Z)
• Attributing and Exploiting Safety Vectors through Global Optimization in Large Language Models [50.91504059485288]
  We propose a framework that identifies safety-critical attention heads through global optimization over all heads simultaneously.<n>We develop a novel inference-time white-box jailbreak method that exploits the identified safety vectors through activation repatching.
  arXiv  Detail & Related papers  (2026-01-22T09:32:43Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• When Safe Unimodal Inputs Collide: Optimizing Reasoning Chains for Cross-Modal Safety in Multimodal Large Language Models [50.66979825532277]
  We introduce Safe-Semantics-but-Unsafe-Interpretation (SSUI), the first dataset featuring interpretable reasoning paths tailored for a cross-modal challenge.<n>A novel training framework, Safety-aware Reasoning Path Optimization (SRPO), is also designed based on the SSUI dataset.<n> Experimental results show that our SRPO-trained models achieve state-of-the-art results on key safety benchmarks.
  arXiv  Detail & Related papers  (2025-09-15T15:40:58Z)
• SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attacks [29.963044242980345]
  Jailbreak attacks pose a serious threat to the safety of Large Language Models.<n>We propose SafeLLM, a novel unlearning-based defense framework.<n>We show that SafeLLM substantially reduces attack success rates while maintaining high general-purpose performance.
  arXiv  Detail & Related papers  (2025-08-21T02:39:14Z)
• Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models [83.80177564873094]
  We propose a unified multimodal universal jailbreak attack framework.<n>We evaluate the undesirable context generation of MLLMs like LLaVA, Yi-VL, MiniGPT4, MiniGPT-v2, and InstructBLIP.<n>This study underscores the urgent need for robust safety measures in MLLMs.
  arXiv  Detail & Related papers  (2025-06-02T04:33:56Z)
• Safety Alignment Can Be Not Superficial With Explicit Safety Signals [8.297367440457508]
  Recent studies on the safety alignment of large language models (LLMs) have revealed that existing approaches often operate superficially.<n>This paper identifies a fundamental cause of this superficiality: existing alignment approaches presume that models can implicitly learn a safety-related reasoning task during the alignment process.<n>By explicitly introducing a safety-related binary classification task and integrating its signals with our attention and decoding strategies, we eliminate this ambiguity.
  arXiv  Detail & Related papers  (2025-05-19T20:40:46Z)
• DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Models [37.104276926258095]
  Multimodal Large Language Models (MLLMs) pose unique safety challenges due to their integration of visual and textual data.<n>We introduce textbfDREAM (textittextbfDisentangling textbfRisks to textbfEnhance Safety textbfAlignment in textbfMLLMs), a novel approach that enhances safety alignment in MLLMs through supervised fine-tuning and iterative Reinforcement Learning from AI Feedback.
  arXiv  Detail & Related papers  (2025-04-25T03:54:24Z)
• Representation Bending for Large Language Model Safety [27.842146980762934]
  Large Language Models (LLMs) have emerged as powerful tools, but their inherent safety risks pose significant challenges.<n>This paper introduces RepBend, a novel approach that fundamentally disrupts the representations underlying harmful behaviors in LLMs.<n>RepBend achieves state-of-the-art performance, outperforming prior methods such as Circuit Breaker, RMU, and NPO, with up to 95% reduction in attack success rates.
  arXiv  Detail & Related papers  (2025-04-02T09:47:01Z)
• Transferable Adversarial Attacks on SAM and Its Downstream Models [87.23908485521439]
  This paper explores the feasibility of adversarial attacking various downstream models fine-tuned from the segment anything model (SAM)<n>To enhance the effectiveness of the adversarial attack towards models fine-tuned on unknown datasets, we propose a universal meta-initialization (UMI) algorithm.
  arXiv  Detail & Related papers  (2024-10-26T15:04:04Z)
• RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content [62.685566387625975]
  Current mitigation strategies, while effective, are not resilient under adversarial attacks. This paper introduces Resilient Guardrails for Large Language Models (RigorLLM), a novel framework designed to efficiently moderate harmful and unsafe inputs.
  arXiv  Detail & Related papers  (2024-03-19T07:25:02Z)
• Suppress and Rebalance: Towards Generalized Multi-Modal Face Anti-Spoofing [26.901402236963374]
  Face Anti-Spoofing (FAS) is crucial for securing face recognition systems against presentation attacks. Many multi-modal FAS approaches have emerged, but they face challenges in generalizing to unseen attacks and deployment conditions.
  arXiv  Detail & Related papers  (2024-02-29T16:06:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.