ConceptGuard: Proactive Safety in Text-and-Image-to-Video Generation through Multimodal Risk Detection

• URL: http://arxiv.org/abs/2511.18780v3
• Date: Wed, 26 Nov 2025 06:26:38 GMT
• Title: ConceptGuard: Proactive Safety in Text-and-Image-to-Video Generation through Multimodal Risk Detection
• Abstract summary: ConceptGuard is a framework for proactively detecting and mitigating unsafe semantics in multimodal video generation.<n>A contrastive detection module identifies latent safety risks by projecting fused image-text inputs into a structured concept space.<n>A semantic suppression mechanism steers the generative process away from unsafe concepts by intervening in the prompt's multimodal conditioning.
• Score: 27.47621607462884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent progress in video generative models has enabled the creation of high-quality videos from multimodal prompts that combine text and images. While these systems offer enhanced controllability, they also introduce new safety risks, as harmful content can emerge from individual modalities or their interaction. Existing safety methods are often text-only, require prior knowledge of the risk category, or operate as post-generation auditors, struggling to proactively mitigate such compositional, multimodal risks. To address this challenge, we present ConceptGuard, a unified safeguard framework for proactively detecting and mitigating unsafe semantics in multimodal video generation. ConceptGuard operates in two stages: First, a contrastive detection module identifies latent safety risks by projecting fused image-text inputs into a structured concept space; Second, a semantic suppression mechanism steers the generative process away from unsafe concepts by intervening in the prompt's multimodal conditioning. To support the development and rigorous evaluation of this framework, we introduce two novel benchmarks: ConceptRisk, a large-scale dataset for training on multimodal risks, and T2VSafetyBench-TI2V, the first benchmark adapted from T2VSafetyBench for the Text-and-Image-to-Video (TI2V) safety setting. Comprehensive experiments on both benchmarks show that ConceptGuard consistently outperforms existing baselines, achieving state-of-the-art results in both risk detection and safe video generation. Our code is available at https://github.com/Ruize-Ma/ConceptGuard.

Related papers

• The Price of Consistency: Exploiting Visual Anchors for Multimodal Jailbreaking in Video Generation [83.99528259002504]
  We reveal the visual anchoring effect: by enforcing consistency, the mechanism prevents the generated content from drifting away from the original harmful intent.<n>We propose Decoupling Intent via Visual Anchors (DIVA), a training-free multimodal jailbreak framework for video generation that exploits this vulnerability.<n>To facilitate future research, we additionally contribute TI2SafetyBench, the first safety benchmark for multi-conditional video generation.
  arXiv  Detail & Related papers  (2026-09-07T08:34:00Z)
• Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation [48.255342872647766]
  Multi2AV-Safety is the first safety benchmark to cover all 11 non-singleton T/I/A/V conditioning configurations for audio-video generation.<n> evaluation reveals systematic weaknesses in representative multimodal safety guards across attack mechanisms and harm-evidence structures.<n>Results identify emphcompositional risk perception as a central capability gap in safeguarding multimodal-conditioned audio-video generation.
  arXiv  Detail & Related papers  (2026-08-27T02:12:25Z)
• ReFrame: Evidence-Guided Test-Time Safety Alignment in Multimodal Large Language Models [19.730586833276032]
  Multimodal safety alignment methods must address cross-modal jailbreaks, safety-awareness failures, and over-sensitive refusals.<n>Existing methods often rely on retraining or internal-state inspection, limiting their applicability to deployed closed-source MLLMs.<n>We propose ReFrame, a training-free multimodal input reframing framework where two agents share a lightweight locally deployed MLLM.
  arXiv  Detail & Related papers  (2026-08-21T13:47:21Z)
• SafeRedir: Prompt Embedding Redirection for Robust Unlearning in Image Generation Models [67.84174763413178]
  We introduce SafeRedir, a lightweight inference-time framework for robust unlearning via prompt embedding redirection.<n>We show that SafeRedir achieves effective unlearning capability, high semantic and perceptual preservation, robust image quality, and enhanced resistance to adversarial attacks.
  arXiv  Detail & Related papers  (2026-01-13T15:01:38Z)
• CGCE: Classifier-Guided Concept Erasure in Generative Models [53.7410000675294]
  Concept erasure has been developed to remove undesirable concepts from pre-trained models.<n>Existing methods remain vulnerable to adversarial attacks that can regenerate the erased content.<n>We introduce an efficient plug-and-play framework that provides robust concept erasure for diverse generative models.
  arXiv  Detail & Related papers  (2025-11-08T05:38:18Z)
• SafeGuider: Robust and Practical Content Safety Control for Text-to-Image Models [74.11062256255387]
  Text-to-image models are highly vulnerable to adversarial prompts, which can bypass safety measures and produce harmful content.<n>We introduce SafeGuider, a two-step framework designed for robust safety control without compromising generation quality.<n>SafeGuider demonstrates exceptional effectiveness in minimizing attack success rates, achieving a maximum rate of only 5.48% across various attack scenarios.
  arXiv  Detail & Related papers  (2025-10-05T10:24:48Z)
• PromptSafe: Gated Prompt Tuning for Safe Text-to-Image Generation [30.2092299298228]
  Text-to-image (T2I) models are vulnerable to producing not-safe-for-work (NSFW) content, such as violent or explicit imagery.<n>We propose PromptSafe, a gated prompt tuning framework that combines a lightweight, text-only supervised soft embedding with an inference-time gated control network.<n>We show that PromptSafe achieves a SOTA unsafe generation rate (2.36%) while preserving high benign fidelity.
  arXiv  Detail & Related papers  (2025-08-02T09:09:40Z)
• HoliSafe: Holistic Safety Benchmarking and Modeling for Vision-Language Model [58.12612140992874]
  We introduce a holistic safety dataset and benchmark, textbfHoliSafe, that spans all five safe/unsafe image-text combinations.<n>We also propose a novel modular framework for enhancing VLM safety with a visual guard module (VGM) designed to assess the harmfulness of input images.<n> Experiments show that Safe-VLM with VGM, trained on our HoliSafe, achieves state-of-the-art safety performance across multiple VLM benchmarks.
  arXiv  Detail & Related papers  (2025-06-05T07:26:34Z)
• T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Models [88.63040835652902]
  Text to video models are vulnerable to jailbreak attacks, where specially crafted prompts bypass safety mechanisms and lead to the generation of harmful or unsafe content.<n>We propose T2VShield, a comprehensive and model agnostic defense framework designed to protect text to video models from jailbreak threats.<n>Our method systematically analyzes the input, model, and output stages to identify the limitations of existing defenses.
  arXiv  Detail & Related papers  (2025-04-22T01:18:42Z)
• T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models [39.15695612766001]
  We introduce T2VSafetyBench, a new benchmark for safety-critical assessments of text-to-video models. We define 12 critical aspects of video generation safety and construct a malicious prompt dataset. No single model excels in all aspects, with different models showing various strengths. There is a trade-off between the usability and safety of text-to-video generative models.
  arXiv  Detail & Related papers  (2024-07-08T14:04:58Z)
• Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models? [52.238883592674696]
  Ring-A-Bell is a model-agnostic red-teaming tool for T2I diffusion models. It identifies problematic prompts for diffusion models with the corresponding generation of inappropriate content. Our results show that Ring-A-Bell, by manipulating safe prompting benchmarks, can transform prompts that were originally regarded as safe to evade existing safety mechanisms.
  arXiv  Detail & Related papers  (2023-10-16T02:11:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.