Failures to Surface Harmful Contents in Video Large Language Models

• URL: http://arxiv.org/abs/2508.10974v1
• Date: Thu, 14 Aug 2025 17:49:36 GMT
• Title: Failures to Surface Harmful Contents in Video Large Language Models
• Abstract summary: Video Large Language Models (VideoLLMs) are increasingly deployed on numerous critical applications.<n>We show that if harmful content is embedded in a video, VideoLLMs rarely mention the harmful content in the output.<n>A root-cause analysis reveals three compounding design flaws.
• Score: 12.841793719841524
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video Large Language Models (VideoLLMs) are increasingly deployed on numerous critical applications, where users rely on auto-generated summaries while casually skimming the video stream. We show that this interaction hides a critical safety gap: if harmful content is embedded in a video, either as full-frame inserts or as small corner patches, state-of-the-art VideoLLMs rarely mention the harmful content in the output, despite its clear visibility to human viewers. A root-cause analysis reveals three compounding design flaws: (1) insufficient temporal coverage resulting from the sparse, uniformly spaced frame sampling used by most leading VideoLLMs, (2) spatial information loss introduced by aggressive token downsampling within sampled frames, and (3) encoder-decoder disconnection, whereby visual cues are only weakly utilized during text generation. Leveraging these insights, we craft three zero-query black-box attacks, aligning with these flaws in the processing pipeline. Our large-scale evaluation across five leading VideoLLMs shows that the harmfulness omission rate exceeds 90% in most cases. Even when harmful content is clearly present in all frames, these models consistently fail to identify it. These results underscore a fundamental vulnerability in current VideoLLMs' designs and highlight the urgent need for sampling strategies, token compression, and decoding mechanisms that guarantee semantic coverage rather than speed alone.

Related papers

• Do Input-Level Defenses Transfer to Observation-Level Attacks on VideoLLMs? [20.483915081902087]
  Video Large Language Models (VideoLLMs) are increasingly deployed in safety-critical applications such as content moderation and video analytics.<n>Recent observation-level attacks exploit this pipeline to prevent the model from perceiving harmful content.<n>We introduce DefTEval, a controlled evaluation framework that assesses whether input-level adversarial defenses can mitigate observation-level attacks.
  arXiv  Detail & Related papers  (2026-09-08T06:59:19Z)
• Distributed Implicit Harm: A Compositional Safety Blind Spot in MLLM-Based Video Moderation [58.10046058931267]
  Distributed Implicit Harm (DIH) is where harm arises from relations among components distributed along a decomposition axis of the video.<n>We study two representative cases: temporally distributed harm across visual segments (DIH-T) and cross-modal harm between audio and visual streams (DIH-M)<n>We develop a multi-agent framework that composes individually benign components into harmful scenarios and generates diverse DIH videos with explicit reasoning annotations.
  arXiv  Detail & Related papers  (2026-08-31T18:17:49Z)
• Benign Fine-Tuning Breaks Safety Alignment in Audio LLMs [19.239722054750832]
  We present the first systematic study of benign fine-tuning safety in Audio LLMs.<n>We show that benign fine-tuning elevates Jailbreak Success Rate (JSR) from single digits to as high as 87.12%.<n>We propose two defenses: filtering training data to maximize distance from harmful embeddings, and a textual system prompt at inference.
  arXiv  Detail & Related papers  (2026-04-17T19:28:07Z)
• Gloria: Consistent Character Video Generation via Content Anchors [63.7455328319917]
  We propose representing character visual attributes through a compact set of anchor frames.<n>This design provides stable references for consistency, while reference-based video generation faces challenges of copy-pasting and multi-reference conflicts.<n> Experiments show our method generates high-quality character videos exceeding 10 minutes, and achieves expressive identity and appearance consistency across views, surpassing existing methods.
  arXiv  Detail & Related papers  (2026-03-31T16:06:16Z)
• Shot-Aware Frame Sampling for Video Understanding [10.771430339569795]
  InfoShot is a task-agnostic, shot-aware frame sampler for long-video understanding.<n>Design is guided by an information-theoretic objective that encourages the sampled set to retain high information about both shot structure and sparse within-shot deviations.<n>Experiments show that InfoShot improves anomaly hit rate and downstream Video-QA accuracy under frame number constraints.
  arXiv  Detail & Related papers  (2026-03-18T05:36:57Z)
• Two Frames Matter: A Temporal Attack for Text-to-Video Model Jailbreaking [3.1471354211996956]
  Recent text-to-video (T2V) models can synthesize complex videos from lightweight natural language prompts.<n>Prior jailbreak attacks typically rewrite unsafe prompts into paraphrases that evade content filters.<n>In this paper, we identify a temporal trajectory infilling vulnerability of T2V systems under fragmented prompts.
  arXiv  Detail & Related papers  (2026-03-07T04:21:54Z)
• KFFocus: Highlighting Keyframes for Enhanced Video Understanding [33.69757683688046]
  We propose KFFocus, a method designed to efficiently compress video tokens and emphasize the informative context present within video frames.<n>By assigning varying condensation ratios to frames based on their contextual relevance, KFFocus efficiently reduces token redundancy while preserving informative content details.<n>We also introduce a multimodal modeling module that encodes both the temporal relationships between video frames and the spatial structure within each frame.
  arXiv  Detail & Related papers  (2025-08-12T14:57:03Z)
• Threading Keyframe with Narratives: MLLMs as Strong Long Video Comprehenders [62.58375366359421]
  Multimodal Large Language Models (MLLMs) for long video understanding remains a challenging problem.<n>Traditional uniform sampling leads to selection of irrelevant content.<n>Post-training MLLMs on thousands of frames imposes a substantial computational burden.<n>We propose threadings with narratives (Nar-KFC) to facilitate effective and efficient long video perception.
  arXiv  Detail & Related papers  (2025-05-30T03:04:28Z)
• T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attacks [67.91652526657599]
  We formalize the T2V jailbreak attack as a discrete optimization problem and propose a joint objective-based optimization framework, called T2V-OptJail.<n>We conduct large-scale experiments on several T2V models, covering both open-source models and real commercial closed-source models.<n>The proposed method improves 11.4% and 10.0% over the existing state-of-the-art method in terms of attack success rate.
  arXiv  Detail & Related papers  (2025-05-10T16:04:52Z)
• T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Models [88.63040835652902]
  Text to video models are vulnerable to jailbreak attacks, where specially crafted prompts bypass safety mechanisms and lead to the generation of harmful or unsafe content.<n>We propose T2VShield, a comprehensive and model agnostic defense framework designed to protect text to video models from jailbreak threats.<n>Our method systematically analyzes the input, model, and output stages to identify the limitations of existing defenses.
  arXiv  Detail & Related papers  (2025-04-22T01:18:42Z)
• M3-CVC: Controllable Video Compression with Multimodal Generative Models [17.49397141459785]
  M3-CVC is a controllable video compression framework incorporating generative models.<n>We show that M3-CVC significantly outperforms the state-the-art VVC standard in ultralow scenarios.
  arXiv  Detail & Related papers  (2024-11-24T11:56:59Z)
• GMMFormer v2: An Uncertainty-aware Framework for Partially Relevant Video Retrieval [60.70901959953688]
  We present GMMFormer v2, an uncertainty-aware framework for PRVR. For clip modeling, we improve a strong baseline GMMFormer with a novel temporal consolidation module. We propose a novel optimal matching loss for fine-grained text-clip alignment.
  arXiv  Detail & Related papers  (2024-05-22T16:55:31Z)
• Self-Adaptive Sampling for Efficient Video Question-Answering on Image--Text Models [41.12711820047315]
  Video understanding models usually randomly sample a set of frames or clips, regardless of internal correlations between their visual contents, nor their relevance to the problem. We propose two frame sampling strategies, namely the most domain frames (MDF) and most implied frames (MIF), to maximally preserve those frames that are most likely vital to the given questions.
  arXiv  Detail & Related papers  (2023-07-09T14:54:30Z)
• Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation [55.36617538438858]
  We propose a novel approach that strengthens the interaction between spatial and temporal perceptions. We curate a large-scale and open-source video dataset called HD-VG-130M.
  arXiv  Detail & Related papers  (2023-05-18T11:06:15Z)
• NSNet: Non-saliency Suppression Sampler for Efficient Video Recognition [89.84188594758588]
  A novel Non-saliency Suppression Network (NSNet) is proposed to suppress the responses of non-salient frames. NSNet achieves the state-of-the-art accuracy-efficiency trade-off and presents a significantly faster (2.44.3x) practical inference speed than state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-07-21T09:41:22Z)
• A Coding Framework and Benchmark towards Low-Bitrate Video Understanding [63.05385140193666]
  We propose a traditional-neural mixed coding framework that takes advantage of both traditional codecs and neural networks (NNs) The framework is optimized by ensuring that a transportation-efficient semantic representation of the video is preserved. We build a low-bitrate video understanding benchmark with three downstream tasks on eight datasets, demonstrating the notable superiority of our approach.
  arXiv  Detail & Related papers  (2022-02-06T16:29:15Z)
• Adversarially Robust Frame Sampling with Bounded Irregularities [11.434633941880143]
  Video analysis tools for automatically extracting meaningful information from videos are widely studied and deployed. Most of them use deep neural networks which are computationally expensive, feeding only a subset of video frames into such algorithms is desired. We present an elegant solution to this sampling problem that is provably robust against adversarial attacks and introduces bounded irregularities as well.
  arXiv  Detail & Related papers  (2020-02-04T06:33:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.