Pop-Up Distractions Reveal Bag-of-Events Behavior in Video Large Language Models

• URL: http://arxiv.org/abs/2605.27101v1
• Date: Tue, 26 May 2026 14:41:05 GMT
• Title: Pop-Up Distractions Reveal Bag-of-Events Behavior in Video Large Language Models
• Abstract summary: We evaluate whether Video Large Language Models (VideoLLMs) can robustly link subjects and events in the presence of unrelated video segments.<n>We show that VideoLLMs frequently hallucinate interactions between entities from different segments, incorrectly attributing actions from injected advertisements to subjects in the main video.<n>Our findings suggest that VideoLLMs lack reliable mechanisms for temporal grounding and motivate the development of models with more robust subject-event association.
• Score: 13.934639779174107
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A key capability for video understanding is reliably linking subjects to events across time, yet whether Video Large Language Models (VideoLLMs) actually achieve this remains unclear. In this work, we introduce DistractionBench to evaluate whether VideoLLMs can robustly link subjects and events in the presence of unrelated video segments. Through controlled interventions, such as inserting short advertisement clips into longer videos, we show that VideoLLMs frequently hallucinate interactions between entities from different segments, incorrectly attributing actions from injected advertisements to subjects in the main video. We characterize this systematic hallucination as bag-of-events (BoE) behavior, where models process videos as collections of events rather than temporally structured sequences. Evaluating 11 popular VideoLLMs, we find that all models exhibit substantial BoE behavior. Our findings suggest that VideoLLMs lack reliable mechanisms for temporal grounding and motivate the development of models with more robust subject-event association.

Related papers

• ActionParty: Multi-Subject Action Binding in Generative Video Games [117.52562594944679]
  ActionParty is a controllable multi-subject world model for generative video games.<n>We demonstrate the first video world model capable of controlling up to seven players simultaneously across 46 diverse environments.
  arXiv  Detail & Related papers  (2026-04-02T17:59:58Z)
• While recognizing actions, LMMs struggle to detect core interaction events [18.828641379675243]
  We introduce a first of its kind, large-scale dataset with more than 20K annotated interactions on videos from the Something-Something-V2 dataset.<n>250 AMTurk human annotators labeled core interaction events, particularly when and where objects and agents become attached.<n>We show that although the models can reliably name the target objects, identify the action and provide coherent reasoning, they consistently fail to identify the frame where the interaction begins or ends.
  arXiv  Detail & Related papers  (2025-11-25T10:38:41Z)
• Episodic Memory Representation for Long-form Video Understanding [52.33907540905242]
  Large Video Language Models excel at general video understanding but struggle with long-form context window limits.<n>We introduce Video-EM, a training free framework inspired by the principles of human memory.<n>Video-EM achieves performance gains of 4-9 percent over respective baselines while utilizing fewer frames.
  arXiv  Detail & Related papers  (2025-08-13T04:33:07Z)
• CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videos [59.391265901911005]
  We propose CoT-RVS, a novel framework employing the zero-shot Chain-of-Thought (CoT) capability of MLLM to address complex challenges by temporal-semantic reasoning.<n>CoT-RVS analyzes the visible objects within a given frame that possibly match the language query (semantic), and chooses a corresponding for each object that can be observed effortlessly among all frames (temporal)<n>Our framework's training-free feature further allows its extension to process online video streams, where the CoT is used at test time to update the object of interest when a better target starts to emerge
  arXiv  Detail & Related papers  (2025-05-24T07:01:31Z)
• Divot: Diffusion Powers Video Tokenizer for Comprehension and Generation [54.21476271127356]
  Divot is a Diffusion-Powered Video Tokenizer.<n>We present Divot-unaVic through video-to-text autoregression and text-to-video generation.
  arXiv  Detail & Related papers  (2024-12-05T18:53:04Z)
• EventHallusion: Diagnosing Event Hallucinations in Video LLMs [42.66453293963568]
  Multimodal Large Language Models (MLLMs) have made significant progress in the video comprehension field.<n>We propose EventHallusion, a novel benchmark that focuses on assessing the VideoLLMs' hallucination toward event.<n>We also propose a simple yet effective method, called Temporal Contrastive Decoding (TCD), to tackle the hallucination problems of VideoLLMs.
  arXiv  Detail & Related papers  (2024-09-25T03:49:46Z)
• Enhancing Long Video Understanding via Hierarchical Event-Based Memory [9.800516656566774]
  We propose a Hierarchical Event-based Memory-enhanced LLM (HEM-LLM) for better understanding of long videos. Firstly, we design a novel adaptive sequence segmentation scheme to divide multiple events within long videos. Secondly, while modeling current event, we compress and inject the information of the previous event to enhance the long-term inter-event dependencies in videos.
  arXiv  Detail & Related papers  (2024-09-10T07:53:10Z)
• ViLLa: Video Reasoning Segmentation with Large Language Model [48.75470418596875]
  We present ViLLa: Video reasoning segmentation with Large Language Model.<n>Our ViLLa manages to tackle these challenges through multiple core innovations.<n>To enable efficient processing of long videos, ViLLa incorporates (3) a key segment sampler that adaptively partitions long videos into shorter but semantically dense segments for less redundancy.
  arXiv  Detail & Related papers  (2024-07-18T17:59:17Z)
• Training-Free Robust Interactive Video Object Segmentation [82.05906654403684]
  We propose a training-free prompt tracking framework for interactive video object segmentation (I-PT) We jointly adopt sparse points and boxes tracking, filtering out unstable points and capturing object-wise information. Our framework has demonstrated robust zero-shot video segmentation results on popular VOS datasets.
  arXiv  Detail & Related papers  (2024-06-08T14:25:57Z)
• Glance and Focus: Memory Prompting for Multi-Event Video Question Answering [36.00733800536469]
  VideoQA has emerged as a vital tool to evaluate agents' ability to understand human daily behaviors. Humans can easily tackle it by using a series of episode memories as anchors to quickly locate question-related key moments for reasoning. We propose the Glance-Focus model to mimic this effective reasoning strategy.
  arXiv  Detail & Related papers  (2024-01-03T03:51:16Z)
• SPOT! Revisiting Video-Language Models for Event Understanding [31.49859545456809]
  We introduce SPOT Prober, to benchmark existing video-language models's capacities of distinguishing event-level discrepancies. We evaluate the existing video-language models with these positive and negative captions and find they fail to distinguish most of the manipulated events. Based on our findings, we propose to plug in these manipulated event captions as hard negative samples and find them effective in enhancing models for event understanding.
  arXiv  Detail & Related papers  (2023-11-21T18:43:07Z)
• Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions [81.88294320397826]
  A system does not know what human-object interactions are present in a video as or the actual location of the human and object. We introduce a dataset comprising over 6.5k videos with human-object interaction that have been curated from sentence captions. We demonstrate improved performance over weakly supervised baselines adapted to our annotations on our video dataset.
  arXiv  Detail & Related papers  (2021-10-07T15:30:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.