The Low Frequency Trap: Video Language Models Fail at Simple Event Bookkeeping

• URL: http://arxiv.org/abs/2608.06361v1
• Date: Thu, 06 Aug 2026 17:57:06 GMT
• Title: The Low Frequency Trap: Video Language Models Fail at Simple Event Bookkeeping
• Abstract summary: We introduce trace-grounded parametric profiling for event counting in three controlled video tasks.<n>Across 2,190 videos, we vary event count N and frequency F while holding rendering fixed.<n>At an 80% reliability threshold, Gemini 3.6 Flash reliably counts persistent state transitions up to 12 events at 0.5 and 1.0 Hz, yet demonstrates no reliable positive-count region for transient blinking events.
• Score: 70.07961857926377
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world video benchmarks provide broad coverage, but their fixed clips entangle event count, rate, duration, and visual complexity, making failure modes hard to isolate. While existing programmatic benchmarks offer better control, they score only the final answer rather than auditing reported events against executable ground truth. To bridge this gap, we introduce trace-grounded parametric profiling for event counting in three controlled video tasks: bouncing-ball wall contacts, visual blinks, and categorical state transitions. Across 2,190 videos, we vary event count N and frequency F while holding rendering fixed. Each video includes an executable event trace for capability-surface estimation and timestamp-level evaluation. Our results reveal a staged temporal failure. At an 80% reliability threshold, Gemini 3.6 Flash reliably counts persistent state transitions up to 12 events at 0.5 and 1.0 Hz, yet demonstrates no reliable positive-count region for transient blinking events. Thus, event representation dictates whether a model initially accesses evidence -- a limitation that compounds as count and frequency increase. In the high-count, high-frequency regime, only 0.2% of final counts are correct and the model recovers just 18.1% of true events. To test if visual access is the primary bottleneck, we increase sampling rate. Although this boosts Bounce Ball accuracy from 19.6% to 29.3%, the reported sequence agrees with ground truth only 3.7% of the time. Extra frames can therefore inflate final scores without producing faithful event recovery. Different prompting strategies yield similarly limited gains, and real-world video evaluations show the same concentration of success at low event counts. Ultimately, trace-grounded profiling shifts video evaluation from aggregate accuracy metrics to a detailed diagnostic of where temporal reasoning fails.

Related papers

• Strictly Causal Streaming Video Anomaly Detection with a Theoretically-Grounded State-Space Core [2.648500779572419]
  We introduce a causal streaming anomaly detector with no lookahead and no clip buffering.<n>Its temporal core is a diagonal linear state space recurrence with an input and state dependent decay gate.<n>We derive a closed form relationship between the recurrence decay spectrum and both detection delay and the shortest anomaly it can reliably capture.
  arXiv  Detail & Related papers  (2026-08-25T16:52:32Z)
• CaliBench: Are the Stochastic Dynamics of Video World Models Physically Calibrated? [6.39120343569684]
  CaliBench scores outcomes in a discrete space rather than a learned feature space such as in FID.<n>We decompose performance into two axes that a single accuracy metric conflates: scorability, the fraction of generations yielding a scoreable outcome, and calibration, the total variation distance from the reference on that sample.<n>We apply it to nine scenes and six image-to-video models (WAN-2.7, SeeDance-2.0, HappyHorse-1.0, Veo 3.1, Runway Gen-4.5, Cosmos3-Super), 32 generations each.
  arXiv  Detail & Related papers  (2026-08-17T17:14:50Z)
• Conformal Coverage Guarantees for Any Video Temporal Grounder [2.3982445219832678]
  Event boundaries in continuous video are ambiguous: re-annotate the same query-video pair and independent annotators mark moments that overlap by less than half on a large fraction of samples.<n> COVER changes the output object: a model-agnostic wrapper that turns any grounder, a trained localizer or a black-box video-language model.<n>Two score families, a two-sided boundary-widening score for grounders that emit an interval and a super-level-set score for grounders that emit a relevance signal.
  arXiv  Detail & Related papers  (2026-08-07T17:19:27Z)
• From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQA [44.46419663487345]
  High benchmark accuracy does not guarantee genuine use of visual evidence.<n>We study this problem in traffic accident Video Question Answering (VideoQA)<n>We propose an instance-level Shortcut Score that combines text-only confidence with visual necessity signals.
  arXiv  Detail & Related papers  (2026-06-29T12:33:15Z)
• Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events [52.031070006859544]
  Video multimodal large language models (MLLMs) have made rapid progress on general and long-form video understanding, yet their ability to preserve brief answer-critical visual evidence remains underexplored.<n>We introduce Moment-Video, a benchmark for diagnosing the temporal fidelity of video MLLMs through momentary visual event understanding.
  arXiv  Detail & Related papers  (2026-06-01T17:32:20Z)
• Don't Pause! Every prediction matters in a streaming video [55.509551643600794]
  We introduce SPOT-Bench, featuring multi-turn proactive queries that evaluate general streaming perception and assistive capabilities.<n> SPOT-Bench comes with Timeliness-F1, a metric that measures streaming predictions by their temporal precision and balanced coverage across the entire video.<n>Our benchmark reveals: (i) offline models detect events reliably but spam predictions unprompted; (ii) post-training for silence reduces spamming but induces unresponsiveness; (iii) half of the streaming video expects no response, which we term dead-time - compute spent here does not affect response latency.
  arXiv  Detail & Related papers  (2026-04-27T11:07:03Z)
• From Frames to Events: Rethinking Evaluation in Human-Centric Video Anomaly Detection [9.404376027901277]
  Pose-based Video Anomaly Detection (VAD) has gained significant attention for its privacy-preserving nature and robustness to environmental variations.<n>Traditional frame-level evaluations treat video as a collection of isolated frames, fundamentally misaligned with how anomalies manifest and are acted upon in the real world.<n>We introduce two strategies for temporal event localization: a score-refinement pipeline with hierarchical Gaussian smoothing and adaptive binarization, and an end-to-end Dual-Branch Model that directly generates event-level detections.
  arXiv  Detail & Related papers  (2026-04-10T13:52:18Z)
• VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification [73.02304272829785]
  VideoBench is a hierarchical benchmark for long-video answering that rigorously verifies evidence.<n>It comprises 500 manually annotated questions across 13 domains, paired with temporal intervals and spatial bounding boxes as evidence.<n>Experiments show that even GeminiPro-3-Pro correctly answers fewer than 17% of questions under the standard end-to-end QA setting.<n>Results expose a significant gap between surface-level answer correctness and genuine evidence-based reasoning.
  arXiv  Detail & Related papers  (2026-04-02T03:29:43Z)
• E.M.Ground: A Temporal Grounding Vid-LLM with Holistic Event Perception and Matching [87.38371267983263]
  Temporal Video Grounding aims to precisely localize time segments corresponding to query events.<n>E.M.Ground is a novel Vid-LLM for TVG that focuses on holistic and coherent event perception.<n>E.M.Ground consistently outperforms state-of-the-art Vid-LLMs by significant margins.
  arXiv  Detail & Related papers  (2026-02-05T02:16:00Z)
• Perception Test: A Diagnostic Benchmark for Multimodal Video Models [78.64546291816117]
  We propose a novel multimodal video benchmark to evaluate the perception and reasoning skills of pre-trained multimodal models. The Perception Test focuses on skills (Memory, Abstraction, Physics, Semantics) and types of reasoning (descriptive, explanatory, predictive, counterfactual) across video, audio, and text modalities. The benchmark probes pre-trained models for their transfer capabilities, in a zero-shot / few-shot or limited finetuning regime.
  arXiv  Detail & Related papers  (2023-05-23T07:54:37Z)
• Investigating Modality Bias in Audio Visual Video Parsing [31.83076679253096]
  We focus on the audio-visual video parsing (AVVP) problem that involves detecting audio and visual event labels with temporal boundaries. An existing state-of-the-art model for AVVP uses a hybrid attention network (HAN) to generate cross-modal features for both audio and visual modalities. We propose a variant of feature aggregation in HAN that leads to an absolute gain in F-scores of about 2% and 1.6% for visual and audio-visual events at both segment-level and event-level.
  arXiv  Detail & Related papers  (2022-03-31T07:43:01Z)
• Spatio-Temporal Event Segmentation and Localization for Wildlife Extended Videos [6.279863832853343]
  We present a self-supervised prediction framework capable of temporal event segmentation. We rely on predictions of high-level features computed by a standard deep learning backbone. We demonstrate our approach on video from 10 days (254 hours) of continuous wildlife monitoring data.
  arXiv  Detail & Related papers  (2020-05-05T20:11:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.