What's the Catch? Evaluating Temporal Consistency in Vision-Language Models

• URL: http://arxiv.org/abs/2608.23474v2
• Date: Tue, 25 Aug 2026 11:47:47 GMT
• Title: What's the Catch? Evaluating Temporal Consistency in Vision-Language Models
• Abstract summary: Vision-language models (VLMs) achieve strong performance on video and image-sequence benchmarks, yet it remains unclear whether they capture temporal structure.<n>We introduce TimeCatch, where temporal anomalies are created by swapping consecutive frames and frame-level anomalies by replacing a frame with Gaussian noise.<n>Our evaluation reveals a substantial gap between frame-level and temporal anomaly detection.
• Score: 0.11121828451787184
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language models (VLMs) achieve strong performance on video and image-sequence benchmarks, yet it remains unclear whether they capture temporal structure. To study this question, we formulate temporal grounding as an anomaly detection problem, providing a simple and controlled evaluation that directly tests sensitivity to temporal consistency. We introduce TimeCatch, where temporal anomalies are created by swapping consecutive frames and frame-level anomalies by replacing a frame with Gaussian noise. Models are evaluated on anomaly detection and localization tasks across four synthetic and real-world datasets, alongside a human study. Our evaluation reveals a substantial gap between frame-level and temporal anomaly detection. While VLMs consistently detect frame-level anomalies and often localize them accurately, they perform near chance on temporal anomaly detection and only modestly above chance on localization. Humans, in contrast, achieve near-ceiling performance on both tasks. Additional analyses across model scales, prompting strategies, sequence lengths, and visual similarity suggest that these failures cannot be explained solely by limitations in perception or model capacity. Together, these findings indicate that current VLMs can identify anomalies within individual frames but struggle to integrate information across frames to reason about temporal consistency. TimeCatch provides a controlled benchmark for evaluating temporal grounding in vision-language models.

Related papers

• Adaptive Multi-Granularity Temporal Modeling for Weakly Supervised Video Anomaly Detection [1.2072875728151518]
  weakly supervised video anomaly detection (WSVAD) has emerged as a critical research frontier.<n>Most existing approaches formulate WSVAD within a Multiple Instance Learning (MIL) framework that relies on rigid, hand-crafted temporal priors to supervise anomaly scoring.<n>We propose an adaptive temporal modeling framework for WSVAD that explicitly accounts for variations in video dynamics across multiple temporal granularities.
  arXiv  Detail & Related papers  (2026-09-04T12:27:43Z)
• POST: Prior-Observation Adversarial Learning of Spatio-Temporal Associations for Multivariate Time Series Anomaly Detection [10.26613829763125]
  We propose a novel framework that unifies ad prior-observation adversarial learning paradigm.<n>In the spatial dimension, the model reconstructs the discrepancy between prior and data-driven observation in a minimax manner during training.<n>Such adversarial optimization not only improves the model sensitivity for time-wise detection, but also enables the model to localize anomalies to specific channels.
  arXiv  Detail & Related papers  (2026-05-18T09:34:14Z)
• Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models [154.39583176906893]
  Large language models (MLs) have advanced understanding, they remain highly prone to hallucinations in scenes.<n>We argue this stems from a failure intemporal monitoring, the ability to persistently track object dynamic identities, states, and relations over time.<n>Existing benchmarks obscure this deficit relying on single final-answer queries that can often be resolved via local visual cues or statistical priors.
  arXiv  Detail & Related papers  (2026-05-09T14:32:36Z)
• Conditional Attribution for Root Cause Analysis in Time-Series Anomaly Detection [13.555771273974697]
  We propose a conditional attribution framework that explains anomalies relative to contextually similar normal system states.<n>Our method retrieves representative normal instances conditioned on the anomalous observation, enabling dependency-preserving and operationally meaningful explanations.<n> Experiments on the SWaT and benchmarks demonstrate that the proposed approach consistently improves root-cause identification accuracy, temporal localization, and robustness across multiple anomaly detection models.
  arXiv  Detail & Related papers  (2026-04-19T21:01:52Z)
• TRACES: Temporal Recall with Contextual Embeddings for Real-Time Video Anomaly Detection [0.0]
  This paper addresses the context-aware zero-shot anomaly detection challenge.<n>Our approach defines a memory-augmented pipeline, correlating temporal signals with visual embeddings.<n>We achieve 90.4% AUC on UCF-Crime and 83.67% AP on XD-Violence, a new state-of-the-art among zero-shot models.
  arXiv  Detail & Related papers  (2025-11-01T14:54:08Z)
• CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streams [0.42970700836450476]
  This paper introduces CALM, a novel, end-to-end framework for real-time anomaly detection.<n> CALM is built on the Apache Beam distributed processing framework.<n>It implements a closed-loop, continuous fine-tuning mechanism that allows the anomaly detection model to adapt to evolving data patterns in near real-time.
  arXiv  Detail & Related papers  (2025-08-29T00:27:35Z)
• Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Prompts [57.01985221057047]
  This paper introduces a novel method that learnstemporal prompt embeddings for weakly supervised video anomaly detection and localization (WSVADL) based on pre-trained vision-language models (VLMs) Our method achieves state-of-theart performance on three public benchmarks for the WSVADL task.
  arXiv  Detail & Related papers  (2024-08-12T03:31:29Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach [49.995833831087175]
  This work proposes a novel method for generating generic Video-temporal PAs by inpainting a masked out region of an image. In addition, we present a simple unified framework to detect real-world anomalies under the OCC setting. Our method performs on par with other existing state-of-the-art PAs generation and reconstruction based methods under the OCC setting.
  arXiv  Detail & Related papers  (2023-11-27T13:14:06Z)
• CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
  One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
  arXiv  Detail & Related papers  (2023-08-18T04:45:56Z)
• Harnessing Contrastive Learning and Neural Transformation for Time Series Anomaly Detection [0.0]
  Time series anomaly detection (TSAD) plays a vital role in many industrial applications.<n>Contrastive learning has gained momentum in the time series domain for its prowess in extracting meaningful representations from unlabeled data.<n>In this study, we propose a novel approach, CNT, that incorporates a window-based contrastive learning strategy fortified with learnable transformations.
  arXiv  Detail & Related papers  (2023-04-16T21:36:19Z)
• Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy [68.86835407617778]
  Anomaly Transformer achieves state-of-the-art performance on six unsupervised time series anomaly detection benchmarks. Anomaly Transformer achieves state-of-the-art performance on six unsupervised time series anomaly detection benchmarks.
  arXiv  Detail & Related papers  (2021-10-06T10:33:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.