Deep Temporal Reasoning in Video Language Models: A Cross-Linguistic Evaluation of Action Duration and Completion through Perfect Times

• URL: http://arxiv.org/abs/2506.00928v1
• Date: Sun, 01 Jun 2025 09:45:41 GMT
• Title: Deep Temporal Reasoning in Video Language Models: A Cross-Linguistic Evaluation of Action Duration and Completion through Perfect Times
• Authors: Olga Loginova, Sofía Ortega Loguinova,
• Abstract summary: We introduce the textbfPerfect Times dataset, a quadrilingual (English, Italian, Russian, and Japanese) multiple-choice question-answering benchmark designed to assess video-language models (VLMs) on temporal reasoning.<n> Experimental results indicate that state-of-the-art models, despite their success on text-based tasks, struggle to mirror human-like temporal and causal reasoning grounded in video.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human perception of events is intrinsically tied to distinguishing between completed (perfect and telic) and ongoing (durative) actions, a process mediated by both linguistic structure and visual cues. In this work, we introduce the \textbf{Perfect Times} dataset, a novel, quadrilingual (English, Italian, Russian, and Japanese) multiple-choice question-answering benchmark designed to assess video-language models (VLMs) on temporal reasoning. By pairing everyday activity videos with event completion labels and perfectivity-tailored distractors, our dataset probes whether models truly comprehend temporal dynamics or merely latch onto superficial markers. Experimental results indicate that state-of-the-art models, despite their success on text-based tasks, struggle to mirror human-like temporal and causal reasoning grounded in video. This study underscores the necessity of integrating deep multimodal cues to capture the nuances of action duration and completion within temporal and causal video dynamics, setting a new standard for evaluating and advancing temporal reasoning in VLMs.

Related papers

• ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Events [0.20132569095596248]
  We present ChronoSense, a new benchmark for evaluating Large Language Models' temporal understanding.<n>We assess the performance of seven recent LLMs using this benchmark and the results indicate that models handle Allen relations, even symmetrical ones, quite differently.<n>Overall, the models' low performance highlights the need for improved temporal understanding in LLMs.
  arXiv  Detail & Related papers  (2025-01-06T14:27:41Z)
• Temporal Contrastive Learning for Video Temporal Reasoning in Large Vision-Language Models [44.99833362998488]
  Temporal Semantic Alignment via Dynamic Prompting (TSADP) is a novel framework that enhances temporal reasoning capabilities.<n>We evaluate TSADP on the VidSitu dataset, augmented with enriched temporal annotations.<n>Our analysis highlights the robustness, efficiency, and practical utility of TSADP, making it a step forward in the field of video-language understanding.
  arXiv  Detail & Related papers  (2024-12-16T02:37:58Z)
• TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models [75.42002690128486]
  TemporalBench is a new benchmark dedicated to evaluating fine-grained temporal understanding in videos. It consists of 10K video question-answer pairs, derived from 2K high-quality human annotations detailing the temporal dynamics in video clips. Results show that state-of-the-art models like GPT-4o achieve only 38.5% question answering accuracy on TemporalBench.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)
• An Overview Of Temporal Commonsense Reasoning and Acquisition [20.108317515225504]
  Temporal commonsense reasoning refers to the ability to understand the typical temporal context of phrases, actions, and events. Recent research on the performance of large language models suggests that they often take shortcuts in their reasoning and fall prey to simple linguistic traps.
  arXiv  Detail & Related papers  (2023-07-28T01:30:15Z)
• Test of Time: Instilling Video-Language Models with a Sense of Time [42.290970800790184]
  Seven existing video-language models struggle to understand simple temporal relations. We propose a temporal adaptation recipe on top of one such model, VideoCLIP, based on post-pretraining on a small amount of video-text data. We observe encouraging performance gains especially when the task needs higher time awareness.
  arXiv  Detail & Related papers  (2023-01-05T14:14:36Z)
• HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training [49.52679453475878]
  We propose a Temporal-Aware video-language pre-training framework, HiTeA, for modeling cross-modal alignment between moments and texts. We achieve state-of-the-art results on 15 well-established video-language understanding and generation tasks.
  arXiv  Detail & Related papers  (2022-12-30T04:27:01Z)
• Revisiting the "Video" in Video-Language Understanding [56.15777956496518]
  We propose the atemporal probe (ATP), a new model for video-language analysis. We characterize the limitations and potential of current video-language benchmarks. We show that effectively integrating ATP into full video-level temporal models can improve efficiency and state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2022-06-03T17:57:33Z)
• TIMEDIAL: Temporal Commonsense Reasoning in Dialog [43.24596551545824]
  We present the first study to investigate pre-trained language models for their temporal reasoning capabilities in dialogs. We formulate TIME-DIAL as a multiple-choice cloze task with over 1.1K carefully curated dialogs. Empirical results demonstrate that even the best performing models struggle on this task compared to humans.
  arXiv  Detail & Related papers  (2021-06-08T17:59:21Z)
• MERLOT: Multimodal Neural Script Knowledge Models [74.05631672657452]
  We introduce MERLOT, a model that learns multimodal script knowledge by watching millions of YouTube videos with transcribed speech. MERLOT exhibits strong out-of-the-box representations of temporal commonsense, and achieves state-of-the-art performance on 12 different video QA datasets. On Visual Commonsense Reasoning, MERLOT answers questions correctly with 80.6% accuracy, outperforming state-of-the-art models of similar size by over 3%.
  arXiv  Detail & Related papers  (2021-06-04T17:57:39Z)
• DORi: Discovering Object Relationship for Moment Localization of a Natural-Language Query in Video [98.54696229182335]
  We study the task of temporal moment localization in a long untrimmed video using natural language query. Our key innovation is to learn a video feature embedding through a language-conditioned message-passing algorithm. A temporal sub-graph captures the activities within the video through time.
  arXiv  Detail & Related papers  (2020-10-13T09:50:29Z)
• Local-Global Video-Text Interactions for Temporal Grounding [77.5114709695216]
  This paper addresses the problem of text-to-video temporal grounding, which aims to identify the time interval in a video semantically relevant to a text query. We tackle this problem using a novel regression-based model that learns to extract a collection of mid-level features for semantic phrases in a text query. The proposed method effectively predicts the target time interval by exploiting contextual information from local to global.
  arXiv  Detail & Related papers  (2020-04-16T08:10:41Z)
• Spatio-Temporal Graph for Video Captioning with Knowledge Distillation [50.034189314258356]
  We propose a graph model for video captioning that exploits object interactions in space and time. Our model builds interpretable links and is able to provide explicit visual grounding. To avoid correlations caused by the variable number of objects, we propose an object-aware knowledge distillation mechanism.
  arXiv  Detail & Related papers  (2020-03-31T03:58:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.