TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models

• URL: http://arxiv.org/abs/2605.09904v1
• Date: Mon, 11 May 2026 02:47:59 GMT
• Title: TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models
• Abstract summary: Video large language models (Video-LLMs) have achieved remarkable progress in general video understanding.<n>We introduce a diagnostic benchmark specifically designed to evaluate temporal object consistency in Video-LLMs.<n>To ensure that benchmark items depend on temporally ordered visual evidence, we propose a three-layer temporal-necessity filtering protocol.
• Score: 10.22422531424139
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video large language models (Video-LLMs) have achieved remarkable progress in general video understanding, yet their ability to maintain temporal object consistency remains insufficiently explored. Existing benchmarks primarily focus on event recognition, action understanding, or coarse temporal reasoning, but rarely evaluate whether a model can consistently preserve the identity, state, and temporal continuity of the same object across occlusion, disappearance, reappearance, state transitions, and cross-object interactions. As a result, current evaluations may overestimate temporal reasoning ability while overlooking failures in object-centric temporal coherence. To address this issue, we introduce TOC-Bench, a diagnostic benchmark specifically designed to evaluate temporal object consistency in Video-LLMs. TOC-Bench is explicitly object-track grounded, where each queried subject is associated with a per frame object trajectory and structured temporal event timeline. To ensure that benchmark items depend on temporally ordered visual evidence rather than language priors, single-frame shortcuts, or unordered frame cues, we propose a three-layer temporal-necessity filtering protocol that removes 60.7% of candidate QA pairs and retains 17,900 temporally dependent items spanning 10 diagnostic dimensions. From this filtered pool, we further construct a human-verified benchmark containing 2,323 high-quality QA pairs over 1,951 videos. Experiments on representative Video-LLMs show that temporal object consistency remains a major unsolved challenge. Current models exhibit substantial weaknesses in event counting, event ordering, identity-sensitive reasoning, and hallucination-aware verification, despite strong performance on general video understanding benchmarks.

Related papers

• StateTrace: An Object-Centric Framework for Hidden-State Spatiotemporal Reasoning in Long Videos [45.587387649513325]
  StateTrace builds a reusabletemporal state memory that organizes trajectories, inter-object relations, and state-transition events into a structured reasoning substrate.<n>We show that StateTrace consistently improves performance on both public benchmarks and HSR-Bench.
  arXiv  Detail & Related papers  (2026-08-19T04:31:56Z)
• Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoning [28.325120357282156]
  Video temporal-logical reasoning requires models to maintain, update, and compose evidence as visual states evolve across frames.<n>We introduce Video-MME-Logical, a benchmark organized around five temporal-logical operations.<n>It enables difficulty-controlled final-answer evaluation by varying temporal horizon and reasoning complexity.
  arXiv  Detail & Related papers  (2026-06-26T08:12:04Z)
• Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models [154.39583176906893]
  Large language models (MLs) have advanced understanding, they remain highly prone to hallucinations in scenes.<n>We argue this stems from a failure intemporal monitoring, the ability to persistently track object dynamic identities, states, and relations over time.<n>Existing benchmarks obscure this deficit relying on single final-answer queries that can often be resolved via local visual cues or statistical priors.
  arXiv  Detail & Related papers  (2026-05-09T14:32:36Z)
• TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs [24.299498301173255]
  TimeBlind is a diagnostic benchmark for fine-grained temporal understanding.<n>We evaluate over 20 state-of-the-art MLLMs on 600 instances.<n>The Instance Accuracy of the best performing MLLM is only 48.2%, far below the human performance (98.2%)
  arXiv  Detail & Related papers  (2026-01-30T20:21:46Z)
• Temporal Prompting Matters: Rethinking Referring Video Object Segmentation [64.82333675385802]
  Referring Video Object (RVOS) aims to segment the object referred to by the query sentence in the video.<n>Most existing methods require end-to-end training with dense mask annotations.<n>We propose a Temporal Prompt Generation and Selection (Tenet) framework to address the referring and video factors.
  arXiv  Detail & Related papers  (2025-10-08T17:59:57Z)
• DATE: Dynamic Absolute Time Enhancement for Long Video Understanding [8.720269393713451]
  Long video understanding remains a fundamental challenge for multimodal large language models (MLLMs)<n>We propose Dynamic Absolute Time Enhancement (DATE) that enhances temporal awareness in MLLMs.<n>We introduce a two-stage algorithm to ensure both semantic relevance and temporal coverage.
  arXiv  Detail & Related papers  (2025-09-11T08:49:22Z)
• When and What: Diffusion-Grounded VideoLLM with Entity Aware Segmentation for Long Video Understanding [12.410012029024342]
  We present Grounded VideoDiT, a Video LLM designed to overcome limitations by introducing three key innovations.<n>First, a Diffusion Temporal Latent (DTL) encoder enhances boundary sensitivity and maintains temporal consistency.<n>Second, object grounded representations explicitly bind query entities to localized visual evidence, strengthening alignment.<n>Third, a mixed token scheme with discrete temporal timestamp tokens provides explicit modeling, enabling fine grained temporal reasoning.
  arXiv  Detail & Related papers  (2025-08-21T15:12:14Z)
• Causality Matters: How Temporal Information Emerges in Video Language Models [17.570777893613137]
  We find that removing or modifying positional encodings in video inputs yields minimal degradation in the performance of temporal understanding.<n>To explain this behavior, we conduct substantial analysis experiments to trace how temporal information is integrated within the model.<n>We propose two efficiency-oriented strategies: staged cross-modal attention and a temporal exit mechanism for early token truncation.
  arXiv  Detail & Related papers  (2025-08-15T16:33:14Z)
• V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning [40.18308199837137]
  We introduce a Video S-Temporal Reasoning (V-STa) benchmark to address these shortcomings.<n>We construct a dataset to elicit the spatial-temporal reasoning process of Video-LLMs.<n>Experiments from 14 Video-LLMs reveal significant gaps between current Video-LLMs and the needs for robust and consistent consistent reasoning.
  arXiv  Detail & Related papers  (2025-03-14T15:21:44Z)
• TimeLogic: A Temporal Logic Benchmark for Video QA [64.32208175236323]
  We introduce the TimeLogic QA (TLQA) framework to automatically generate temporal logical questions.<n>We leverage 4 datasets, STAR, Breakfast, AGQA, and CrossTask, and generate 2k and 10k QA pairs for each category.<n>We assess the VideoQA model's temporal reasoning performance on 16 categories of temporal logic with varying temporal complexity.
  arXiv  Detail & Related papers  (2025-01-13T11:12:59Z)
• On the Consistency of Video Large Language Models in Temporal Comprehension [57.985769348320616]
  Video large language models (Video-LLMs) can temporally ground language queries and retrieve video moments.<n>We conduct a study on prediction consistency -- a key indicator for robustness and trustworthiness of temporal grounding.
  arXiv  Detail & Related papers  (2024-11-20T00:47:17Z)
• TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models [75.42002690128486]
  TemporalBench is a new benchmark dedicated to evaluating fine-grained temporal understanding in videos. It consists of 10K video question-answer pairs, derived from 2K high-quality human annotations detailing the temporal dynamics in video clips. Results show that state-of-the-art models like GPT-4o achieve only 38.5% question answering accuracy on TemporalBench.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.