TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models

• URL: http://arxiv.org/abs/2605.09904v2
• Date: Tue, 12 May 2026 03:09:23 GMT
• Title: TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models
• Abstract summary: Video large language models (Video-LLMs) have made strong progress in general video understanding, but their ability to maintain temporal object consistency remains underexplored.<n>We introduce TOC-Bench, a diagnostic benchmark for evaluating temporal object consistency in Video-LLMs.
• Score: 9.648992690108086
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video large language models (Video-LLMs) have made strong progress in general video understanding, but their ability to maintain temporal object consistency remains underexplored. Existing benchmarks often emphasize event recognition, action understanding, or coarse temporal reasoning, while rarely testing whether models can preserve the identity, state, and continuity of the same object across occlusion, disappearance, reappearance, state transitions, and cross-object interactions. We introduce TOC-Bench, a diagnostic benchmark for evaluating temporal object consistency in Video-LLMs. TOC-Bench is object-track grounded: each queried subject is linked to a per-frame trajectory and a structured temporal event timeline. To ensure that questions require temporally ordered visual evidence rather than language priors, single-frame shortcuts, or unordered frame cues, we design a three-layer temporal-necessity filtering protocol, which removes 60.7% of candidate QA pairs and retains 17,900 temporally dependent items across 10 diagnostic dimensions. From this pool, we construct a human-verified benchmark with 2,323 high-quality QA pairs over 1,951 videos. Experiments on representative Video-LLMs show that temporal object consistency remains a major unsolved challenge, with notable weaknesses in event counting, event ordering, identity-sensitive reasoning, and hallucination-aware verification, even when models perform well on general video understanding benchmarks. These results suggest that object-centric temporal coherence is a key bottleneck for current Video-LLMs, and that TOC-Bench provides a focused platform for diagnosing and improving object-aware temporal reasoning. The resource is available at https://github.com/cjzcjz666/toc_bench.git.

Related papers

• StateTrace: An Object-Centric Framework for Hidden-State Spatiotemporal Reasoning in Long Videos [45.587387649513325]
  StateTrace builds a reusabletemporal state memory that organizes trajectories, inter-object relations, and state-transition events into a structured reasoning substrate.<n>We show that StateTrace consistently improves performance on both public benchmarks and HSR-Bench.
  arXiv  Detail & Related papers  (2026-08-19T04:31:56Z)
• Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoning [28.325120357282156]
  Video temporal-logical reasoning requires models to maintain, update, and compose evidence as visual states evolve across frames.<n>We introduce Video-MME-Logical, a benchmark organized around five temporal-logical operations.<n>It enables difficulty-controlled final-answer evaluation by varying temporal horizon and reasoning complexity.
  arXiv  Detail & Related papers  (2026-06-26T08:12:04Z)
• Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models [154.39583176906893]
  Large language models (MLs) have advanced understanding, they remain highly prone to hallucinations in scenes.<n>We argue this stems from a failure intemporal monitoring, the ability to persistently track object dynamic identities, states, and relations over time.<n>Existing benchmarks obscure this deficit relying on single final-answer queries that can often be resolved via local visual cues or statistical priors.
  arXiv  Detail & Related papers  (2026-05-09T14:32:36Z)
• TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs [24.299498301173255]
  TimeBlind is a diagnostic benchmark for fine-grained temporal understanding.<n>We evaluate over 20 state-of-the-art MLLMs on 600 instances.<n>The Instance Accuracy of the best performing MLLM is only 48.2%, far below the human performance (98.2%)
  arXiv  Detail & Related papers  (2026-01-30T20:21:46Z)
• Temporal Prompting Matters: Rethinking Referring Video Object Segmentation [64.82333675385802]
  Referring Video Object (RVOS) aims to segment the object referred to by the query sentence in the video.<n>Most existing methods require end-to-end training with dense mask annotations.<n>We propose a Temporal Prompt Generation and Selection (Tenet) framework to address the referring and video factors.
  arXiv  Detail & Related papers  (2025-10-08T17:59:57Z)
• DATE: Dynamic Absolute Time Enhancement for Long Video Understanding [8.720269393713451]
  Long video understanding remains a fundamental challenge for multimodal large language models (MLLMs)<n>We propose Dynamic Absolute Time Enhancement (DATE) that enhances temporal awareness in MLLMs.<n>We introduce a two-stage algorithm to ensure both semantic relevance and temporal coverage.
  arXiv  Detail & Related papers  (2025-09-11T08:49:22Z)
• When and What: Diffusion-Grounded VideoLLM with Entity Aware Segmentation for Long Video Understanding [12.410012029024342]
  We present Grounded VideoDiT, a Video LLM designed to overcome limitations by introducing three key innovations.<n>First, a Diffusion Temporal Latent (DTL) encoder enhances boundary sensitivity and maintains temporal consistency.<n>Second, object grounded representations explicitly bind query entities to localized visual evidence, strengthening alignment.<n>Third, a mixed token scheme with discrete temporal timestamp tokens provides explicit modeling, enabling fine grained temporal reasoning.
  arXiv  Detail & Related papers  (2025-08-21T15:12:14Z)
• Causality Matters: How Temporal Information Emerges in Video Language Models [17.570777893613137]
  We find that removing or modifying positional encodings in video inputs yields minimal degradation in the performance of temporal understanding.<n>To explain this behavior, we conduct substantial analysis experiments to trace how temporal information is integrated within the model.<n>We propose two efficiency-oriented strategies: staged cross-modal attention and a temporal exit mechanism for early token truncation.
  arXiv  Detail & Related papers  (2025-08-15T16:33:14Z)
• V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning [40.18308199837137]
  We introduce a Video S-Temporal Reasoning (V-STa) benchmark to address these shortcomings.<n>We construct a dataset to elicit the spatial-temporal reasoning process of Video-LLMs.<n>Experiments from 14 Video-LLMs reveal significant gaps between current Video-LLMs and the needs for robust and consistent consistent reasoning.
  arXiv  Detail & Related papers  (2025-03-14T15:21:44Z)
• TimeLogic: A Temporal Logic Benchmark for Video QA [64.32208175236323]
  We introduce the TimeLogic QA (TLQA) framework to automatically generate temporal logical questions.<n>We leverage 4 datasets, STAR, Breakfast, AGQA, and CrossTask, and generate 2k and 10k QA pairs for each category.<n>We assess the VideoQA model's temporal reasoning performance on 16 categories of temporal logic with varying temporal complexity.
  arXiv  Detail & Related papers  (2025-01-13T11:12:59Z)
• On the Consistency of Video Large Language Models in Temporal Comprehension [57.985769348320616]
  Video large language models (Video-LLMs) can temporally ground language queries and retrieve video moments.<n>We conduct a study on prediction consistency -- a key indicator for robustness and trustworthiness of temporal grounding.
  arXiv  Detail & Related papers  (2024-11-20T00:47:17Z)
• TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models [75.42002690128486]
  TemporalBench is a new benchmark dedicated to evaluating fine-grained temporal understanding in videos. It consists of 10K video question-answer pairs, derived from 2K high-quality human annotations detailing the temporal dynamics in video clips. Results show that state-of-the-art models like GPT-4o achieve only 38.5% question answering accuracy on TemporalBench.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.