Conditional Multi-Event Temporal Grounding in Long-Form Video

• URL: http://arxiv.org/abs/2606.15320v1
• Date: Sat, 13 Jun 2026 14:27:58 GMT
• Title: Conditional Multi-Event Temporal Grounding in Long-Form Video
• Abstract summary: Existing benchmarks fall short: they localize only a single moment per query, count without temporal conditions, or treat grounding and counting as disjoint tasks.<n>We introduce CoMET-Bench for Multi-Event Temporal Grounding in long-form video, comprising 2789 queries over 600 videos averaging 33.8 minutes across five real-world domains.<n>We propose CoMET-Agent, a training-free agentic framework that reformulates the task as structured search-and-aggregate, improving F1@0.5 by 6.1% over GPT-5 purely through structural reasoning.
• Score: 19.941856770065854
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multimodal large language models have made rapid progress in video temporal grounding, yet real-world applications routinely require localizing every event that satisfies compositional temporal and spatial conditions. Existing benchmarks fall short: they localize only a single moment per query, count without temporal conditions, or treat grounding and counting as disjoint tasks. We introduce CoMET-Bench for Conditional Multi-Event Temporal Grounding in long-form video, comprising 2789 queries over 600 videos averaging 33.8 minutes across five real-world domains, with each query composed from 4 temporal conditions, 3 spatial conditions, and a dedicated negative-query subset. We further propose a unified evaluation protocol jointly measuring counting, grounding, and negative-query recognition, including a new Rejection-F1 metric that prevents trivial gaming by lazy "always-empty" models. Benchmarking a broad suite of MLLMs, agent-based, and grounding-specialized methods reveals that existing approaches remain far from solving this task. Building on these findings, we propose CoMET-Agent, a training-free agentic framework that reformulates the task as structured search-and-aggregate, improving F1@0.5 by 6.1% over GPT-5 purely through structural reasoning. Failure analysis further surfaces three open directions: fine-grained entity tracking, position-uniform retrieval, and causal event pairing.

Related papers

• Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games [69.57330692969543]
  We introduce RNG-Bench, a benchmark suite designed to isolate a base model's ability to reconstruct past observations.<n>RNG-Bench includes two complementary games: Matching Pairs, where card identities briefly revealed at specific locations must later be recalled, and 3D Maze, where egocentric views must be integrated into a spatial map.<n>The hardest configurations require contexts of roughly 128K tokens and 350 image inputs per episode, and remain far from saturated by frontier MLLMs.
  arXiv  Detail & Related papers  (2026-06-17T17:59:34Z)
• Towards One-to-Many Temporal Grounding [37.64021368680856]
  Real-world scenarios often require localizing multiple disjoint segments for a single query.<n>Previous state-of-the-art MLLMs, optimized for one-to-one settings, struggle in this context.<n>We present a systematic solution with three key contributions.
  arXiv  Detail & Related papers  (2026-06-04T15:31:22Z)
• TLG: Temporal-Logic Grounding for Video Question Answering via Source-Annotation Reconstruction and Category-Targeted Reasoning [0.0]
  TimeLogic Challenge evaluates formal temporal-logic reasoning over video.<n>End-to-end video-language models (VLMs) hover near chance on this task because they treat video as a bag of frames and cannot localize when actions occur.<n>We present TLG (Temporal-Logic Grounding), a three-tier system that reconstructs each video's action timeline from the public source-dataset annotations the benchmark was generated from.
  arXiv  Detail & Related papers  (2026-06-01T02:40:25Z)
• Towards Open-World Referring Expression Comprehension: A Benchmark with Training-free Multi-task Consistency Checker [4.731771934553726]
  Referring expression comprehension (REC) aims to localize a target object within an image based on a given expression.<n>OpenRef is a new benchmark for REC in complex visual and linguistic scenarios.
  arXiv  Detail & Related papers  (2026-05-25T11:05:37Z)
• ARGOS: Who, Where, and When in Agentic Multi-Camera Person Search [49.61286310968402]
  We introduce ARGOS, the first benchmark and framework that reformulates multi-camera person search as an interactive reasoning problem.<n>An ARGOS agent receives a vague witness statement and must decide what to ask, when to invoke spatial or temporal tools, and how to interpret ambiguous responses.<n>The benchmark comprises 2,691 tasks across 14 real-world scenarios in three progressive tracks: semantic perception (Who), spatial reasoning (Where), and temporal reasoning (When)
  arXiv  Detail & Related papers  (2026-04-14T14:06:19Z)
• Spatio-Temporal Grounding of Large Language Models from Perception Streams [3.426888015461207]
  We introduce Formally Explainable Scenes (FESTS) that injects verifiable temporal supervision into Large Language Models.<n>Training a 3-billion-level model on 27k suchs boosts frame size from magnitude to 88.5%, matching GPT-4.1 on complex complex-temporal reasoning.
  arXiv  Detail & Related papers  (2026-04-08T20:49:50Z)
• VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification [73.02304272829785]
  VideoBench is a hierarchical benchmark for long-video answering that rigorously verifies evidence.<n>It comprises 500 manually annotated questions across 13 domains, paired with temporal intervals and spatial bounding boxes as evidence.<n>Experiments show that even GeminiPro-3-Pro correctly answers fewer than 17% of questions under the standard end-to-end QA setting.<n>Results expose a significant gap between surface-level answer correctness and genuine evidence-based reasoning.
  arXiv  Detail & Related papers  (2026-04-02T03:29:43Z)
• ForeSea: AI Forensic Search with Multi-modal Queries for Video Surveillance [56.15563109738998]
  ForeSea is an AI forensic search system with a 3-stage, plug-and-play pipeline.<n>ForeSea improves accuracy by 3.5% and temporal IoU by 11.0 over prior VideoRAG models.
  arXiv  Detail & Related papers  (2026-03-24T07:15:28Z)
• OmniGround: A Comprehensive Spatio-Temporal Grounding Benchmark for Real-World Complex Scenarios [39.58602686069029]
  We introduce OmniGround, a comprehensive benchmark with 3,475 videos spanning 81 categories and complex real-world queries.<n>We also introduce DeepSTG, a systematic evaluation framework quantifying dataset quality across four complementary dimensions.<n>Experiments demonstrate PG-TAF achieves 25.6% and 35.6% improvements in m_tIoU and m_vIoU with consistent gains across four benchmarks.
  arXiv  Detail & Related papers  (2025-11-21T04:23:04Z)
• Towards Anytime Retrieval: A Benchmark for Anytime Person Re-Identification [85.78039373517021]
  Anytime Person Re-identification (AT-ReID) aims to achieve effective retrieval in multiple scenarios based on variations in time.<n>We collect the first large-scale dataset, AT-USTC, which contains 403k images of individuals wearing multiple clothes.<n>We propose a unified model named Uni-AT, which comprises a multi-scenario ReID framework for scenario-specific features learning.
  arXiv  Detail & Related papers  (2025-09-20T11:20:22Z)
• Grounding-MD: Grounded Video-language Pre-training for Open-World Moment Detection [67.70328796057466]
  Grounding-MD is an innovative, grounded video-language pre-training framework tailored for open-world moment detection.<n>Our framework incorporates an arbitrary number of open-ended natural language queries through a structured prompt mechanism.<n>Grounding-MD demonstrates exceptional semantic representation learning capabilities, effectively handling diverse and complex query conditions.
  arXiv  Detail & Related papers  (2025-04-20T09:54:25Z)
• Temporal Action Localization with Enhanced Instant Discriminability [66.76095239972094]
  Temporal action detection (TAD) aims to detect all action boundaries and their corresponding categories in an untrimmed video. We propose a one-stage framework named TriDet to resolve imprecise predictions of action boundaries by existing methods. Experimental results demonstrate the robustness of TriDet and its state-of-the-art performance on multiple TAD datasets.
  arXiv  Detail & Related papers  (2023-09-11T16:17:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.