LongVidSearch: An Agentic Benchmark for Multi-hop Evidence Retrieval Planning in Long Videos

• URL: http://arxiv.org/abs/2603.14468v1
• Date: Sun, 15 Mar 2026 16:20:23 GMT
• Title: LongVidSearch: An Agentic Benchmark for Multi-hop Evidence Retrieval Planning in Long Videos
• Abstract summary: LongVidSearch is a benchmark for evaluating agentic multi-hop evidence retrieval planning in long videos.<n>A Hop-k question requires exactly k necessary evidence clips, and removing any single clip renders the question unsolvable.
• Score: 7.139631028105273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long video question answering (Long-Video QA) increasingly relies on agentic tool use to retrieve evidence from long videos. In realistic settings, this process often requires multi-hop retrieval, where agents must iteratively gather multiple discontinuous evidence clips. However, existing long-video benchmarks are largely static: they rarely enforce strict multi-hop retrieval and typically lack a standardized evidence-access interface, making it difficult to separate failures in retrieval planning from those in answer generation. To address this gap, we introduce LongVidSearch, a benchmark for evaluating agentic multi-hop evidence retrieval planning in long videos under standardized access constraints. LongVidSearch enforces retrieval necessity: a Hop-k question requires exactly k necessary evidence clips, and removing any single clip renders the question unsolvable. The benchmark contains 3,000 questions over 447 long videos (average length 26 minutes), covering four reasoning categories: State Mutation, Causal Inference, Global Summary, and Visual Tracking, with 2-hop, 3-hop, and 4-hop evidence requirements. To ensure fair and controlled evaluation, all agents interact with LongVidSearch through a unified tool interface, which fixes the retrieval backend and isolates the agent's ability to formulate queries and plan iterative retrieval. In addition to answer accuracy, we measure tool-call cost to analyze the accuracy-efficiency trade-off under identical access conditions. We evaluate VideoAgent-style QA agents with multiple backbone LLMs using three-judge majority voting. GPT-5 achieves the highest accuracy (42.43), outperforming Gemini 3 Pro (30.97) and GPT-4o (19.20), yet remaining below 50 %, highlighting the difficulty of multi-hop retrieval planning. With gold evidence clips, performance becomes near-perfect, confirming retrieval planning as the primary bottleneck.

Related papers

• SearchMaster: Grounded and Regulated Self-Play for Search Agents [57.537666863340284]
  Training search agents requires high-quality search data.<n> Existing pipelines often depend on human-written tasks, expert demonstrations, or stronger teacher models.<n>We present SearchMaster, a self-play framework that trains a single LLM from search tasks it generates, solves, and verifies in a local search environment.
  arXiv  Detail & Related papers  (2026-08-03T07:29:11Z)
• SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain [11.397893639012272]
  We introduce SVFSearch, the first open benchmark for short-video frame search in the Chinese gaming domain.<n>We evaluate representative paradigms ranging from direct QA and RAG workflow to Plan-Act-Replan agents and learned search models.<n>Results reveal a large gap between model-only answering, practical agentic search, and oracle knowledge.
  arXiv  Detail & Related papers  (2026-05-18T07:03:48Z)
• VideoSEAL: Mitigating Evidence Misalignment in Agentic Long Video Understanding by Decoupling Answer Authority [10.961731621875918]
  Long video question answering requires locating sparse, time-inspected visual evidence within highly redundant content.<n>We show that existing LVU agents can exhibit "evidence misalignment"<n>We propose the decoupled planner-or framework, which separates planning from answer authority and gates final answering on pixel-level verification.
  arXiv  Detail & Related papers  (2026-05-12T10:37:49Z)
• ForeSea: AI Forensic Search with Multi-modal Queries for Video Surveillance [56.15563109738998]
  ForeSea is an AI forensic search system with a 3-stage, plug-and-play pipeline.<n>ForeSea improves accuracy by 3.5% and temporal IoU by 11.0 over prior VideoRAG models.
  arXiv  Detail & Related papers  (2026-03-24T07:15:28Z)
• Video-BrowseComp: Benchmarking Agentic Video Research on Open Web [64.53060049124961]
  Video-BrowseComp is a benchmark comprising 210 questions tailored for open-web agentic video reasoning.<n>It enforces a mandatory dependency on temporal visual evidence, ensuring answers cannot be derived solely through text search.<n>As the first open-web video research benchmark, Video-BrowseComp advances the field beyond passive perception toward proactive video reasoning.
  arXiv  Detail & Related papers  (2025-12-28T19:08:27Z)
• HERBench: A Benchmark for Multi-Evidence Integration in Video Question Answering [13.370338205427911]
  We present HERBench, a VideoQA benchmark to assess multi-evidence integration across time.<n>HERBench comprises 26K five-way multiple-choice questions organized into twelve compositional tasks.<n>We show that HERBench imposes substantially higher demand than prior datasets.
  arXiv  Detail & Related papers  (2025-12-16T19:34:47Z)
• LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling [87.98096428508181]
  LongVT is an end-to-end agentic framework that enables "Thinking with Long Videos" via interleaved Multimodal Chain-of-Tool-Thought.<n>We exploit LMMs' inherent temporal grounding ability as a native video cropping tool to zoom in on a specific video clip and resample finer-grained video frames.<n>Our training dataset consists of 247.9K samples for tool-integrated cold-start supervised fine-tuning, 1.6K samples for agentic reinforcement learning, and 15.4K samples for agentic reinforcement fine-tuning.
  arXiv  Detail & Related papers  (2025-11-25T19:22:48Z)
• Beyond the limitation of a single query: Train your LLM for query expansion with Reinforcement Learning [23.104182075898297]
  Reasoning-augmented search agents, such as Search-R1, are trained to reason, search, and generate the final answer iteratively.<n>We train an LLM-based search agent with the native capability of query expansion through reinforcement learning.<n>With the assistance of the squeezer model, we discover that even a small-scale 3B LLM can demonstrate a strong capability of query expansion.
  arXiv  Detail & Related papers  (2025-10-11T04:23:30Z)
• MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks [67.31276358668424]
  We introduce a novel task named AV-HaystacksQA, where the goal is to identify salient segments across different videos in response to a query and link them together to generate the most informative answer.<n> AVHaystacks is an audio-visual benchmark comprising 3100 annotated QA pairs designed to assess the capabilities of LMMs in multi-video retrieval and temporal grounding task.<n>We propose a model-agnostic, multi-agent framework to address this challenge, achieving up to 89% and 65% relative improvements over baseline methods on BLEU@4 and GPT evaluation scores in QA task on our proposed AVHaystack
  arXiv  Detail & Related papers  (2025-06-08T06:34:29Z)
• Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding [60.88843818016968]
  Long-form video understanding presents significant challenges due to temporal-spatial complexity and difficulty of question answering.<n>We propose the Deep Video Discovery (DVD) agent to leverage an agentic search strategy over segmented video clips.<n>Our DVD agent achieves state-of-the-art performance on the challenging LVBench dataset, reaching an accuracy of 74.2%.
  arXiv  Detail & Related papers  (2025-05-23T16:37:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.