Routing Before Looking: Query-Adaptive Evidence Acquisition for Long-form Video Understanding

• URL: http://arxiv.org/abs/2608.20805v1
• Date: Fri, 21 Aug 2026 07:23:35 GMT
• Title: Routing Before Looking: Query-Adaptive Evidence Acquisition for Long-form Video Understanding
• Abstract summary: Route2Look is a lightweight and model-agnostic framework for query-adaptive evidence acquisition in long-form video understanding.<n> Route2Look operates in a Route-Look-Memorize loop with three tools: Global Browse for holistic context, Temporal Ground for explicit temporal cues, and Semantic Retrieve for semantic search.<n> Experiments on challenging long-video benchmarks show that Route2Look achieves state-of-the-art performance while maintaining strong frame efficiency across datasets and query types.
• Score: 34.47367969610384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-form video understanding remains challenging for video agents due to the mismatch between query demands and evidence acquisition strategies. Although recent planning-before-perception methods outperform query-agnostic pipelines, they often rely on a single dominant strategy, either generation-based strategy or retrieval-based strategy, limiting their ability to handle diverse query demands. We propose Route2Look, a lightweight and model-agnostic framework for query-adaptive evidence acquisition in long-form video understanding. Route2Look operates in a Route-Look-Memorize loop with three tools: Global Browse for holistic context, Temporal Ground for explicit temporal cues, and Semantic Retrieve for semantic search. The core component is a routing policy that dynamically selects evidence acquisition tools based on the query. To build this policy, Route2Look adopts a two-stage design: first distilling the routing skill from differential contrastive analysis between generation-based and retrieval-based trajectories, and then applying the distilled skill with hard routing rules and continue-or-stop criteria during inference. Experiments on challenging long-video benchmarks show that Route2Look achieves state-of-the-art performance while maintaining strong frame efficiency across datasets and query types. Oracle routing analysis further reveals the potential of query-adaptive evidence acquisition for future long-form video understanding.

Related papers

• ORDER: Task-Conditioned Routing for Retrieval-Augmented Generation [3.6028767235872965]
  In this paper, we introduce a query-conditioned RAG framework that jointly adapts indexing and retrieval to the incoming query.<n>We evaluate our framework on large-scale, heterogeneous historical archives and show that conditioning both indexing and retrieval on the query consistently outperforms both naive baselines and strong state-of-the-art RAG systems.
  arXiv  Detail & Related papers  (2026-09-15T11:23:16Z)
• VisualRouter: Query-Grounded Visual Sampling for Long Video Understanding [60.161767365040554]
  We propose a training-free and plug-and-play framework for query-grounded visual sampling.<n>Visual first classifies each query as either global or local and then applies the corresponding sampling strategy.<n>Experiments show that Visual consistently improves multiple LVLMs over uniform sampling, achieving gains of 5.2%, 7.7%, and 11.6% on Video-MME, LongVideoBench, and MLVU with Qwen2.5-VL-7B, and outperforming existing training-free visual sampling methods under the same setting.
  arXiv  Detail & Related papers  (2026-07-30T16:23:10Z)
• DART: Difficulty-Adaptive Routing for Zero-Shot Video Temporal Grounding [41.985888701957826]
  Zero-shot video temporal grounding (VTG) localizes events in untrimmed videos from natural language queries without task-specific training.<n>We propose DART, which bridges the gap by coupling difficulty-aware routing with structured reasoning in large vision models.<n>On Charades-STA and ActivityNet Captions, DART achieves state-of-the-art zero-shot performance across both identically distributed and multiple out-of-distribution settings.
  arXiv  Detail & Related papers  (2026-07-01T09:18:59Z)
• VideoSearch-R1: Iterative Video Retrieval and Reasoning via Soft Query Refinement [31.30208652334818]
  VideoSearch-R1 is an agentic framework for iterative video retrieval and reasoning through multi-turn interaction with a video search engine.<n>VideoSearch-R1 achieves state-of-the-art performance across three datasets on Video Moment Retrieval (VCMR)
  arXiv  Detail & Related papers  (2026-07-01T04:59:24Z)
• Reason Before You Retrieve: Agentic Planning for Multi-modal RAG [37.890744636035144]
  We propose MM-R2, a multimodal agentic retrieval framework that reasons before retrieval by explicitly modeling both what to retrieve and where to search.<n> MM-R2 first constructs an intent-grounded retrieval state from the image-question pair, capturing the information need, grounded referent, and retrieval constraints.<n>It then performs retrieval over a structured KnowledgeMap, where the agent selects relevant retrieval units before issuing grounded queries within them.<n>Experiments on Infoseek and Encyclopedic VQA datasets show that MM-R2 substantially outperforms strong baselines on answer accuracy while also yielding more interpretable and verifiable retrieval trajectories.
  arXiv  Detail & Related papers  (2026-06-24T02:40:49Z)
• Rethinking RAG in Long Videos: What to Retrieve and How to Use It? [56.38819694781005]
  We introduce V-RAGBench, a benchmark of $langle$query, evidence chunk, answer$rangle$ triplets that enables faithful, decoupled evaluation of retrieval and generation.<n>We also present CARVE, a method that runs parallel retrievers across configurations and employs chunk-adaptive reranking to identify the winning configuration for each chunk.
  arXiv  Detail & Related papers  (2026-06-11T10:05:49Z)
• Learning to Route: A Rule-Driven Agent Framework for Hybrid-Source Retrieval-Augmented Generation [55.47971671635531]
  Large Language Models (LLMs) have shown remarkable performance on general Question Answering (QA)<n>Retrieval-Augmented Generation (RAG) addresses this limitation by enriching LLMs with external knowledge.<n>Existing systems primarily rely on unstructured documents, while largely overlooking relational databases.
  arXiv  Detail & Related papers  (2025-09-30T22:19:44Z)
• Logic-in-Frames: Dynamic Keyframe Search via Visual Semantic-Logical Verification for Long Video Understanding [23.022070084937603]
  We introduce a semantics-driven search framework that reformulates selection under the paradigm of Visual Semantic-Logical Search.<n>Our method establishes new SOTA performance on the manually annotated benchmark in key-frame selection metrics.
  arXiv  Detail & Related papers  (2025-03-17T13:07:34Z)
• Single-Stage Visual Query Localization in Egocentric Videos [79.71065005161566]
  We propose a single-stage VQL framework that is end-to-end trainable. We establish the query-video relationship by considering query-to-frame correspondences between the query and each video frame. Our experiments demonstrate that our approach outperforms prior VQL methods by 20% accuracy while obtaining a 10x improvement in inference speed.
  arXiv  Detail & Related papers  (2023-06-15T17:57:28Z)
• DeepQAMVS: Query-Aware Hierarchical Pointer Networks for Multi-Video Summarization [127.16984421969529]
  We introduce a novel Query-Aware Hierarchical Pointer Network for Multi-Video Summarization, termed DeepQAMVS. DeepQAMVS is trained with reinforcement learning, incorporating rewards that capture representativeness, diversity, query-adaptability and temporal coherence. We achieve state-of-the-art results on the MVS1K dataset, with inference time scaling linearly with the number of input video frames.
  arXiv  Detail & Related papers  (2021-05-13T17:33:26Z)
• Query Resolution for Conversational Search with Limited Supervision [63.131221660019776]
  We propose QuReTeC (Query Resolution by Term Classification), a neural query resolution model based on bidirectional transformers. We show that QuReTeC outperforms state-of-the-art models, and furthermore, that our distant supervision method can be used to substantially reduce the amount of human-curated data required to train QuReTeC.
  arXiv  Detail & Related papers  (2020-05-24T11:37:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.