VideoSearch-R1: Iterative Video Retrieval and Reasoning via Soft Query Refinement

• URL: http://arxiv.org/abs/2607.00446v1
• Date: Wed, 01 Jul 2026 04:59:24 GMT
• Title: VideoSearch-R1: Iterative Video Retrieval and Reasoning via Soft Query Refinement
• Abstract summary: VideoSearch-R1 is an agentic framework for iterative video retrieval and reasoning through multi-turn interaction with a video search engine.<n>VideoSearch-R1 achieves state-of-the-art performance across three datasets on Video Moment Retrieval (VCMR)
• Score: 31.30208652334818
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As video corpora continue to expand in both scale and task complexity, there is increasing demand for approaches that retrieve relevant videos from large-scale corpora (inter-video reasoning) and subsequently perform fine-grained, query-conditioned tasks (intra-video reasoning) within the retrieved content, such as temporal grounding. However, existing approaches typically treat retrieval as a preprocessing step, and consequently, when the initial retrieval fails, there is no mechanism to refine the search, leading to the failure of subsequent fine-grained intra-video reasoning. Moreover, while recent agentic frameworks have advanced video understanding, they typically assume that the query-relevant video is already given, focusing exclusively on intra-video reasoning tasks. To address these limitations, we propose VideoSearch-R1, an agentic framework for iterative video retrieval and reasoning through multi-turn interaction with a video search engine. Specifically, we introduce Soft Query Refinement (SQR) to refine search query tokens in a continuous latent space rather than rewriting queries in the discrete text space, enabling more efficient and fine-grained adjustments. SQR and its reasoning process are trained using Group Relative Policy Optimization (GRPO), guided by task-level reward signals derived from retrieval and downstream tasks. Building upon this, VideoSearch-R1 achieves state-of-the-art performance across three datasets on Video Corpus Moment Retrieval (VCMR), iteratively retrieving videos from large-scale corpora, refining search queries, and performing precise query-conditioned temporal grounding within the retrieved content. Our analyses show that SQR effectively refines the original query, requiring significantly fewer generated tokens than explicit text-level query refinement. Code and model checkpoints are publicly available at mlvlab.github.io/VideoSearch-R1.

Related papers

• QSVideo: Query-Conditioned Semantic Temporal Retrieval for Video Understanding [31.789999490173255]
  Multimodal retrieval has emerged as a critical component of video understanding.<n>Existing multimodal retrieval methods suffer from biased relevance estimation, limited diversity, and temporal collapse.<n>We propose QSVideo, a unified framework that addresses relevance, diversity, and temporal modeling in video retrieval.
  arXiv  Detail & Related papers  (2026-07-06T00:19:42Z)
• Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language [60.91064560080974]
  We propose an efficient approach, SpotVMR, to trim the query-relevant clip.<n>Our proposed SpotVMR can serve as plug-and-play module, which achieves efficiency for state-of-the-art VMR methods.
  arXiv  Detail & Related papers  (2026-05-28T11:42:28Z)
• Video-BrowseComp: Benchmarking Agentic Video Research on Open Web [64.53060049124961]
  Video-BrowseComp is a benchmark comprising 210 questions tailored for open-web agentic video reasoning.<n>It enforces a mandatory dependency on temporal visual evidence, ensuring answers cannot be derived solely through text search.<n>As the first open-web video research benchmark, Video-BrowseComp advances the field beyond passive perception toward proactive video reasoning.
  arXiv  Detail & Related papers  (2025-12-28T19:08:27Z)
• TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learning [30.895558843927862]
  Temporal search aims to identify a minimal set of relevant frames from tens of thousands based on a given query.<n>TimeSearch-R reformulates temporal search as interleaved text-video thinking.<n>It seamlessly integrates searching video clips into the reasoning process through reinforcement learning.
  arXiv  Detail & Related papers  (2025-11-07T18:58:25Z)
• CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videos [59.391265901911005]
  We propose CoT-RVS, a novel framework employing the zero-shot Chain-of-Thought (CoT) capability of MLLM to address complex challenges by temporal-semantic reasoning.<n>CoT-RVS analyzes the visible objects within a given frame that possibly match the language query (semantic), and chooses a corresponding for each object that can be observed effortlessly among all frames (temporal)<n>Our framework's training-free feature further allows its extension to process online video streams, where the CoT is used at test time to update the object of interest when a better target starts to emerge
  arXiv  Detail & Related papers  (2025-05-24T07:01:31Z)
• A Flexible and Scalable Framework for Video Moment Search [51.47907684209207]
  This paper introduces a flexible framework for retrieving a ranked list of moments from collection of videos in any length to match a text query.<n>Our framework, called Segment-Proposal-Ranking (SPR), simplifies the search process into three independent stages: segment retrieval, proposal generation, and moment refinement with re-ranking.<n> Evaluations on the TVR-Ranking dataset demonstrate that our framework achieves state-of-the-art performance with significant reductions in computational cost and processing time.
  arXiv  Detail & Related papers  (2025-01-09T08:54:19Z)
• Query-centric Audio-Visual Cognition Network for Moment Retrieval, Segmentation and Step-Captioning [56.873534081386]
  A new topic HIREST is presented, including video retrieval, moment retrieval, moment segmentation, and step-captioning.<n>We propose a query-centric audio-visual cognition network to construct a reliable multi-modal representation for three tasks.<n>This can cognize user-preferred content and thus attain a query-centric audio-visual representation for three tasks.
  arXiv  Detail & Related papers  (2024-12-18T06:43:06Z)
• Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach [56.610806615527885]
  A key challenge in text-video retrieval (TVR) is the information asymmetry between video and text.<n>This paper introduces a data-centric framework to bridge this gap by enriching textual representations to better match the richness of video content.<n>We propose a query selection mechanism that identifies the most relevant and diverse queries, reducing computational cost while improving accuracy.
  arXiv  Detail & Related papers  (2024-08-14T01:24:09Z)
• Improving Video Corpus Moment Retrieval with Partial Relevance Enhancement [72.7576395034068]
  Video Corpus Moment Retrieval (VCMR) is a new video retrieval task aimed at retrieving a relevant moment from a large corpus of untrimmed videos using a text query. We argue that effectively capturing the partial relevance between the query and video is essential for the VCMR task. For video retrieval, we introduce a multi-modal collaborative video retriever, generating different query representations for the two modalities. For moment localization, we propose the focus-then-fuse moment localizer, utilizing modality-specific gates to capture essential content.
  arXiv  Detail & Related papers  (2024-02-21T07:16:06Z)
• Zero-shot Audio Topic Reranking using Large Language Models [42.774019015099704]
  Multimodal Video Search by Examples (MVSE) investigates using video clips as the query term for information retrieval. This work aims to compensate for any performance loss from this rapid archive search by examining reranking approaches. Performance is evaluated for topic-based retrieval on a publicly available video archive, the BBC Rewind corpus.
  arXiv  Detail & Related papers  (2023-09-14T11:13:36Z)
• Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrieval [98.62404433761432]
  The rapid growth of user-generated videos on the Internet has intensified the need for text-based video retrieval systems. Traditional methods mainly favor the concept-based paradigm on retrieval with simple queries. We propose a Tree-augmented Cross-modal. method by jointly learning the linguistic structure of queries and the temporal representation of videos.
  arXiv  Detail & Related papers  (2020-07-06T02:50:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.