Not All Inputs Are Valid: Towards Open-Set Video Moment Retrieval Using Language

• URL: http://arxiv.org/abs/2605.29812v1
• Date: Thu, 28 May 2026 11:57:14 GMT
• Title: Not All Inputs Are Valid: Towards Open-Set Video Moment Retrieval Using Language
• Abstract summary: Video Moment Retrieval (VMR) targets to retrieve the specific moment corresponding to a sentence query from an untrimmed video.<n>To this end, we creatively explore a brand-new VMR setting termed Open-Set Video Moment Retrieval (OS-VMR)<n>In this paper, we make the first attempt to step toward OS-VMR and propose a novel model textOpenVMR, which first distinguishes ID and OOD queries based on the normalizing flow technology.
• Score: 82.18047702848021
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video Moment Retrieval (VMR) targets to retrieve the specific moment corresponding to a sentence query from an untrimmed video. Although recent works have made remarkable progress in this task, they implicitly are rooted in the closed-set assumption that all the given queries as video-relevant\footnote{In this paper, we treat ``video-relevant query'' as ``in-distribution (ID) query'' and ``video-irrelevant query'' as ``out-of-distribution (OOD) query''.}. Given an OOD query in open-set scenarios, they still utilize it for wrong retrieval, which might lead to irrecoverable losses in high-risk scenarios, \textit{e.g.}, criminal activity detection. To this end, we creatively explore a brand-new VMR setting termed Open-Set Video Moment Retrieval (OS-VMR), where we should not only retrieve the precise moments based on ID query, but also reject OOD queries. In this paper, we make the first attempt to step toward OS-VMR and propose a novel model \textbf{OpenVMR}, which first distinguishes ID and OOD queries based on the normalizing flow technology, and then conducts moment retrieval based on ID queries. Specifically, we first learn the ID distribution by constructing a normalizing flow, and assume the ID query distribution obeys the multi-variate Gaussian distribution. Then, we introduce an uncertainty score to search the ID-OOD separating boundary. After that, we refine the ID-OOD boundary by pulling together ID query features. Besides, video-query matching and frame-query matching are designed for coarse-grained and fine-grained cross-modal interaction, respectively. Finally, a positive-unlabeled learning module is introduced for moment retrieval. Experimental results on three VMR datasets show the effectiveness of our OpenVMR.

Related papers

• VideoSearch-R1: Iterative Video Retrieval and Reasoning via Soft Query Refinement [31.30208652334818]
  VideoSearch-R1 is an agentic framework for iterative video retrieval and reasoning through multi-turn interaction with a video search engine.<n>VideoSearch-R1 achieves state-of-the-art performance across three datasets on Video Moment Retrieval (VCMR)
  arXiv  Detail & Related papers  (2026-07-01T04:59:24Z)
• Rethinking RAG in Long Videos: What to Retrieve and How to Use It? [56.38819694781005]
  We introduce V-RAGBench, a benchmark of $langle$query, evidence chunk, answer$rangle$ triplets that enables faithful, decoupled evaluation of retrieval and generation.<n>We also present CARVE, a method that runs parallel retrievers across configurations and employs chunk-adaptive reranking to identify the winning configuration for each chunk.
  arXiv  Detail & Related papers  (2026-06-11T10:05:49Z)
• Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language [60.91064560080974]
  We propose an efficient approach, SpotVMR, to trim the query-relevant clip.<n>Our proposed SpotVMR can serve as plug-and-play module, which achieves efficiency for state-of-the-art VMR methods.
  arXiv  Detail & Related papers  (2026-05-28T11:42:28Z)
• Beyond Caption-Based Queries for Video Moment Retrieval [60.31221310786333]
  We investigate degradation of VMR methods when trained on caption-based queries but evaluated on search queries.<n>We introduce three benchmarks by modifying the textual queries in three public VMR datasets.<n>Our approach improves performance on search queries by up to 14.82% mAP_m, and up to 21.83% mAP_m on multi-moment search queries.
  arXiv  Detail & Related papers  (2026-03-02T20:06:41Z)
• WeDetect: Fast Open-Vocabulary Object Detection as Retrieval [74.39703419628829]
  Open-vocabulary object detection aims to detect arbitrary classes via text prompts.<n> Methods without cross-modal fusion layers (non-fusion) offer faster inference by treating recognition as a retrieval problem.<n>We develop a model family named WeDetect to achieve state-of-the-art performance across 15 benchmarks with high inference efficiency.
  arXiv  Detail & Related papers  (2025-12-13T12:40:28Z)
• Action tube generation by person query matching for spatio-temporal action detection [0.0]
  Method generates action tubes from the original video without relying on post-processing steps such as IoU-based linking and clip splitting.<n>Our approach applies query-based detection (DETR) to each frame and matches DETR queries to link the same person across frames.<n>Action classes are predicted using the sequence of queries obtained from QMM matching, allowing for variable-length inputs from videos longer than a single clip.
  arXiv  Detail & Related papers  (2025-03-17T09:26:06Z)
• Towards Video Anomaly Retrieval from Video Anomaly Detection: New Benchmarks and Model [70.97446870672069]
  Video anomaly detection (VAD) has been paid increasing attention due to its potential applications. Video Anomaly Retrieval ( VAR) aims to pragmatically retrieve relevant anomalous videos by cross-modalities. We present two benchmarks, UCFCrime-AR and XD-Violence, constructed on top of prevalent anomaly datasets.
  arXiv  Detail & Related papers  (2023-07-24T06:22:37Z)
• Modal-specific Pseudo Query Generation for Video Corpus Moment Retrieval [20.493241098064665]
  Video corpus moment retrieval (VCMR) is the task to retrieve the most relevant video moment from a large video corpus using a natural language query. We propose a self-supervised learning framework: Modal-specific Pseudo Query Generation Network (MPGN) MPGN generates pseudo queries exploiting both visual and textual information from selected temporal moments. We show that MPGN successfully learns to localize the video corpus moment without any explicit annotation.
  arXiv  Detail & Related papers  (2022-10-23T05:05:18Z)
• Temporal Query Networks for Fine-grained Video Understanding [88.9877174286279]
  We cast this into a query-response mechanism, where each query addresses a particular question, and has its own response label set. We evaluate the method extensively on the FineGym and Diving48 benchmarks for fine-grained action classification and surpass the state-of-the-art using only RGB features.
  arXiv  Detail & Related papers  (2021-04-19T17:58:48Z)
• VLANet: Video-Language Alignment Network for Weakly-Supervised Video Moment Retrieval [21.189093631175425]
  Video Moment Retrieval (VMR) is a task to localize the temporal moment in untrimmed video specified by natural language query. This paper explores methods for performing VMR in a weakly-supervised manner (wVMR) The experiments show that the method achieves state-of-the-art performance on Charades-STA and DiDeMo datasets.
  arXiv  Detail & Related papers  (2020-08-24T07:54:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.