Enhancing Partially Relevant Video Retrieval with Robust Alignment Learning

• URL: http://arxiv.org/abs/2509.01383v1
• Date: Mon, 01 Sep 2025 11:30:43 GMT
• Title: Enhancing Partially Relevant Video Retrieval with Robust Alignment Learning
• Authors: Long Zhang, Peipei Song, Jianfeng Dong, Kun Li, Xun Yang,
• Abstract summary: Partially Relevant Video Retrieval aims to retrieve videos partially relevant to a given query.<n>The core challenge lies in learning robust query-video alignment against spurious semantic correlations.<n>We propose Robust Alignment Learning framework, which explicitly models the uncertainty in data.
• Score: 29.427720184101474
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos partially relevant to a given query. The core challenge lies in learning robust query-video alignment against spurious semantic correlations arising from inherent data uncertainty: 1) query ambiguity, where the query incompletely characterizes the target video and often contains uninformative tokens, and 2) partial video relevance, where abundant query-irrelevant segments introduce contextual noise in cross-modal alignment. Existing methods often focus on enhancing multi-scale clip representations and retrieving the most relevant clip. However, the inherent data uncertainty in PRVR renders them vulnerable to distractor videos with spurious similarities, leading to suboptimal performance. To fill this research gap, we propose Robust Alignment Learning (RAL) framework, which explicitly models the uncertainty in data. Key innovations include: 1) we pioneer probabilistic modeling for PRVR by encoding videos and queries as multivariate Gaussian distributions. This not only quantifies data uncertainty but also enables proxy-level matching to capture the variability in cross-modal correspondences; 2) we consider the heterogeneous informativeness of query words and introduce learnable confidence gates to dynamically weight similarity. As a plug-and-play solution, RAL can be seamlessly integrated into the existing architectures. Extensive experiments across diverse retrieval backbones demonstrate its effectiveness.

Related papers

• Ambiguity-Restrained Text-Video Representation Learning for Partially Relevant Video Retrieval [0.0]
  Partially Relevant Video Retrieval(PRVR) aims to retrieve a video where a specific segment is relevant to a given text query.<n>We point out the inherent ambiguity between text and video content based on their conceptual scope.<n>We propose a framework that incorporates this ambiguity into the model learning process.
  arXiv  Detail & Related papers  (2025-06-09T06:44:45Z)
• Exploiting Inter-Sample Correlation and Intra-Sample Redundancy for Partially Relevant Video Retrieval [5.849812241074385]
  PRVR aims to retrieve the target video that is partially relevant to a text query.<n>Existing methods coarsely align paired videos and text queries to construct the semantic space.<n>We propose a novel PRVR framework to systematically exploit inter-sample correlation and intra-sample redundancy.
  arXiv  Detail & Related papers  (2025-04-28T09:52:46Z)
• QD-VMR: Query Debiasing with Contextual Understanding Enhancement for Video Moment Retrieval [7.313447367245476]
  Video Moment Retrieval (VMR) aims to retrieve relevant moments of an untrimmed video corresponding to the query. We propose a novel model called QD-VMR, a query debiasing model with enhanced contextual understanding.
  arXiv  Detail & Related papers  (2024-08-23T10:56:42Z)
• Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach [56.610806615527885]
  A key challenge in text-video retrieval (TVR) is the information asymmetry between video and text.<n>This paper introduces a data-centric framework to bridge this gap by enriching textual representations to better match the richness of video content.<n>We propose a query selection mechanism that identifies the most relevant and diverse queries, reducing computational cost while improving accuracy.
  arXiv  Detail & Related papers  (2024-08-14T01:24:09Z)
• GMMFormer v2: An Uncertainty-aware Framework for Partially Relevant Video Retrieval [60.70901959953688]
  We present GMMFormer v2, an uncertainty-aware framework for PRVR. For clip modeling, we improve a strong baseline GMMFormer with a novel temporal consolidation module. We propose a novel optimal matching loss for fine-grained text-clip alignment.
  arXiv  Detail & Related papers  (2024-05-22T16:55:31Z)
• Answering from Sure to Uncertain: Uncertainty-Aware Curriculum Learning for Video Question Answering [63.12469700986452]
  We introduce the concept of uncertainty-aware curriculum learning (CL) Here, uncertainty serves as the guiding principle for dynamically adjusting the difficulty. In practice, we seamlessly integrate the VideoQA model into our framework and conduct comprehensive experiments.
  arXiv  Detail & Related papers  (2024-01-03T02:29:34Z)
• Towards Video Anomaly Retrieval from Video Anomaly Detection: New Benchmarks and Model [70.97446870672069]
  Video anomaly detection (VAD) has been paid increasing attention due to its potential applications. Video Anomaly Retrieval ( VAR) aims to pragmatically retrieve relevant anomalous videos by cross-modalities. We present two benchmarks, UCFCrime-AR and XD-Violence, constructed on top of prevalent anomaly datasets.
  arXiv  Detail & Related papers  (2023-07-24T06:22:37Z)
• Query-Dependent Video Representation for Moment Retrieval and Highlight Detection [8.74967598360817]
  Key objective of MR/HD is to localize the moment and estimate clip-wise accordance level, i.e., saliency score, to a given text query. Recent transformer-based models do not fully exploit the information of a given query. We introduce Query-Dependent DETR (QD-DETR), a detection transformer tailored for MR/HD.
  arXiv  Detail & Related papers  (2023-03-24T09:32:50Z)
• Mitigating Representation Bias in Action Recognition: Algorithms and Benchmarks [76.35271072704384]
  Deep learning models perform poorly when applied to videos with rare scenes or objects. We tackle this problem from two different angles: algorithm and dataset. We show that the debiased representation can generalize better when transferred to other datasets and tasks.
  arXiv  Detail & Related papers  (2022-09-20T00:30:35Z)
• Deconfounded Video Moment Retrieval with Causal Intervention [80.90604360072831]
  We tackle the task of video moment retrieval (VMR), which aims to localize a specific moment in a video according to a textual query. Existing methods primarily model the matching relationship between query and moment by complex cross-modal interactions. We propose a causality-inspired VMR framework that builds structural causal model to capture the true effect of query and video content on the prediction.
  arXiv  Detail & Related papers  (2021-06-03T01:33:26Z)
• Reference-Aided Part-Aligned Feature Disentangling for Video Person Re-Identification [18.13546384207381]
  We propose a textbfReference-textbfAided textbfPart-textbfAligned (textbfRAPA) framework to disentangle robust features of different parts. By using both modules, the informative parts of pedestrian in videos are well aligned and more discriminative feature representation is generated.
  arXiv  Detail & Related papers  (2021-03-21T06:53:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.