HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrieval

• URL: http://arxiv.org/abs/2512.02792v1
• Date: Tue, 02 Dec 2025 14:10:16 GMT
• Title: HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrieval
• Authors: Zhiwei Chen, Yupeng Hu, Zixu Li, Zhiheng Fu, Haokun Wen, Weili Guan,
• Abstract summary: We propose a novel Composed Video Retrieval (CVR) framework, namely the Hierarchical Uncertainty-aware Disambiguation network (HUD)<n>HUD is the first framework that leverages the disparity in information density between video and text to enhance multi-modal query understanding.<n>Our proposed HUD is also applicable to the Composed Image Retrieval (CIR) task and achieves state-of-the-art performance across three benchmark datasets for both CVR and CIR tasks.
• Score: 39.457158192955106
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Composed Video Retrieval (CVR) is a challenging video retrieval task that utilizes multi-modal queries, consisting of a reference video and modification text, to retrieve the desired target video. The core of this task lies in understanding the multi-modal composed query and achieving accurate composed feature learning. Within multi-modal queries, the video modality typically carries richer semantic content compared to the textual modality. However, previous works have largely overlooked the disparity in information density between these two modalities. This limitation can lead to two critical issues: 1) modification subject referring ambiguity and 2) limited detailed semantic focus, both of which degrade the performance of CVR models. To address the aforementioned issues, we propose a novel CVR framework, namely the Hierarchical Uncertainty-aware Disambiguation network (HUD). HUD is the first framework that leverages the disparity in information density between video and text to enhance multi-modal query understanding. It comprises three key components: (a) Holistic Pronoun Disambiguation, (b) Atomistic Uncertainty Modeling, and (c) Holistic-to-Atomistic Alignment. By exploiting overlapping semantics through holistic cross-modal interaction and fine-grained semantic alignment via atomistic-level cross-modal interaction, HUD enables effective object disambiguation and enhances the focus on detailed semantics, thereby achieving precise composed feature learning. Moreover, our proposed HUD is also applicable to the Composed Image Retrieval (CIR) task and achieves state-of-the-art performance across three benchmark datasets for both CVR and CIR tasks. The codes are available on https://zivchen-ty.github.io/HUD.github.io/.

Related papers

• GranAlign: Granularity-Aware Alignment Framework for Zero-Shot Video Moment Retrieval [12.668753075288308]
  Zero-shot video moment retrieval (ZVMR) is the task of localizing a temporal moment within an untrimmed video using a natural language query without relying on task-specific training data.<n>Previous studies in ZVMR have attempted to achieve alignment by leveraging high-quality pre-trained knowledge that represents video and language in a joint space.<n>We propose a training-free framework, called Granularity-Aware Alignment (GranAlign), that bridges this gap between coarse and fine semantic representations.
  arXiv  Detail & Related papers  (2026-01-02T06:04:58Z)
• Qwen3-VL Technical Report [153.3964813640593]
  Qwen3-VL is the most capable vision-language model to date, achieving superior performance across a broad range of multimodal benchmarks.<n>It supports interleaved contexts of up to 256K tokens, seamlessly integrating text, images, and video.<n>Qwen3-VL delivers three core pillars: (i) markedly stronger pure-text understanding, surpassing comparable text-only backbones in several cases; (ii) robust long-token comprehension with a native 256K-token window for both text and interleaved multimodal inputs; and (iii) advanced multimodal reasoning across single-image, multi-image, and video tasks
  arXiv  Detail & Related papers  (2025-11-26T17:59:08Z)
• Feature Hallucination for Self-supervised Action Recognition [37.20267786858476]
  We propose a deep translational action recognition framework that enhances recognition accuracy by jointly predicting action concepts and auxiliary features from RGB video frames.<n>Our framework achieves state-of-the-art performance on multiple benchmarks, including Kinetics-400, Kinetics-600, and Something-Something V2, demonstrating its effectiveness in capturing fine-grained action dynamics.
  arXiv  Detail & Related papers  (2025-06-25T11:50:23Z)
• Video-Level Language-Driven Video-Based Visible-Infrared Person Re-Identification [47.40091830500585]
  Video-based Visible-basedInfrared Person Re-Identification (VVIReID) aims to match pedestrian sequences across modalities by extracting modality-in sequence-level features.<n>A framework, video-level language-driven VVI-ReID (VLD), consists of two core modules: inmodality language (IMLP) and spatialtemporal aggregation.
  arXiv  Detail & Related papers  (2025-06-03T04:49:08Z)
• Cognitive Disentanglement for Referring Multi-Object Tracking [28.325814292139686]
  We propose a Cognitive Disentanglement for Referring Multi-Object Tracking (CDRMT) framework.<n>CDRMT adapts the "what" and "where" pathways from the human visual processing system to RMOT tasks.<n>Experiments on different benchmark datasets demonstrate that CDRMT achieves substantial improvements over state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-14T15:21:54Z)
• Object Segmentation by Mining Cross-Modal Semantics [68.88086621181628]
  We propose a novel approach by mining the Cross-Modal Semantics to guide the fusion and decoding of multimodal features. Specifically, we propose a novel network, termed XMSNet, consisting of (1) all-round attentive fusion (AF), (2) coarse-to-fine decoder (CFD), and (3) cross-layer self-supervision.
  arXiv  Detail & Related papers  (2023-05-17T14:30:11Z)
• Boosting Video-Text Retrieval with Explicit High-Level Semantics [115.66219386097295]
  We propose a novel visual-linguistic aligning model named HiSE for VTR. It improves the cross-modal representation by incorporating explicit high-level semantics. Our method achieves the superior performance over state-of-the-art methods on three benchmark datasets.
  arXiv  Detail & Related papers  (2022-08-08T15:39:54Z)
• Dual-path CNN with Max Gated block for Text-Based Person Re-identification [6.1534388046236765]
  A novel Dual-path CNN with Max Gated block (DCMG) is proposed to extract discriminative word embeddings. The framework is based on two deep residual CNNs jointly optimized with cross-modal projection matching. Our approach achieves the rank-1 score of 55.81% and outperforms the state-of-the-art method by 1.3%.
  arXiv  Detail & Related papers  (2020-09-20T03:33:29Z)
• Tasks Integrated Networks: Joint Detection and Retrieval for Image Search [99.49021025124405]
  In many real-world searching scenarios (e.g., video surveillance), the objects are seldom accurately detected or annotated. We first introduce an end-to-end Integrated Net (I-Net), which has three merits. We further propose an improved I-Net, called DC-I-Net, which makes two new contributions.
  arXiv  Detail & Related papers  (2020-09-03T03:57:50Z)
• Dense-Caption Matching and Frame-Selection Gating for Temporal Localization in VideoQA [96.10612095576333]
  We propose a video question answering model which effectively integrates multi-modal input sources and finds the temporally relevant information to answer questions. Our model is also comprised of dual-level attention (word/object and frame level), multi-head self-cross-integration for different sources (video and dense captions), and which pass more relevant information to gates. We evaluate our model on the challenging TVQA dataset, where each of our model components provides significant gains, and our overall model outperforms the state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2020-05-13T16:35:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.