Uneven Event Modeling for Partially Relevant Video Retrieval

• URL: http://arxiv.org/abs/2506.00891v2
• Date: Tue, 03 Jun 2025 03:11:35 GMT
• Title: Uneven Event Modeling for Partially Relevant Video Retrieval
• Authors: Sa Zhu, Huashan Chen, Wanqian Zhang, Jinchao Zhang, Zexian Yang, Xiaoshuai Hao, Bo Li,
• Abstract summary: We propose an Uneven Event Modeling (UEM) framework for partially relevant video retrieval (PRVR)<n>We first introduce the Progressive-Grouped Video (PGVS) module, to iteratively formulate events in light of both temporal dependencies and semantic similarity between consecutive frames.<n>We also propose the Context-Aware Event Refinement (CAER) module to refine the event representation conditioned the text's cross-attention.
• Score: 17.05658098065075
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given a text query, partially relevant video retrieval (PRVR) aims to retrieve untrimmed videos containing relevant moments, wherein event modeling is crucial for partitioning the video into smaller temporal events that partially correspond to the text. Previous methods typically segment videos into a fixed number of equal-length clips, resulting in ambiguous event boundaries. Additionally, they rely on mean pooling to compute event representations, inevitably introducing undesired misalignment. To address these, we propose an Uneven Event Modeling (UEM) framework for PRVR. We first introduce the Progressive-Grouped Video Segmentation (PGVS) module, to iteratively formulate events in light of both temporal dependencies and semantic similarity between consecutive frames, enabling clear event boundaries. Furthermore, we also propose the Context-Aware Event Refinement (CAER) module to refine the event representation conditioned the text's cross-attention. This enables event representations to focus on the most relevant frames for a given text, facilitating more precise text-video alignment. Extensive experiments demonstrate that our method achieves state-of-the-art performance on two PRVR benchmarks. Code is available at https://github.com/Sasa77777779/UEM.git.

Related papers

• Mind the Time: Temporally-Controlled Multi-Event Video Generation [65.05423863685866]
  We present MinT, a multi-event video generator with temporal control.<n>Our key insight is to bind each event to a specific period in the generated video, which allows the model to focus on one event at a time.<n>For the first time in the literature, our model offers control over the timing of events in generated videos.
  arXiv  Detail & Related papers  (2024-12-06T18:52:20Z)
• Grounding Partially-Defined Events in Multimodal Data [61.0063273919745]
  We introduce a multimodal formulation for partially-defined events and cast the extraction of these events as a three-stage span retrieval task. We propose a benchmark for this task, MultiVENT-G, that consists of 14.5 hours of densely annotated current event videos and 1,168 text documents, containing 22.8K labeled event-centric entities. Results illustrate the challenges that abstract event understanding poses and demonstrates promise in event-centric video-language systems.
  arXiv  Detail & Related papers  (2024-10-07T17:59:48Z)
• Training-free Video Temporal Grounding using Large-scale Pre-trained Models [41.71055776623368]
  Video temporal grounding aims to identify video segments within untrimmed videos that are most relevant to a given natural language query. Existing video temporal localization models rely on specific datasets for training and have high data collection costs. We propose a Training-Free Video Temporal Grounding approach that leverages the ability of pre-trained large models.
  arXiv  Detail & Related papers  (2024-08-29T02:25:12Z)
• EA-VTR: Event-Aware Video-Text Retrieval [97.30850809266725]
  Event-Aware Video-Text Retrieval model achieves powerful video-text retrieval ability through superior video event awareness. EA-VTR can efficiently encode frame-level and video-level visual representations simultaneously, enabling detailed event content and complex event temporal cross-modal alignment.
  arXiv  Detail & Related papers  (2024-07-10T09:09:58Z)
• Event-aware Video Corpus Moment Retrieval [79.48249428428802]
  Video Corpus Moment Retrieval (VCMR) is a practical video retrieval task focused on identifying a specific moment within a vast corpus of untrimmed videos. Existing methods for VCMR typically rely on frame-aware video retrieval, calculating similarities between the query and video frames to rank videos. We propose EventFormer, a model that explicitly utilizes events within videos as fundamental units for video retrieval.
  arXiv  Detail & Related papers  (2024-02-21T06:55:20Z)
• GMMFormer: Gaussian-Mixture-Model Based Transformer for Efficient Partially Relevant Video Retrieval [59.47258928867802]
  Given a text query, partially relevant video retrieval (PRVR) seeks to find videos containing pertinent moments in a database. This paper proposes GMMFormer, a Gaussian-Mixture-Model based Transformer which models clip representations implicitly. Experiments on three large-scale video datasets demonstrate the superiority and efficiency of GMMFormer.
  arXiv  Detail & Related papers  (2023-10-08T15:04:50Z)
• Local Compressed Video Stream Learning for Generic Event Boundary Detection [25.37983456118522]
  Event boundary detection aims to localize the generic, taxonomy-free event boundaries that segment videos into chunks. Existing methods typically require video frames to be decoded before feeding into the network. We propose a novel event boundary detection method that is fully end-to-end leveraging rich information in the compressed domain.
  arXiv  Detail & Related papers  (2023-09-27T06:49:40Z)
• Learning Grounded Vision-Language Representation for Versatile Understanding in Untrimmed Videos [57.830865926459914]
  We propose a vision-language learning framework for untrimmed videos, which automatically detects informative events. Instead of coarse-level video-language alignments, we present two dual pretext tasks to encourage fine-grained segment-level alignments. Our framework is easily to tasks covering visually-grounded language understanding and generation.
  arXiv  Detail & Related papers  (2023-03-11T11:00:16Z)
• Leveraging the Video-level Semantic Consistency of Event for Audio-visual Event Localization [8.530561069113716]
  We propose a novel video-level semantic consistency guidance network for the AVE localization task. It consists of two components: a cross-modal event representation extractor and an intra-modal semantic consistency enhancer. We perform extensive experiments on the public AVE dataset and outperform the state-of-the-art methods in both fully- and weakly-supervised settings.
  arXiv  Detail & Related papers  (2022-10-11T08:15:57Z)
• Team RUC_AIM3 Technical Report at Activitynet 2020 Task 2: Exploring Sequential Events Detection for Dense Video Captioning [63.91369308085091]
  We propose a novel and simple model for event sequence generation and explore temporal relationships of the event sequence in the video. The proposed model omits inefficient two-stage proposal generation and directly generates event boundaries conditioned on bi-directional temporal dependency in one pass. The overall system achieves state-of-the-art performance on the dense-captioning events in video task with 9.894 METEOR score on the challenge testing set.
  arXiv  Detail & Related papers  (2020-06-14T13:21:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.