IMAGINE: Adaptive Schema-Imagery Enhanced Composition for Composed Video Retrieval

• URL: http://arxiv.org/abs/2606.08144v1
• Date: Sat, 06 Jun 2026 12:46:19 GMT
• Title: IMAGINE: Adaptive Schema-Imagery Enhanced Composition for Composed Video Retrieval
• Abstract summary: Composed Video Retrieval (CVR) is designed to retrieve a target video that matches a reference video modified by a modification text.<n>IMAGINE materializes implicit semantics (termed schema imagery) via dynamic multimodal prototypes.<n>IMAGINE achieves state-of-the-art performance in both CVR and Composed Image Retrieval (CIR) across three widely used benchmarks.
• Score: 25.497247372090758
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Composed Video Retrieval (CVR) is designed to retrieve a target video that matches a reference video modified by a modification text. While existing methods explore cross-modal correspondences, they often assume modified objects appear directly in videos. However, modification texts frequently describe concepts not explicitly presented but implicitly expressed through semantically related visual cues (e.g., "cake" implying "birthday party"). Current approaches typically rely on aligning explicit feature representations within the concrete space, neglecting critical latent associations. To address this, we propose an adaptIve scheMa-ImAGery enhanced composItional NEtwork (IMAGINE). Unlike standard explicit matching, IMAGINE materializes implicit semantics (termed schema imagery) via dynamic multimodal prototypes. These prototypes capture shared latent concepts to adaptively modulate visual features, effectively injecting implicit guidance into the retrieval process. By bridging the gap between explicit visual contents and implicit retrieval intentions, IMAGINE achieves state-of-the-art performance in both CVR and Composed Image Retrieval (CIR) across three widely used benchmarks.

Related papers

• Motion-Guided Semantic Alignment with Negative Prompts for Zero-Shot Video Action Recognition [12.523217686212197]
  We present a novel framework that enhances CLIP with disentangled embeddings and semantic-guided interaction.<n>Experiments on standard benchmarks demonstrate that our method consistently outperforms prior CLIP-based approaches.
  arXiv  Detail & Related papers  (2026-04-18T16:34:29Z)
• MCoT-MVS: Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning for Composed Image Retrieval [23.52156477012636]
  Composed Image Retrieval (CIR) aims to retrieve target images based on a reference image and modified texts.<n>We propose a novel Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning (MCoT-MVS) for CIR.
  arXiv  Detail & Related papers  (2026-03-18T04:49:19Z)
• Novel Semantic Prompting for Zero-Shot Action Recognition [0.0]
  Zero-shot action recognition relies on transferring knowledge from vision-language models to unseen actions using semantic descriptions.<n>We introduce SP-CLIP, a lightweight framework that augments frozen vision-language models with structured semantic prompts describing actions at multiple levels of abstraction.
  arXiv  Detail & Related papers  (2026-03-09T12:07:55Z)
• Fine-Grained Zero-Shot Composed Image Retrieval with Complementary Visual-Semantic Integration [64.12127577975696]
  Zero-shot composed image retrieval (ZS-CIR) is a rapidly growing area with significant practical applications.<n>Existing ZS-CIR methods often struggle to capture fine-grained changes and integrate visual and semantic information effectively.<n>We propose a novel Fine-Grained Zero-Shot Composed Image Retrieval method with Complementary Visual-Semantic Integration.
  arXiv  Detail & Related papers  (2026-01-20T15:17:14Z)
• OVMR: Open-Vocabulary Recognition with Multi-Modal References [96.21248144937627]
  Existing works have proposed different methods to embed category cues into the model, eg, through few-shot fine-tuning. This paper tackles open-vocabulary recognition from a different perspective by referring to multi-modal clues composed of textual descriptions and exemplar images. The proposed OVMR is a plug-and-play module, and works well with exemplar images randomly crawled from the Internet.
  arXiv  Detail & Related papers  (2024-06-07T06:45:28Z)
• Reasoning over the Behaviour of Objects in Video-Clips for Adverb-Type Recognition [54.938128496934695]
  We propose a new framework that reasons over object-behaviours extracted from raw-video-clips to recognize the clip's corresponding adverb-types. Specifically, we propose a novel pipeline that extracts human-interpretable object-behaviour-facts from raw video clips. We release two new datasets of object-behaviour-facts extracted from raw video clips.
  arXiv  Detail & Related papers  (2023-07-09T09:04:26Z)
• Towards Generalisable Video Moment Retrieval: Visual-Dynamic Injection to Image-Text Pre-Training [70.83385449872495]
  The correlation between the vision and text is essential for video moment retrieval (VMR) Existing methods rely on separate pre-training feature extractors for visual and textual understanding. We propose a generic method, referred to as Visual-Dynamic Injection (VDI), to empower the model's understanding of video moments.
  arXiv  Detail & Related papers  (2023-02-28T19:29:05Z)
• Two-stage Visual Cues Enhancement Network for Referring Image Segmentation [89.49412325699537]
  Referring Image (RIS) aims at segmenting the target object from an image referred by one given natural language expression. In this paper, we tackle this problem by devising a Two-stage Visual cues enhancement Network (TV-Net) Through the two-stage enhancement, our proposed TV-Net enjoys better performances in learning fine-grained matching behaviors between the natural language expression and image.
  arXiv  Detail & Related papers  (2021-10-09T02:53:39Z)
• StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval [119.03470556503942]
  Crossmodal matching problem is typically solved by learning a joint embedding space where semantic content shared between photo and sketch modalities are preserved. An effective model needs to explicitly account for this style diversity, crucially, to unseen user styles. Our model can not only disentangle the cross-modal shared semantic content, but can adapt the disentanglement to any unseen user style as well, making the model truly agnostic.
  arXiv  Detail & Related papers  (2021-03-29T15:44:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.