UniCVR: From Alignment to Reranking for Unified Zero-Shot Composed Visual Retrieval

• URL: http://arxiv.org/abs/2604.20318v1
• Date: Wed, 22 Apr 2026 08:16:50 GMT
• Title: UniCVR: From Alignment to Reranking for Unified Zero-Shot Composed Visual Retrieval
• Abstract summary: We propose UniCVR, the first unified zero-shot composed visual retrieval framework.<n>UniCVR jointly addresses all three tasks without any task-specific human-annotated data.<n>In Stage I, we train the MLLM as a compositional query embedder via contrastive learning on a curated dataset of approximately 3.5M samples.<n>In Stage II, we introduce an MLLM-guided dual-level reranking mechanism that applies adaptive budgeted subset scoring to a small number of top-ranked candidates.
• Score: 87.12447641908143
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Composed image retrieval, multi-turn composed image retrieval, and composed video retrieval all share a common paradigm: composing the reference visual with modification text to retrieve the desired target. Despite this shared structure, the three tasks have been studied in isolation, with no prior work proposing a unified framework, let alone a zero-shot solution. In this paper, we propose UniCVR, the first unified zero-shot composed visual retrieval framework that jointly addresses all three tasks without any task-specific human-annotated data. UniCVR strategically combines two complementary strengths: Multimodal Large Language Models (MLLMs) for compositional query understanding and Vision-Language Pre-trained (VLP) models for structured visual retrieval. Concretely, UniCVR operates in two stages. In Stage I, we train the MLLM as a compositional query embedder via contrastive learning on a curated multi-source dataset of approximately 3.5M samples, bridging the heterogeneous embedding spaces between the MLLM and the frozen VLP gallery encoder. A cluster-based hard negative sampling strategy is proposed to strengthen contrastive supervision. In Stage II, we introduce an MLLM-guided dual-level reranking mechanism that applies adaptive budgeted subset scoring to a small number of top-ranked candidates, and then exploits the resulting relevance signals through a dual-level re-scoring scheme, producing more accurate final rankings with minimal computational overhead. Extensive experiments across five benchmarks covering all three tasks demonstrate that UniCVR achieves cutting-edge performance, validating its effectiveness and generalizability. Our data and code will be released upon acceptance.

Related papers

• Bootstrapping MLLM for Weakly-Supervised Class-Agnostic Object Counting [59.37613121962146]
  We propose WS-COC, the first MLLM-driven weakly-supervised framework for class-agnostic object counting.<n> WS-COC matches or even surpasses many state-of-art fully-supervised methods while significantly reducing annotation costs.
  arXiv  Detail & Related papers  (2026-02-13T09:58:35Z)
• Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by Chain-of-Thought Reasoning [47.868429337792314]
  Fine-R1 is an R1-style MLLM tailored for Fine-Grained Visual Recognition.<n>With only 4-shot training, Fine-R1 outperforms existing general MLLMs, reasoning MLLMs, and even contrastive CLIP models.
  arXiv  Detail & Related papers  (2026-02-07T16:16:51Z)
• CSMCIR: CoT-Enhanced Symmetric Alignment with Memory Bank for Composed Image Retrieval [54.15776146365823]
  Composed Image Retrieval (CIR) enables users to search for target images using both a reference image and manipulation text.<n>We propose CSMCIR, a unified representation framework that achieves efficient query-target alignment through three synergistic components.
  arXiv  Detail & Related papers  (2026-01-07T09:21:38Z)
• Exploring a Unified Vision-Centric Contrastive Alternatives on Multi-Modal Web Documents [99.62178668680578]
  We propose Vision-Centric Contrastive Learning (VC2L), a unified framework that models text, images, and their combinations using a single vision transformer.<n> VC2L operates entirely in pixel space by rendering all inputs, whether textual, visual, or combined, as images.<n>To capture complex cross-modal relationships in web documents, VC2L employs a snippet-level contrastive learning objective that aligns consecutive multimodal segments.
  arXiv  Detail & Related papers  (2025-10-21T14:59:29Z)
• Automatic Synthesis of High-Quality Triplet Data for Composed Image Retrieval [25.071276825826047]
  Composed Image Retrieval (CIR) aims to retrieve target images using multimodal (image+text) queries.<n>We propose a scalable pipeline for automatic triplet generation, along with a fully synthetic dataset named Composed Image Retrieval on High-quality Synthetic Triplets (CIRHS)
  arXiv  Detail & Related papers  (2025-07-08T13:24:05Z)
• CoLLM: A Large Language Model for Composed Image Retrieval [76.29725148964368]
  Composed Image Retrieval (CIR) is a complex task that aims to retrieve images based on a multimodal query.<n>We present CoLLM, a one-stop framework that generates triplets on-the-fly from image-caption pairs.<n>We leverage Large Language Models (LLMs) to generate joint embeddings of reference images and modification texts.
  arXiv  Detail & Related papers  (2025-03-25T17:59:50Z)
• REF-VLM: Triplet-Based Referring Paradigm for Unified Visual Decoding [36.376220619032225]
  REF-VLM is an end-to-end framework for unified training of various visual decoding tasks.<n>We construct a large-scale multi-task dataset containing over 100 million multimodal dialogue samples.<n> REF-VLM outperforms other MLLMs across a variety of standard benchmarks.
  arXiv  Detail & Related papers  (2025-03-10T14:59:14Z)
• Advancing Single and Multi-task Text Classification through Large Language Model Fine-tuning [29.782832197148487]
  Large language models (LLMs) have been widely used for text classification tasks.<n>This study employed a diverse range of models and methods, varying in size and architecture, and including both fine-tuned and pre-trained approaches.<n>We first assessed the performances of these LLMs on the 20 Newsgroups (20NG) and datasets, comparing them to encoder-only RoBERTa models.<n>We explored the multi-task capabilities of both model types by combining multiple classification tasks, including intent detection and slot-filling, into a single model using data from both datasets.
  arXiv  Detail & Related papers  (2024-12-11T18:06:44Z)
• Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs [61.143381152739046]
  We introduce Cambrian-1, a family of multimodal LLMs (MLLMs) designed with a vision-centric approach.<n>Our study uses LLMs and visual instruction tuning as an interface to evaluate various visual representations.<n>We provide model weights, code, supporting tools, datasets, and detailed instruction-tuning and evaluation recipes.
  arXiv  Detail & Related papers  (2024-06-24T17:59:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.