CoCo-IR: Contextual Composed Image Retrieval

• URL: http://arxiv.org/abs/2608.05149v1
• Date: Wed, 05 Aug 2026 17:59:57 GMT
• Title: CoCo-IR: Contextual Composed Image Retrieval
• Abstract summary: Contextual Composed Image Retrieval (CoCo-IR) is a novel task that enables users to progressively refine search results through interactions.<n>We propose a new model based on a Large Multimodal Model (LMM) that functions as a context-aware reasoner for CoCo-IR.<n>Our model interprets the entire interaction history to generate Transformable Image Embeddings (TIE) that evolve across turns.
• Score: 53.842921823810364
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Current instruction-based image retrieval systems are powerful but limited to single-turn interactions, failing to capture the iterative nature of complex, real-world visual searches. To overcome this limitation, we introduce Contextual Composed Image Retrieval (CoCo-IR), a novel task that enables users to progressively refine search results through interactions. We address this new task by proposing a new model based on a Large Multimodal Model (LMM) that functions as a context-aware reasoner for CoCo-IR. Our model interprets the entire interaction history to generate Transformable Image Embeddings (TIE) that evolve across turns. To fuel the model training without expensive human annotations, we develop a fully autonomous, scalable data engine that leverages LMMs to generate high-quality contextual retrieval data, and uses model-guided verification to mine challenging hard negatives. Extensive experiments demonstrate that our approach establishes new state-of-the-art performance: We achieve 39.4 mAP@5 on the challenging single-turn benchmark CIRCO; furthermore, on our new CoCo-IR benchmark, our model maintains robust performance with 44.1 R@1 on 4-turn dialogues, dramatically outperforming existing methods (28.2 4-turn R@1) that fail to handle multi-turn context. Project page: https://CoCo-IR.github.io.

Related papers

• FiRE: Enhancing MLLMs with Fine-Grained Context Learning for Complex Image Retrieval [39.372170257570595]
  We propose an automated fine-grained multimodal quintuple dataset construction pipeline and a novel fine-grained multimodal fine-tuning strategy.<n>Our approach separates the fine-tuning process into two distinct stages: (1) fine-grained context reasoning-oriented fine-tuning and (2) fine-grained retrieval-oriented fine-tuning.
  arXiv  Detail & Related papers  (2026-07-30T10:06:11Z)
• Generative Editing in the Joint Vision-Language Space for Zero-Shot Composed Image Retrieval [11.724675700368316]
  Composed Image Retrieval (CIR) enables fine-grained visual search by combining a reference image with a textual modification.<n>We propose Fusion-Diff, a novel generative editing framework with high effectiveness and data efficiency designed for multimodal alignment.
  arXiv  Detail & Related papers  (2025-12-01T13:04:55Z)
• Co-Training Vision Language Models for Remote Sensing Multi-task Learning [68.15604397741753]
  Vision language models (VLMs) have achieved promising results in RS image understanding, grounding, and ultra-high-resolution (UHR) image reasoning.<n>We present RSCoVLM, a simple yet flexible VLM baseline for RS MTL.<n>We propose a unified dynamic-resolution strategy to address the diverse image scales inherent in RS imagery.
  arXiv  Detail & Related papers  (2025-11-26T10:55:07Z)
• DAFM: Dynamic Adaptive Fusion for Multi-Model Collaboration in Composed Image Retrieval [2.330678113289435]
  We propose Dynamic Adaptive Fusion (DAFM) for multi-model collaboration in Composed Image Retrieval (CIR)<n>Our method achieves a Recall@10 of 93.21 and an Rmean of 84.43 on CIRR, and an average Rmean of 67.48 on FashionIQ, surpassing recent strong baselines by up to 4.5%.
  arXiv  Detail & Related papers  (2025-11-07T06:51:10Z)
• CIR-CoT: Towards Interpretable Composed Image Retrieval via End-to-End Chain-of-Thought Reasoning [93.05917922306196]
  Composed Image Retrieval (CIR) aims to find a target image from a reference image and a modification text.<n>CIR-CoT is the first end-to-end retrieval-oriented MLLM designed to integrate explicit Chain-of-Thought (CoT) reasoning.
  arXiv  Detail & Related papers  (2025-10-09T09:41:45Z)
• Generalized Contrastive Learning for Universal Multimodal Retrieval [53.70202081784898]
  Cross-modal retrieval models (e.g., CLIP) show degraded performances with retrieving keys composed of fused image-text modality.<n>This paper proposes Generalized Contrastive Learning (GCL), a novel loss formulation that improves multimodal retrieval performance without the need for new dataset curation.
  arXiv  Detail & Related papers  (2025-09-30T01:25:04Z)
• CoLLM: A Large Language Model for Composed Image Retrieval [76.29725148964368]
  Composed Image Retrieval (CIR) is a complex task that aims to retrieve images based on a multimodal query.<n>We present CoLLM, a one-stop framework that generates triplets on-the-fly from image-caption pairs.<n>We leverage Large Language Models (LLMs) to generate joint embeddings of reference images and modification texts.
  arXiv  Detail & Related papers  (2025-03-25T17:59:50Z)
• Zero-shot Composed Text-Image Retrieval [72.43790281036584]
  We consider the problem of composed image retrieval (CIR) It aims to train a model that can fuse multi-modal information, e.g., text and images, to accurately retrieve images that match the query, extending the user's expression ability.
  arXiv  Detail & Related papers  (2023-06-12T17:56:01Z)
• Data Roaming and Quality Assessment for Composed Image Retrieval [25.452015862927766]
  Composed Image Retrieval (CoIR) involves queries that combine image and text modalities, allowing users to express their intent more effectively. We introduce the Large Scale Composed Image Retrieval (LaSCo) dataset, a new CoIR dataset which is ten times larger than existing ones. We also introduce a new CoIR baseline, the Cross-Attention driven Shift (CASE)
  arXiv  Detail & Related papers  (2023-03-16T16:02:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.