TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrieval

• URL: http://arxiv.org/abs/2604.21806v2
• Date: Fri, 24 Apr 2026 09:24:52 GMT
• Title: TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrieval
• Abstract summary: Composed Image Retrieval (CIR) enables users to retrieve a target image using a multimodal query that consists of a reference image and modification text.<n>We construct two instruction-rich multi-modification datasets, M-FashionIQ and M-CIRR.<n>We also propose TEMA, the first CIR framework designed for multi-modification while also accommodating simple modifications.
• Score: 61.77510161631665
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Composed Image Retrieval (CIR) is an important image retrieval paradigm that enables users to retrieve a target image using a multimodal query that consists of a reference image and modification text. Although research on CIR has made significant progress, prevailing setups still rely simple modification texts that typically cover only a limited range of salient changes, which induces two limitations highly relevant to practical applications, namely Insufficient Entity Coverage and Clause-Entity Misalignment. In order to address these issues and bring CIR closer to real-world use, we construct two instruction-rich multi-modification datasets, M-FashionIQ and M-CIRR. In addition, we propose TEMA, the Text-oriented Entity Mapping Architecture, which is the first CIR framework designed for multi-modification while also accommodating simple modifications. Extensive experiments on four benchmark datasets demonstrate that TEMA's superiority in both original and multi-modification scenarios, while maintaining an optimal balance between retrieval accuracy and computational efficiency. Our codes and constructed multi-modification dataset (M-FashionIQ and M-CIRR) are available at https://github.com/lee-zixu/ACL26-TEMA/.

Related papers

• HINT: Composed Image Retrieval with Dual-path Compositional Contextualized Network [32.657095507841085]
  Composed Image Retrieval (CIR) is a challenging image retrieval paradigm.<n>We propose a dual-patHsItional coNtextualized neTwork (HINT) to perform contextualized encoding and amplify the similarity differences between matching and non-matching samples.<n>Our HINT model achieves optimal performance on all metrics across two CIR benchmark datasets.
  arXiv  Detail & Related papers  (2026-03-27T12:01:40Z)
• Hierarchical Scheduling for Multi-Vector Image Retrieval [17.023146933530484]
  HiMIR is an efficient scheduling framework for image retrieval.<n>We introduce a novel hierarchical paradigm, employing multiple intermediate granularities for varying image objects to enhance alignment.<n>Our empirical study shows that, HiMIR not only achieves substantial accuracy improvements but also reduces computation by up to 3.5 times over the existing MVR system.
  arXiv  Detail & Related papers  (2025-10-10T03:36:18Z)
• Recurrence Meets Transformers for Universal Multimodal Retrieval [59.92546492752452]
  ReT-2 is a unified retrieval model that supports multimodal queries composed of both images and text.<n>We evaluate ReT-2 on the challenging M2KR and M-BEIR benchmarks across different retrieval configurations.<n>When integrated into retrieval-augmented generation pipelines, ReT-2 also improves downstream performance on Encyclopedic-VQA and InfoSeek datasets.
  arXiv  Detail & Related papers  (2025-09-10T18:00:29Z)
• FineCIR: Explicit Parsing of Fine-Grained Modification Semantics for Composed Image Retrieval [48.63758678240123]
  Composed Image Retrieval (CIR) facilitates image retrieval through a multimodal query consisting of a reference image and modification text.<n>Existing CIR datasets predominantly employ coarse-grained modification text (CoarseMT), which inadequately captures fine-grained retrieval intents.<n>We develop a robust fine-grained CIR data annotation pipeline that minimizes imprecise positive samples and enhances CIR systems' ability to discern modification intents accurately.
  arXiv  Detail & Related papers  (2025-03-27T09:34:21Z)
• CoLLM: A Large Language Model for Composed Image Retrieval [76.29725148964368]
  Composed Image Retrieval (CIR) is a complex task that aims to retrieve images based on a multimodal query.<n>We present CoLLM, a one-stop framework that generates triplets on-the-fly from image-caption pairs.<n>We leverage Large Language Models (LLMs) to generate joint embeddings of reference images and modification texts.
  arXiv  Detail & Related papers  (2025-03-25T17:59:50Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• Compositional Image Retrieval via Instruction-Aware Contrastive Learning [40.54022628032561]
  Composed Image Retrieval (CIR) involves retrieving a target image based on a composed query of an image paired with text that specifies modifications or changes to the visual reference.<n>In practice, due to the scarcity of annotated data in downstream tasks, Zero-Shot CIR (ZS-CIR) is desirable.<n>We propose a novel embedding method utilizing an instruction-tuned Multimodal LLM (MLLM) to generate composed representation.
  arXiv  Detail & Related papers  (2024-12-07T22:46:52Z)
• Large Language Models for Multimodal Deformable Image Registration [50.91473745610945]
  We propose a novel coarse-to-fine MDIR framework,LLM-Morph, for aligning the deep features from different modal medical images. Specifically, we first utilize a CNN encoder to extract deep visual features from cross-modal image pairs, then we use the first adapter to adjust these tokens, and use LoRA in pre-trained LLMs to fine-tune their weights. Third, for the alignment of tokens, we utilize other four adapters to transform the LLM-encoded tokens into multi-scale visual features, generating multi-scale deformation fields and facilitating the coarse-to-fine MDIR task
  arXiv  Detail & Related papers  (2024-08-20T09:58:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.