HINT: Composed Image Retrieval with Dual-path Compositional Contextualized Network

• URL: http://arxiv.org/abs/2603.26341v1
• Date: Fri, 27 Mar 2026 12:01:40 GMT
• Title: HINT: Composed Image Retrieval with Dual-path Compositional Contextualized Network
• Abstract summary: Composed Image Retrieval (CIR) is a challenging image retrieval paradigm.<n>We propose a dual-patHsItional coNtextualized neTwork (HINT) to perform contextualized encoding and amplify the similarity differences between matching and non-matching samples.<n>Our HINT model achieves optimal performance on all metrics across two CIR benchmark datasets.
• Score: 32.657095507841085
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Composed Image Retrieval (CIR) is a challenging image retrieval paradigm. It aims to retrieve target images from large-scale image databases that are consistent with the modification semantics, based on a multimodal query composed of a reference image and modification text. Although existing methods have made significant progress in cross-modal alignment and feature fusion, a key flaw remains: the neglect of contextual information in discriminating matching samples. However, addressing this limitation is not an easy task due to two challenges: 1) implicit dependencies and 2) the lack of a differential amplification mechanism. To address these challenges, we propose a dual-patH composItional coNtextualized neTwork (HINT), which can perform contextualized encoding and amplify the similarity differences between matching and non-matching samples, thus improving the upper performance of CIR models in complex scenarios. Our HINT model achieves optimal performance on all metrics across two CIR benchmark datasets, demonstrating the superiority of our HINT model. Codes are available at https://github.com/zh-mingyu/HINT.

Related papers

• Do Composed Image Retrieval Benchmarks Require Multimodal Composition? [86.99911649534795]
  We show that strong performance on Composed Image Retrieval benchmarks is assumed to require multimodal composition.<n>Across four widely used CIR benchmarks and eleven Generalist Multimodal Embedding models, a large fraction of queries can be solved using a single modality.<n>High CIR performance can arise from unimodal signals rather than true multimodal composition.
  arXiv  Detail & Related papers  (2026-05-14T12:56:36Z)
• TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrieval [61.77510161631665]
  Composed Image Retrieval (CIR) enables users to retrieve a target image using a multimodal query that consists of a reference image and modification text.<n>We construct two instruction-rich multi-modification datasets, M-FashionIQ and M-CIRR.<n>We also propose TEMA, the first CIR framework designed for multi-modification while also accommodating simple modifications.
  arXiv  Detail & Related papers  (2026-04-23T16:03:12Z)
• HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval [32.85064790572752]
  Composed Image Retrieval (CIR) is a flexible image retrieval paradigm that enables users to accurately locate the target image.<n>This paper proposes a cHrono-synergiA roBust progressIve learning framework for composed image reTrieval (HABIT)<n>Experiments conducted on two standard CIR datasets demonstrate that HABIT significantly outperforms most methods under various noise ratios.
  arXiv  Detail & Related papers  (2026-04-20T10:02:18Z)
• MELT: Improve Composed Image Retrieval via the Modification Frequentation-Rarity Balance Network [34.61012653883818]
  Composed Image Retrieval (CIR) uses a reference image and a modification text as a query to retrieve a target image.<n>CIR methods face two limitations: (1) frequency bias leading to Rare Sample Neglect'', and (2) susceptibility of similarity scores to interference from hard negative samples and noise.<n>We propose the Modification frEquentation-rarity baLance neTwork MELT.
  arXiv  Detail & Related papers  (2026-03-31T05:52:58Z)
• WISER: Wider Search, Deeper Thinking, and Adaptive Fusion for Training-Free Zero-Shot Composed Image Retrieval [36.577766022251446]
  ZS-CIR aims to retrieve target images given a multimodal query without training on triplets annotated.<n>We propose WISER, a training-free framework that unifies T2I and I2I via a "retrieve-verify-refine" pipeline.
  arXiv  Detail & Related papers  (2026-02-26T14:11:10Z)
• SETR: A Two-Stage Semantic-Enhanced Framework for Zero-Shot Composed Image Retrieval [4.230223288110963]
  Zero-shot Composed Image Retrieval (ZS-CIR) aims to retrieve a target image given a reference image and a relative text, without relying on costly triplet annotations.<n>Existing CLIP-based methods face two core challenges: (1) union-based feature fusion indiscriminately aggregates all visual cues, carrying over irrelevant background details that dilute the intended modification, and (2) global cosine similarity from CLIP embeddings lacks the ability to resolve fine-grained semantic relations.
  arXiv  Detail & Related papers  (2025-09-30T09:41:52Z)
• CoLLM: A Large Language Model for Composed Image Retrieval [76.29725148964368]
  Composed Image Retrieval (CIR) is a complex task that aims to retrieve images based on a multimodal query.<n>We present CoLLM, a one-stop framework that generates triplets on-the-fly from image-caption pairs.<n>We leverage Large Language Models (LLMs) to generate joint embeddings of reference images and modification texts.
  arXiv  Detail & Related papers  (2025-03-25T17:59:50Z)
• MMAIF: Multi-task and Multi-degradation All-in-One for Image Fusion with Language Guidance [11.023241681116295]
  We propose a unified framework for multi-task, multi-degradation, and language-guided image fusion.<n>Our framework includes two key components: 1) a practical degradation pipeline that simulates real-world image degradations and generates interactive prompts to guide the model; 2) an all-in-one Transformer (DiT) operating in latent space, which fuses a clean image conditioned on both the degraded inputs and the generated prompts.
  arXiv  Detail & Related papers  (2025-03-19T07:20:02Z)
• Modality and Task Adaptation for Enhanced Zero-shot Composed Image Retrieval [20.612534837883892]
  Zero-Shot Composed Image Retrieval (ZS-CIR) is designed to retrieve target images using bi-modal (image+text) queries.<n>We propose a lightweight post-hoc framework, consisting of two components.<n>Experiments show that, with the incorporation of our proposed components, inversion-based methods achieve significant improvements.
  arXiv  Detail & Related papers  (2024-10-31T08:49:05Z)
• Image2Sentence based Asymmetrical Zero-shot Composed Image Retrieval [92.13664084464514]
  The task of composed image retrieval (CIR) aims to retrieve images based on the query image and the text describing the users' intent. Existing methods have made great progress with the advanced large vision-language (VL) model in CIR task, however, they generally suffer from two main issues: lack of labeled triplets for model training and difficulty of deployment on resource-restricted environments. We propose Image2Sentence based Asymmetric zero-shot composed image retrieval (ISA), which takes advantage of the VL model and only relies on unlabeled images for composition learning.
  arXiv  Detail & Related papers  (2024-03-03T07:58:03Z)
• Sentence-level Prompts Benefit Composed Image Retrieval [69.78119883060006]
  Composed image retrieval (CIR) is the task of retrieving specific images by using a query that involves both a reference image and a relative caption. We propose to leverage pretrained V-L models, e.g., BLIP-2, to generate sentence-level prompts. Our proposed method performs favorably against the state-of-the-art CIR methods on the Fashion-IQ and CIRR datasets.
  arXiv  Detail & Related papers  (2023-10-09T07:31:44Z)
• Data Roaming and Quality Assessment for Composed Image Retrieval [25.452015862927766]
  Composed Image Retrieval (CoIR) involves queries that combine image and text modalities, allowing users to express their intent more effectively. We introduce the Large Scale Composed Image Retrieval (LaSCo) dataset, a new CoIR dataset which is ten times larger than existing ones. We also introduce a new CoIR baseline, the Cross-Attention driven Shift (CASE)
  arXiv  Detail & Related papers  (2023-03-16T16:02:24Z)
• Reference-based Image and Video Super-Resolution via C2-Matching [100.0808130445653]
  We propose C2-Matching, which performs explicit robust matching crossing transformation and resolution. C2-Matching significantly outperforms state of the arts on the standard CUFED5 benchmark. We also extend C2-Matching to Reference-based Video Super-Resolution task, where an image taken in a similar scene serves as the HR reference image.
  arXiv  Detail & Related papers  (2022-12-19T16:15:02Z)
• DDet: Dual-path Dynamic Enhancement Network for Real-World Image Super-Resolution [69.2432352477966]
  Real image super-resolution(Real-SR) focus on the relationship between real-world high-resolution(HR) and low-resolution(LR) image. In this article, we propose a Dual-path Dynamic Enhancement Network(DDet) for Real-SR. Unlike conventional methods which stack up massive convolutional blocks for feature representation, we introduce a content-aware framework to study non-inherently aligned image pair.
  arXiv  Detail & Related papers  (2020-02-25T18:24:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.