MELT: Improve Composed Image Retrieval via the Modification Frequentation-Rarity Balance Network

• URL: http://arxiv.org/abs/2603.29291v1
• Date: Tue, 31 Mar 2026 05:52:58 GMT
• Title: MELT: Improve Composed Image Retrieval via the Modification Frequentation-Rarity Balance Network
• Abstract summary: Composed Image Retrieval (CIR) uses a reference image and a modification text as a query to retrieve a target image.<n>CIR methods face two limitations: (1) frequency bias leading to Rare Sample Neglect'', and (2) susceptibility of similarity scores to interference from hard negative samples and noise.<n>We propose the Modification frEquentation-rarity baLance neTwork MELT.
• Score: 34.61012653883818
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Composed Image Retrieval (CIR) uses a reference image and a modification text as a query to retrieve a target image satisfying the requirement of ``modifying the reference image according to the text instructions''. However, existing CIR methods face two limitations: (1) frequency bias leading to ``Rare Sample Neglect'', and (2) susceptibility of similarity scores to interference from hard negative samples and noise. To address these limitations, we confront two key challenges: asymmetric rare semantic localization and robust similarity estimation under hard negative samples. To solve these challenges, we propose the Modification frEquentation-rarity baLance neTwork MELT. MELT assigns increased attention to rare modification semantics in multimodal contexts while applying diffusion-based denoising to hard negative samples with high similarity scores, enhancing multimodal fusion and matching. Extensive experiments on two CIR benchmarks validate the superior performance of MELT. Codes are available at https://github.com/luckylittlezhi/MELT.

Related papers

• INTENT: Invariance and Discrimination-aware Noise Mitigation for Robust Composed Image Retrieval [33.738463561724444]
  We argue that noise in Composed Image Retrieval (CIR) can be categorized into two types: cross-modal correspondence noise and modality-inherent noise.<n>The latter arises from mismatches across modalities, whereas the latter originates from intra-modal background interference or visual factors irrelevant to the coarse-grained modification annotations.<n>We propose the Invariance and discrimiNaTion-awarE Noise neTwork (INTENT) to handle the two-aspect noise.
  arXiv  Detail & Related papers  (2026-04-20T10:19:07Z)
• HINT: Composed Image Retrieval with Dual-path Compositional Contextualized Network [32.657095507841085]
  Composed Image Retrieval (CIR) is a challenging image retrieval paradigm.<n>We propose a dual-patHsItional coNtextualized neTwork (HINT) to perform contextualized encoding and amplify the similarity differences between matching and non-matching samples.<n>Our HINT model achieves optimal performance on all metrics across two CIR benchmark datasets.
  arXiv  Detail & Related papers  (2026-03-27T12:01:40Z)
• NCL-CIR: Noise-aware Contrastive Learning for Composed Image Retrieval [16.460121977322224]
  Composed Image Retrieval (CIR) seeks to find a target image using a multi-modal query, which combines an image with modification text to pinpoint the target.<n> pairs are often partially or completely mismatched due to issues like inaccurate modification texts, low-quality target images, and annotation errors.<n>We propose the Noise-aware Contrastive Learning for CIR (NCL-CIR) comprising two key components: the Weight Compensation Block (WCB) and the Noise-pair Filter Block (NFB).
  arXiv  Detail & Related papers  (2025-04-06T03:27:23Z)
• CoLLM: A Large Language Model for Composed Image Retrieval [76.29725148964368]
  Composed Image Retrieval (CIR) is a complex task that aims to retrieve images based on a multimodal query.<n>We present CoLLM, a one-stop framework that generates triplets on-the-fly from image-caption pairs.<n>We leverage Large Language Models (LLMs) to generate joint embeddings of reference images and modification texts.
  arXiv  Detail & Related papers  (2025-03-25T17:59:50Z)
• Retinex-RAWMamba: Bridging Demosaicing and Denoising for Low-Light RAW Image Enhancement [71.13353154514418]
  Low-light image enhancement, particularly in cross-domain tasks such as mapping from the raw domain to the sRGB domain, remains a significant challenge.<n>We propose a novel Mamba-based method customized for low light RAW images, called RAWMamba, to effectively handle raw images with different CFAs.<n>By bridging demosaicing and denoising, better enhancement for low light RAW images is achieved.
  arXiv  Detail & Related papers  (2024-09-11T06:12:03Z)
• Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2) [62.44395685571094]
  We introduce T2IScoreScore, a curated set of semantic error graphs containing a prompt and a set of increasingly erroneous images. These allow us to rigorously judge whether a given prompt faithfulness metric can correctly order images with respect to their objective error count. We find that the state-of-the-art VLM-based metrics fail to significantly outperform simple (and supposedly worse) feature-based metrics like CLIPScore.
  arXiv  Detail & Related papers  (2024-04-05T17:57:16Z)
• Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation [15.411325887412413]
  This paper proposes a novel model named "Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment" (FSA-CDM) FSA-CDM introduces contrastive positive/negative samples into the diffusion model to boost performance for markup-to-image generation. Experiments are conducted on four benchmark datasets from different domains.
  arXiv  Detail & Related papers  (2023-08-02T13:43:03Z)
• Boosting Few-shot Fine-grained Recognition with Background Suppression and Foreground Alignment [53.401889855278704]
  Few-shot fine-grained recognition (FS-FGR) aims to recognize novel fine-grained categories with the help of limited available samples. We propose a two-stage background suppression and foreground alignment framework, which is composed of a background activation suppression (BAS) module, a foreground object alignment (FOA) module, and a local to local (L2L) similarity metric. Experiments conducted on multiple popular fine-grained benchmarks demonstrate that our method outperforms the existing state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2022-10-04T07:54:40Z)
• MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution [74.24676600271253]
  We propose the MASA network for RefSR, where two novel modules are designed to address these problems. The proposed Match & Extraction Module significantly reduces the computational cost by a coarse-to-fine correspondence matching scheme. The Spatial Adaptation Module learns the difference of distribution between the LR and Ref images, and remaps the distribution of Ref features to that of LR features in a spatially adaptive way.
  arXiv  Detail & Related papers  (2021-06-04T07:15:32Z)
• Robust Reference-based Super-Resolution via C2-Matching [77.51610726936657]
  Super-Resolution (Ref-SR) has recently emerged as a promising paradigm to enhance a low-resolution (LR) input image by introducing an additional high-resolution (HR) reference image. Existing Ref-SR methods mostly rely on implicit correspondence matching to borrow HR textures from reference images to compensate for the information loss in input images. We propose C2-Matching, which produces explicit robust matching crossing transformation and resolution.
  arXiv  Detail & Related papers  (2021-06-03T16:40:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.