Resolving Ambiguity in Composed Image Retrieval via Calibrated Interaction

• URL: http://arxiv.org/abs/2605.24634v3
• Date: Sun, 31 May 2026 03:56:14 GMT
• Title: Resolving Ambiguity in Composed Image Retrieval via Calibrated Interaction
• Abstract summary: Composed image retrieval (CIR) searches a corpus with a reference image and a text describing how to modify it.<n>We introduce AmbiCIR, a benchmark and human-validated user simulator that revives the dormant auxiliary and dialogue annotations of CIRR.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Composed image retrieval (CIR) searches a corpus with a reference image and a text describing how to modify it. Despite rapid progress from triplet-trained compositors to zero-shot and generative methods, essentially all systems share one assumption: that a query maps to a single target, scored by Recall@K against one annotation. We argue this is fundamentally at odds with the task. A query such as make it more formal does not name an image but a region of the corpus, and which member the user intends is genuinely underdetermined. This underspecification is the root of the well-known false-negative problem and leaves current models unable to tell a precise query from an ambiguous one. We reframe CIR as calibrated intent resolution under uncertainty: a retriever is wrapped in a conformal prediction layer that returns a candidate set with a coverage guarantee and whose size is a principled measure of ambiguity; when the set is large, an expected-information-gain policy asks the single most useful clarifying question, drawn from interpretable ambiguity axes, and the set contracts. We introduce AmbiCIR, a benchmark and human-validated user simulator that revive the dormant auxiliary and dialogue annotations of CIRR and extend the multiple-positive setting of CIRCO. Across open-domain and fashion benchmarks our method matches single-turn state of the art, confirming calibrated resolution is cost-free on precise queries, while reaching the intended target in a fraction of the interaction budget required by naive conversational baselines, and it is the first to report valid coverage and calibration for the task.

Related papers

• MMAgent-R$^2$: Learning to Rerank and Reject for Agentic mRAG [50.46218163777298]
  Knowledge-based Visual Question Answering (KB-VQA) requires models to retrieve visual entities matching the query image from large-scale encyclopedic knowledge bases and answer related questions.<n>We propose MMAgent-R$2$, an agentic mRAG framework that integrates visual reranking and active rejection as its internal verification mechanism.
  arXiv  Detail & Related papers  (2026-07-08T13:15:39Z)
• REAL-OW: Rehearsal-free Open World Object Detection with Low-Rank Adaptation and Dual-Stage Objectness Modeling [56.888597798431704]
  Open-World Object Detection (OWOD) requires detectors to identify previously unseen objects as unknown and incrementally incorporate them into the set of known categories.<n>Existing frameworks rely heavily on exemplar replay to mitigate catastrophic forgetting, but in some real applications, storing raw data conflicts with data access restrictions and leads to data exposure risks.<n>We propose REAL-OW, a novel rehearsal-free framework that decouples incremental knowledge through a collaborative adapter architecture.
  arXiv  Detail & Related papers  (2026-07-03T06:38:54Z)
• Thinking Before Retrieving: Robust Zero-Shot Composed Image Retrieval via Strategic Planning and Self-Criticism [43.95253902432504]
  We introduce PEC-CIR, a training-free framework that structures query construction as a multi-stage reasoning pipeline.<n>PEC-CIR reduces the propagation of generative errors by explicitly evaluating candidate queries before retrieval.
  arXiv  Detail & Related papers  (2026-06-30T07:04:05Z)
• Show, Don't Ask: Generative Visual Disambiguation for Composed Image Retrieval with Turn-Valid Coverage [0.0]
  Composed image retrieval (CIR) uses a reference image and a text modification to search for a target image.<n>We propose CLARA, a clarification framework that resolves ambiguity by showing users a small panel of visual alternatives.
  arXiv  Detail & Related papers  (2026-06-17T12:13:57Z)
• A Sanity Check on Composed Image Retrieval [91.95275287747499]
  Composed Image Retrieval (CIR) aims to retrieve a target image based on a query composed of a reference image, and a relative caption that specifies the desired modification.<n>We introduce FISD, a Fully-Informed Semantically-Diverse benchmark, which employs generative models to precisely control the variables of reference-target image pairs.<n>We propose an automatic multi-round agentic evaluation framework to probe the potential of the existing models in the interactive scenarios.
  arXiv  Detail & Related papers  (2026-04-14T15:52:22Z)
• DQE-CIR: Distinctive Query Embeddings through Learnable Attribute Weights and Target Relative Negative Sampling in Composed Image Retrieval [53.482391830683014]
  Composed image retrieval (CIR) addresses the task of retrieving a target image by jointly interpreting a reference image and a modification text that specifies the intended change.<n>Most existing methods are still built upon contrastive learning frameworks that treat the ground truth image as the only positive instance and all remaining images as negatives.<n>We propose distinctive query embeddings through learnable attribute weights and target relative negative sampling.
  arXiv  Detail & Related papers  (2026-03-04T13:17:44Z)
• Learning Fine-Grained Correspondence with Cross-Perspective Perception for Open-Vocabulary 6D Object Pose Estimation [14.262846967061947]
  Fine-grained Correspondence Pose Estimation (FiCoP) is a framework that transitions from noise-prone global matching to spatially-constrained patch-level correspondence.<n>FiCoP improves Average Recall by 8.0% and 6.1%, respectively, compared to the state-of-the-art method.
  arXiv  Detail & Related papers  (2026-01-20T03:48:54Z)
• SaFiRe: Saccade-Fixation Reiteration with Mamba for Referring Image Segmentation [58.80001825332851]
  Referring Image (RIS) aims to segment the target object in an image given a natural language expression.<n>Recent methods predominantly focus on simple expressions like "red car" or "left girl"
  arXiv  Detail & Related papers  (2025-10-11T10:50:58Z)
• SETR: A Two-Stage Semantic-Enhanced Framework for Zero-Shot Composed Image Retrieval [4.230223288110963]
  Zero-shot Composed Image Retrieval (ZS-CIR) aims to retrieve a target image given a reference image and a relative text, without relying on costly triplet annotations.<n>Existing CLIP-based methods face two core challenges: (1) union-based feature fusion indiscriminately aggregates all visual cues, carrying over irrelevant background details that dilute the intended modification, and (2) global cosine similarity from CLIP embeddings lacks the ability to resolve fine-grained semantic relations.
  arXiv  Detail & Related papers  (2025-09-30T09:41:52Z)
• CONFLARE: CONFormal LArge language model REtrieval [0.0]
  Retrieval-augmented generation (RAG) frameworks enable large language models (LLMs) to retrieve relevant information from a knowledge base and incorporate it into the context for generating responses. RAG does not guarantee valid responses if retrieval fails to identify the necessary information as the context for response generation. We introduce a four-step framework for applying conformal prediction to quantify retrieval uncertainty in RAG frameworks.
  arXiv  Detail & Related papers  (2024-04-04T02:58:21Z)
• On the Estimation of Image-matching Uncertainty in Visual Place Recognition [7.769607568805291]
  In Visual Place Recognition (VPR) the pose of a query image is estimated by comparing the image to a map of reference images with known reference poses. This work compares for the first time the main approaches for estimating the image-matching uncertainty. We formulate a simple baseline method, SUE'', which unlike the other methods considers the freely-available poses of the reference images in the map.
  arXiv  Detail & Related papers  (2024-03-31T03:24:48Z)
• Noisy-Correspondence Learning for Text-to-Image Person Re-identification [50.07634676709067]
  We propose a novel Robust Dual Embedding method (RDE) to learn robust visual-semantic associations even with noisy correspondences. Our method achieves state-of-the-art results both with and without synthetic noisy correspondences on three datasets.
  arXiv  Detail & Related papers  (2023-08-19T05:34:13Z)
• Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization [73.04187954213471]
  We introduce a unified learning approach to simultaneously modeling the coarse- and fine-grained retrieval. The proposed method has achieved +4.03%, +3.38%, and +2.40% Recall@50 accuracy over a strong baseline.
  arXiv  Detail & Related papers  (2022-11-14T14:25:40Z)
• Iterative Shrinking for Referring Expression Grounding Using Deep Reinforcement Learning [20.23920009396818]
  We are tackling the proposal-free referring expression grounding task, aiming at localizing the target object according to a query sentence. Existing proposal-free methods employ a query-image matching branch to select the highest-score point in the image feature map as the target box center. We propose an iterative shrinking mechanism to localize the target, where the shrinking direction is decided by a reinforcement learning agent.
  arXiv  Detail & Related papers  (2021-03-09T02:36:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.