Show, Don't Ask: Generative Visual Disambiguation for Composed Image Retrieval with Turn-Valid Coverage

• URL: http://arxiv.org/abs/2606.18992v1
• Date: Wed, 17 Jun 2026 12:13:57 GMT
• Title: Show, Don't Ask: Generative Visual Disambiguation for Composed Image Retrieval with Turn-Valid Coverage
• Abstract summary: Composed image retrieval (CIR) uses a reference image and a text modification to search for a target image.<n>We propose CLARA, a clarification framework that resolves ambiguity by showing users a small panel of visual alternatives.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Composed image retrieval (CIR) uses a reference image and a text modification to search for a target image. However, such queries often describe several possible images rather than one exact target, making the user's intent ambiguous. Recent methods address this by using conformal prediction to estimate ambiguity and by asking users clarifying text questions. However, these methods have two limitations: their coverage guarantee only holds at the first interaction, and text questions are often insufficient for resolving fine-grained visual differences such as appearance, attributes, or viewpoint. We propose CLARA, a clarification framework that resolves ambiguity by showing users a small panel of visual alternatives. Instead of answering text questions, the user simply selects the prototype image closest to the intended target. This provides a direct visual signal and avoids relying on a model to predict the user's answer. To maintain valid conformal guarantees across multiple interaction rounds, CLARA reweights calibration using the likelihood ratio induced by the user's selection. The displayed prototypes are also constrained to represent the current candidate set and are snapped to real corpus images, ensuring that generated images cannot artificially improve coverage. Experiments on open-domain and fashion benchmarks show that CLARA matches single-turn state-of-the-art retrieval performance, maintains nominal coverage across interaction rounds, and finds the intended target in fewer rounds than strong text-question baselines. Its advantage is especially clear when ambiguity involves viewpoint or fine-grained attributes, where visual clarification is more effective than textual questioning.

Related papers

• Automatic Red Teaming for Implicit Vulnerabilities of Text-to-Image Models [58.59008920408855]
  We propose Adversarial Probing for Implicit VulnErabilities (AdvPIE) to expose implicit vulnerabilities without requiring access to the parameters of target models.<n> AdvPIE adopts a policy agent to generate and refine implicit adversarial prompts based on the feedback from a judge agent.<n>Experiments on standard and safety-aligned T2I models show that AdvPIE1 effectively uncovers implicit vulnerabilities.
  arXiv  Detail & Related papers  (2026-09-05T13:45:09Z)
• Resolving Ambiguity in Composed Image Retrieval via Calibrated Interaction [0.0]
  Composed image retrieval (CIR) searches a corpus with a reference image and a text describing how to modify it.<n>We introduce AmbiCIR, a benchmark and human-validated user simulator that revives the dormant auxiliary and dialogue annotations of CIRR.
  arXiv  Detail & Related papers  (2026-05-23T15:49:16Z)
• DQE-CIR: Distinctive Query Embeddings through Learnable Attribute Weights and Target Relative Negative Sampling in Composed Image Retrieval [53.482391830683014]
  Composed image retrieval (CIR) addresses the task of retrieving a target image by jointly interpreting a reference image and a modification text that specifies the intended change.<n>Most existing methods are still built upon contrastive learning frameworks that treat the ground truth image as the only positive instance and all remaining images as negatives.<n>We propose distinctive query embeddings through learnable attribute weights and target relative negative sampling.
  arXiv  Detail & Related papers  (2026-03-04T13:17:44Z)
• Seeing Through Words: Controlling Visual Retrieval Quality with Language Models [68.49490036960559]
  We propose a new paradigm of quality-controllable retrieval, which enriches short queries with contextual details while incorporating explicit notions of image quality.<n>Our key idea is to leverage a generative language model as a query completion function, extending underspecified queries into descriptive forms.<n>Our proposed approach significantly improves retrieval results and provides effective quality control, bridging the gap between the expressive capacity of modern VLMs and the underspecified nature of short user queries.
  arXiv  Detail & Related papers  (2026-02-24T18:20:57Z)
• Will It Zero-Shot?: Predicting Zero-Shot Classification Performance For Arbitrary Queries [19.511404894563455]
  We build on prior work using text-only comparisons to evaluate how well a model works for a given natural language task.<n>We explore approaches that also generate synthetic images relevant to that task to evaluate and refine the prediction of zero-shot accuracy.
  arXiv  Detail & Related papers  (2026-01-24T17:30:23Z)
• SGDiff: Scene Graph Guided Diffusion Model for Image Collaborative SegCaptioning [53.638998508418545]
  This paper introduces a new task Image Collaborative and Captioning'' (SegCaptioning)<n>SegCaptioning aims to translate a straightforward prompt, like a bounding box around an object, into diverse semantic interpretations represented by (caption, masks) pairs.<n>This task poses significant challenges, including accurately capturing a user's intention from a minimal prompt while simultaneously predicting multiple semantically aligned caption words and masks.
  arXiv  Detail & Related papers  (2025-12-01T18:33:04Z)
• Leveraging Hierarchical Image-Text Misalignment for Universal Fake Image Detection [58.927873049646024]
  We show that fake images cannot be properly aligned with corresponding captions compared to real images.<n>We propose a simple yet effective ITEM by leveraging the image-text misalignment in a joint visual-language space as discriminative clues.
  arXiv  Detail & Related papers  (2025-11-01T06:51:14Z)
• Is a Caption Worth a Thousand Images? A Controlled Study for Representation Learning [88.5382122413913]
  We study whether language supervision can result in vision models with more transferable representations than traditional image-only methods. We find that image-only methods do not match CLIP's transfer performance, even when they are trained with more image data. Motivated by our findings, we devise simple prescriptions to enable CLIP to better leverage the language information present in existing pre-training datasets.
  arXiv  Detail & Related papers  (2022-07-15T17:50:51Z)
• No Token Left Behind: Explainability-Aided Image Classification and Generation [79.4957965474334]
  We present a novel explainability-based approach, which adds a loss term to ensure that CLIP focuses on all relevant semantic parts of the input. Our method yields an improvement in the recognition rate, without additional training or fine-tuning.
  arXiv  Detail & Related papers  (2022-04-11T07:16:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.