ReCQR: Incorporating conversational query rewriting to improve Multimodal Image Retrieval

• URL: http://arxiv.org/abs/2603.26669v1
• Date: Mon, 19 Jan 2026 13:10:54 GMT
• Title: ReCQR: Incorporating conversational query rewriting to improve Multimodal Image Retrieval
• Abstract summary: We introduce the conversational query rewriting task into the image retrieval domain.<n>Built on full dialogue histories, CQR rewrites users' final queries into concise, semantically complete ones.<n>We benchmark several SOTA multimodal models on the ReCQR dataset to assess their performance on image retrieval.
• Score: 24.076540962866236
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the rise of multimodal learning, image retrieval plays a crucial role in connecting visual information with natural language queries. Existing image retrievers struggle with processing long texts and handling unclear user expressions. To address these issues, we introduce the conversational query rewriting (CQR) task into the image retrieval domain and construct a dedicated multi-turn dialogue query rewriting dataset. Built on full dialogue histories, CQR rewrites users' final queries into concise, semantically complete ones that are better suited for retrieval. Specifically, We first leverage Large Language Models (LLMs) to generate rewritten candidates at scale and employ an LLM-as-Judge mechanism combined with manual review to curate approximately 7,000 high-quality multimodal dialogues, forming the ReCQR dataset. Then We benchmark several SOTA multimodal models on the ReCQR dataset to assess their performance on image retrieval. Experimental results demonstrate that CQR not only significantly enhances the accuracy of traditional image retrieval models, but also provides new directions and insights for modeling user queries in multimodal systems.

Related papers

• RCEM: Embedder Equipped with Query Rewriting Skill for Robust Conversational Search in Distributional Shift [7.4048532358172245]
  We propose RCEM, a conversational dense retrieval model that distills the query reformulation capability of LLMs into the embedding model.<n>Unlike prior conversational dense retrieval approaches that learn direct conversation-to-document matching, RCEM aligns conversational-Query embeddings with rewritten-Query embeddings.<n>Experiments on QReCC, TopiOCQA, and TREC CAsT demonstrate that RCEM consistently outperforms strong conversational retrieval baselines.
  arXiv  Detail & Related papers  (2026-06-01T05:04:17Z)
• Memory-Augmented Query Intent Understanding for Efficient Chat-based Image Retrieval [74.93546115702155]
  Chat-based image retrieval allows the human-interactive system to iteratively clarify and refine user intent through multi-round dialogue.<n>The key challenge in this task lies in dynamically understanding and updating the user's query intent across dialogue rounds.<n>This paper proposes a novel and efficient memory-based user intent updating framework for the chat-based image retrieval task.
  arXiv  Detail & Related papers  (2026-05-17T10:17:41Z)
• Recurrence Meets Transformers for Universal Multimodal Retrieval [59.92546492752452]
  ReT-2 is a unified retrieval model that supports multimodal queries composed of both images and text.<n>We evaluate ReT-2 on the challenging M2KR and M-BEIR benchmarks across different retrieval configurations.<n>When integrated into retrieval-augmented generation pipelines, ReT-2 also improves downstream performance on Encyclopedic-VQA and InfoSeek datasets.
  arXiv  Detail & Related papers  (2025-09-10T18:00:29Z)
• VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation for Multilingual Long Document Understanding [49.07705729597171]
  VisR-Bench is a benchmark for question-driven multimodal retrieval in long documents.<n>Our benchmark comprises over 35K high-quality QA pairs across 1.2K documents.<n>We evaluate various retrieval models, including text-based methods, multimodal encoders, and MLLMs.
  arXiv  Detail & Related papers  (2025-08-10T21:44:43Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• MIRe: Enhancing Multimodal Queries Representation via Fusion-Free Modality Interaction for Multimodal Retrieval [26.585985828583304]
  We introduce MIRe, a retrieval framework that achieves modality interaction without fusing textual features during the alignment.<n>Our method allows the textual query to attend to visual embeddings while not feeding text-driven signals back into the visual representations.<n>Our experiments demonstrate that our pre-training strategy significantly enhances the understanding of multimodal queries.
  arXiv  Detail & Related papers  (2024-11-13T04:32:58Z)
• Generating Multi-Aspect Queries for Conversational Search [6.974395116689502]
  We show that the same retrieval model would perform better with more than one rewritten query by 85% in terms of nDCG@3. We propose a multi-aspect query generation and retrieval framework, called MQ4CS.
  arXiv  Detail & Related papers  (2024-03-28T10:40:22Z)
• End-to-end Knowledge Retrieval with Multi-modal Queries [50.01264794081951]
  ReMuQ requires a system to retrieve knowledge from a large corpus by integrating contents from both text and image queries. We introduce a retriever model ReViz'' that can directly process input text and images to retrieve relevant knowledge in an end-to-end fashion. We demonstrate superior performance in retrieval on two datasets under zero-shot settings.
  arXiv  Detail & Related papers  (2023-06-01T08:04:12Z)
• Query Resolution for Conversational Search with Limited Supervision [63.131221660019776]
  We propose QuReTeC (Query Resolution by Term Classification), a neural query resolution model based on bidirectional transformers. We show that QuReTeC outperforms state-of-the-art models, and furthermore, that our distant supervision method can be used to substantially reduce the amount of human-curated data required to train QuReTeC.
  arXiv  Detail & Related papers  (2020-05-24T11:37:22Z)
• MLR: A Two-stage Conversational Query Rewriting Model with Multi-task Learning [16.88648782206587]
  We propose the conversational query rewriting model - MLR, which is a Multi-task model on sequence Labeling and query Rewriting. MLR reformulates the multi-turn conversational queries into a single turn query, which conveys the true intention of users concisely. To train our model, we construct a new Chinese query rewriting dataset and conduct experiments on it.
  arXiv  Detail & Related papers  (2020-04-13T08:04:49Z)
• Using Image Captions and Multitask Learning for Recommending Query Reformulations [11.99358906295761]
  We aim to enhance the query recommendation experience for a commercial image search engine. Our proposed methodology incorporates current state-of-the-art practices from relevant literature.
  arXiv  Detail & Related papers  (2020-03-02T08:22:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.