Unsupervised Query Routing for Retrieval Augmented Generation

• URL: http://arxiv.org/abs/2501.07793v1
• Date: Tue, 14 Jan 2025 02:27:06 GMT
• Title: Unsupervised Query Routing for Retrieval Augmented Generation
• Authors: Feiteng Mu, Liwen Zhang, Yong Jiang, Wenjie Li, Zhen Zhang, Pengjun Xie, Fei Huang,
• Abstract summary: We introduce a novel unsupervised method that constructs the "upper-bound" response to evaluate the quality of retrieval-augmented responses.<n>This evaluation enables the decision of the most suitable search engine for a given query.<n>By eliminating manual annotations, our approach can automatically process large-scale real user queries and create training data.
• Score: 64.47987041500966
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Query routing for retrieval-augmented generation aims to assign an input query to the most suitable search engine. Existing works rely heavily on supervised datasets that require extensive manual annotation, resulting in high costs and limited scalability, as well as poor generalization to out-of-distribution scenarios. To address these challenges, we introduce a novel unsupervised method that constructs the "upper-bound" response to evaluate the quality of retrieval-augmented responses. This evaluation enables the decision of the most suitable search engine for a given query. By eliminating manual annotations, our approach can automatically process large-scale real user queries and create training data. We conduct extensive experiments across five datasets, demonstrating that our method significantly enhances scalability and generalization capabilities.

Related papers

• Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Control [52.405085773954596]
  Retrieval-Augmented Generation (RAG) has emerged as a powerful approach to mitigate large language model hallucinations. Existing RAG frameworks often apply retrieval indiscriminately,leading to inefficiencies-over-retrieving. We introduce a novel user-controllable RAG framework that enables dynamic adjustment of the accuracy-cost trade-off.
  arXiv  Detail & Related papers  (2025-02-17T18:56:20Z)
• Reasoning-Aware Query-Focused Summarization over Multi-Table Data [1.325953054381901]
  We propose QueryTableSummarizer++, an end-to-end generative framework leveraging large language models (LLMs)<n>Our method eliminates the need for intermediate serialization steps and directly generates query-relevant summaries.<n> Experiments on a benchmark dataset demonstrate that QueryTableSummarizer++ significantly outperforms state-of-the-art baselines in terms of BLEU, ROUGE, and F1-score.
  arXiv  Detail & Related papers  (2024-12-12T06:04:31Z)
• Improving Retrieval in Sponsored Search by Leveraging Query Context Signals [6.152499434499752]
  We propose an approach to enhance query understanding by augmenting queries with rich contextual signals. We use web search titles and snippets to ground queries in real-world information and utilize GPT-4 to generate query rewrites and explanations. Our context-aware approach substantially outperforms context-free models.
  arXiv  Detail & Related papers  (2024-07-19T14:28:53Z)
• CLARINET: Augmenting Language Models to Ask Clarification Questions for Retrieval [52.134133938779776]
  We present CLARINET, a system that asks informative clarification questions by choosing questions whose answers would maximize certainty in the correct candidate. Our approach works by augmenting a large language model (LLM) to condition on a retrieval distribution, finetuning end-to-end to generate the question that would have maximized the rank of the true candidate at each turn.
  arXiv  Detail & Related papers  (2024-04-28T18:21:31Z)
• Zero-shot Clarifying Question Generation for Conversational Search [25.514678546942754]
  We propose a constrained clarifying question generation system which uses both question templates and query facets to guide the effective and precise question generation. Experiment results show that our method outperforms existing state-of-the-art zero-shot baselines by a large margin.
  arXiv  Detail & Related papers  (2023-01-30T04:43:02Z)
• AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentation [44.93777271276723]
  We propose two approaches that enable annotation-free and scalable training by creating pseudo querydocument pairs. The query extraction method involves selecting salient spans from the original document to generate pseudo queries. The transferred query generation method utilizes generation models trained for other NLP tasks, such as summarization, to produce pseudo queries.
  arXiv  Detail & Related papers  (2022-12-17T10:43:25Z)
• Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking [56.80065604034095]
  We introduce a kNN approach that re-ranks documents based on their similarity with the query and the documents the user considers relevant. To evaluate our different integration strategies, we transform four existing information retrieval datasets into the relevance feedback scenario.
  arXiv  Detail & Related papers  (2022-10-19T16:19:37Z)
• Improving Passage Retrieval with Zero-Shot Question Generation [109.11542468380331]
  We propose a simple and effective re-ranking method for improving passage retrieval in open question answering. The re-ranker re-scores retrieved passages with a zero-shot question generation model, which uses a pre-trained language model to compute the probability of the input question conditioned on a retrieved passage.
  arXiv  Detail & Related papers  (2022-04-15T14:51:41Z)
• Exposing Query Identification for Search Transparency [69.06545074617685]
  We explore the feasibility of approximate exposing query identification (EQI) as a retrieval task by reversing the role of queries and documents in two classes of search systems. We derive an evaluation metric to measure the quality of a ranking of exposing queries, as well as conducting an empirical analysis focusing on various practical aspects of approximate EQI.
  arXiv  Detail & Related papers  (2021-10-14T20:19:27Z)
• High Quality Related Search Query Suggestions using Deep Reinforcement Learning [0.15229257192293202]
  "High Quality Related Search Query Suggestions" task aims at recommending search queries which are real, accurate, diverse, relevant and engaging. We train a Deep Reinforcement Learning model to predict the query a user would enter next. The reward signal is composed of long-term session-based user feedback, syntactic relatedness and estimated naturalness of generated query.
  arXiv  Detail & Related papers  (2021-08-10T05:22:32Z)
• Abstractive Query Focused Summarization with Query-Free Resources [60.468323530248945]
  In this work, we consider the problem of leveraging only generic summarization resources to build an abstractive QFS system. We propose Marge, a Masked ROUGE Regression framework composed of a novel unified representation for summaries and queries. Despite learning from minimal supervision, our system achieves state-of-the-art results in the distantly supervised setting.
  arXiv  Detail & Related papers  (2020-12-29T14:39:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.