LLM-R2: A Large Language Model Enhanced Rule-based Rewrite System for Boosting Query Efficiency

• URL: http://arxiv.org/abs/2404.12872v1
• Date: Fri, 19 Apr 2024 13:17:07 GMT
• Title: LLM-R2: A Large Language Model Enhanced Rule-based Rewrite System for Boosting Query Efficiency
• Authors: Zhaodonghui Li, Haitao Yuan, Huiming Wang, Gao Cong, Lidong Bing,
• Abstract summary: We propose a novel method of query rewrite named LLM-R2, adopting a large language model (LLM) to propose possible rewrite rules for a database rewrite system. Experimental results have shown that our method can significantly improve the query execution efficiency and outperform the baseline methods.
• Score: 65.01402723259098
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Query rewrite, which aims to generate more efficient queries by altering a SQL query's structure without changing the query result, has been an important research problem. In order to maintain equivalence between the rewritten query and the original one during rewriting, traditional query rewrite methods always rewrite the queries following certain rewrite rules. However, some problems still remain. Firstly, existing methods of finding the optimal choice or sequence of rewrite rules are still limited and the process always costs a lot of resources. Methods involving discovering new rewrite rules typically require complicated proofs of structural logic or extensive user interactions. Secondly, current query rewrite methods usually rely highly on DBMS cost estimators which are often not accurate. In this paper, we address these problems by proposing a novel method of query rewrite named LLM-R2, adopting a large language model (LLM) to propose possible rewrite rules for a database rewrite system. To further improve the inference ability of LLM in recommending rewrite rules, we train a contrastive model by curriculum to learn query representations and select effective query demonstrations for the LLM. Experimental results have shown that our method can significantly improve the query execution efficiency and outperform the baseline methods. In addition, our method enjoys high robustness across different datasets.

Related papers

• Effective Instruction Parsing Plugin for Complex Logical Query Answering on Knowledge Graphs [51.33342412699939]
  Knowledge Graph Query Embedding (KGQE) aims to embed First-Order Logic (FOL) queries in a low-dimensional KG space for complex reasoning over incomplete KGs. Recent studies integrate various external information (such as entity types and relation context) to better capture the logical semantics of FOL queries. We propose an effective Query Instruction Parsing (QIPP) that captures latent query patterns from code-like query instructions.
  arXiv  Detail & Related papers  (2024-10-27T03:18:52Z)
• Crafting the Path: Robust Query Rewriting for Information Retrieval [4.252699657665555]
  We propose a novel structured query rewriting method called Crafting the Path tailored for retrieval systems. We demonstrate that our method is less dependent on the internal parameter knowledge of the model and generates queries with fewer factual inaccuracies.
  arXiv  Detail & Related papers  (2024-07-17T13:11:28Z)
• Learned Graph Rewriting with Equality Saturation: A New Paradigm in Relational Query Rewrite and Beyond [0.3749861135832073]
  Rewriting logical and physical relational query plans is proven to be an NP-hard sequential decision-making problem. In this paper, we address the query rewrite problem by interleaving Equality Saturation and Graph Reinforcement Learning.
  arXiv  Detail & Related papers  (2024-06-19T21:11:19Z)
• Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answers [66.55612528039894]
  AdaQR is a framework for training query rewriting models with limited rewrite annotations from seed datasets and completely no passage label. A novel approach is proposed to assess retriever's preference for these candidates by the probability of answers conditioned on the conversational query.
  arXiv  Detail & Related papers  (2024-06-16T16:09:05Z)
• RaFe: Ranking Feedback Improves Query Rewriting for RAG [83.24385658573198]
  We propose a framework for training query rewriting models free of annotations. By leveraging a publicly available reranker, oursprovides feedback aligned well with the rewriting objectives.
  arXiv  Detail & Related papers  (2024-05-23T11:00:19Z)
• Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewriting [42.35788605017555]
  We propose utilizing large language models (LLMs) as query rewriters. We define four essential properties for well-formed rewrites and incorporate all of them into the instruction. We introduce the role of rewrite editors for LLMs when initial query rewrites are available, forming a "rewrite-then-edit" process.
  arXiv  Detail & Related papers  (2023-10-15T03:04:17Z)
• Context Aware Query Rewriting for Text Rankers using LLM [5.164642900490078]
  We analyze the utility of large-language models for improved query rewriting for text ranking tasks. We adopt a simple, yet surprisingly effective, approach called context aware query rewriting (CAR) We find that fine-tuning a ranker using re-written queries offers a significant improvement of up to 33% on the passage ranking task and up to 28% on the document ranking task.
  arXiv  Detail & Related papers  (2023-08-31T14:19:50Z)
• ConvGQR: Generative Query Reformulation for Conversational Search [37.54018632257896]
  ConvGQR is a new framework to reformulate conversational queries based on generative pre-trained language models. We propose a knowledge infusion mechanism to optimize both query reformulation and retrieval.
  arXiv  Detail & Related papers  (2023-05-25T01:45:06Z)
• Query Rewriting for Retrieval-Augmented Large Language Models [139.242907155883]
  Large Language Models (LLMs) play powerful, black-box readers in the retrieve-then-read pipeline. This work introduces a new framework, Rewrite-Retrieve-Read instead of the previous retrieve-then-read for the retrieval-augmented LLMs.
  arXiv  Detail & Related papers  (2023-05-23T17:27:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.