Query Performance Explanation through Large Language Model for HTAP Systems

• URL: http://arxiv.org/abs/2412.01709v1
• Date: Mon, 02 Dec 2024 16:55:07 GMT
• Title: Query Performance Explanation through Large Language Model for HTAP Systems
• Authors: Haibo Xiu, Li Zhang, Tieying Zhang, Jun Yang, Jianjun Chen,
• Abstract summary: In hybrid transactional and analytical processing systems, users often struggle to understand why query plans from one engine perform slower than those from another. We propose a novel framework that leverages large language models (LLMs) to explain query performance in HTAP systems.
• Score: 8.278943524339264
• License:
• Abstract: In hybrid transactional and analytical processing (HTAP) systems, users often struggle to understand why query plans from one engine (OLAP or OLTP) perform significantly slower than those from another. Although optimizers provide plan details via the EXPLAIN function, these explanations are frequently too technical for non-experts and offer limited insights into performance differences across engines. To address this, we propose a novel framework that leverages large language models (LLMs) to explain query performance in HTAP systems. Built on Retrieval-Augmented Generation (RAG), our framework constructs a knowledge base that stores historical query executions and expert-curated explanations. To enable efficient retrieval of relevant knowledge, query plans are embedded using a lightweight tree-CNN classifier. This augmentation allows the LLM to generate clear, context-aware explanations of performance differences between engines. Our approach demonstrates the potential of LLMs in hybrid engine systems, paving the way for further advancements in database optimization and user support.

Related papers

• MERLIN: Multi-stagE query performance prediction for dynamic paRallel oLap pIpeliNe [8.024724736461328]
  High-performance OLAP database technology has emerged with the growing demand for massive data analysis. Many executors adopt sophisticated designs including SIMD operators, parallel execution, and dynamic pipeline modification. MERLIN is a multi-stage query performance prediction method for high-performance OLAPs.
  arXiv  Detail & Related papers  (2024-12-01T09:58:54Z)
• The Unreasonable Effectiveness of LLMs for Query Optimization [4.50924404547119]
  We show that embeddings of query text contain useful semantic information for query optimization. We show that a simple binary deciding between alternative query plans, trained on a small number of embedded query vectors, can outperform existing systems.
  arXiv  Detail & Related papers  (2024-11-05T07:10:00Z)
• SocialGPT: Prompting LLMs for Social Relation Reasoning via Greedy Segment Optimization [70.11167263638562]
  Social relation reasoning aims to identify relation categories such as friends, spouses, and colleagues from images. We first present a simple yet well-crafted framework named name, which combines the perception capability of Vision Foundation Models (VFMs) and the reasoning capability of Large Language Models (LLMs) within a modular framework.
  arXiv  Detail & Related papers  (2024-10-28T18:10:26Z)
• Towards Enhancing Linked Data Retrieval in Conversational UIs using Large Language Models [1.3980986259786221]
  This paper examines the integration of Large Language Models (LLMs) within existing systems. By leveraging the advanced natural language understanding capabilities of LLMs, our method improves RDF entity extraction within web systems. The evaluation of this methodology shows a marked enhancement in system expressivity and the accuracy of responses to user queries.
  arXiv  Detail & Related papers  (2024-09-24T16:31:33Z)
• The Compressor-Retriever Architecture for Language Model OS [20.56093501980724]
  This paper explores the concept of using a language model as the core component of an operating system (OS) A key challenge in realizing such an LM OS is managing the life-long context and ensuring statefulness across sessions. We introduce compressor-retriever, a model-agnostic architecture designed for life-long context management.
  arXiv  Detail & Related papers  (2024-09-02T23:28:15Z)
• UQE: A Query Engine for Unstructured Databases [71.49289088592842]
  We investigate the potential of Large Language Models to enable unstructured data analytics. We propose a new Universal Query Engine (UQE) that directly interrogates and draws insights from unstructured data collections.
  arXiv  Detail & Related papers  (2024-06-23T06:58:55Z)
• Retrieve-Plan-Generation: An Iterative Planning and Answering Framework for Knowledge-Intensive LLM Generation [47.22520829950929]
  We propose the Retrieve-Plan-Generation (RPG) framework for large language models (LLMs) RPG generates plan tokens to guide subsequent generation in the plan stage. In the answer stage, the model selects relevant fine-grained paragraphs based on the plan and uses them for further answer generation.
  arXiv  Detail & Related papers  (2024-06-21T08:45:52Z)
• Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs [59.76268575344119]
  We introduce a novel framework for enhancing large language models' (LLMs) planning capabilities by using planning data derived from knowledge graphs (KGs) LLMs fine-tuned with KG data have improved planning capabilities, better equipping them to handle complex QA tasks that involve retrieval.
  arXiv  Detail & Related papers  (2024-06-20T13:07:38Z)
• Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? [54.667202878390526]
  Long-context language models (LCLMs) have the potential to revolutionize our approach to tasks traditionally reliant on external tools like retrieval systems or databases. We introduce LOFT, a benchmark of real-world tasks requiring context up to millions of tokens designed to evaluate LCLMs' performance on in-context retrieval and reasoning. Our findings reveal LCLMs' surprising ability to rival state-of-the-art retrieval and RAG systems, despite never having been explicitly trained for these tasks.
  arXiv  Detail & Related papers  (2024-06-19T00:28:58Z)
• An In-Context Schema Understanding Method for Knowledge Base Question Answering [70.87993081445127]
  Large Language Models (LLMs) have shown strong capabilities in language understanding and can be used to solve this task. Existing methods bypass this challenge by initially employing LLMs to generate drafts of logic forms without schema-specific details. We propose a simple In-Context Understanding (ICSU) method that enables LLMs to directly understand schemas by leveraging in-context learning.
  arXiv  Detail & Related papers  (2023-10-22T04:19:17Z)
• Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation [76.76046657162306]
  Large language models (LLMs) have emerged as a new paradigm for Text-to- task. Large language models (LLMs) have emerged as a new paradigm for Text-to- task.
  arXiv  Detail & Related papers  (2023-08-29T14:59:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.