Multi-agent Databases via Independent Learning

• URL: http://arxiv.org/abs/2205.14323v1
• Date: Sat, 28 May 2022 03:47:43 GMT
• Title: Multi-agent Databases via Independent Learning
• Authors: Chi Zhang, Olga Papaemmanouil, Josiah Hanna
• Abstract summary: We introduce MADB (Multi-Agent DB), a proof-of-concept system that incorporates a learned query scheduler and a learned query. Preliminary results demonstrate that MADB can outperform the non-cooperative integration of learned components.
• Score: 11.05491559831151
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning is rapidly being used in database research to improve the effectiveness of numerous tasks included but not limited to query optimization, workload scheduling, physical design, etc. essential database components, such as the optimizer, scheduler, and physical designer. Currently, the research focus has been on replacing a single database component responsible for one task by its learning-based counterpart. However, query performance is not simply determined by the performance of a single component, but by the cooperation of multiple ones. As such, learned based database components need to collaborate during both training and execution in order to develop policies that meet end performance goals. Thus, the paper attempts to address the question "Is it possible to design a database consisting of various learned components that cooperatively work to improve end-to-end query latency?". To answer this question, we introduce MADB (Multi-Agent DB), a proof-of-concept system that incorporates a learned query scheduler and a learned query optimizer. MADB leverages a cooperative multi-agent reinforcement learning approach that allows the two components to exchange the context of their decisions with each other and collaboratively work towards reducing the query latency. Preliminary results demonstrate that MADB can outperform the non-cooperative integration of learned components.

Related papers

• Relational Database Augmented Large Language Model [59.38841050766026]
  Large language models (LLMs) excel in many natural language processing (NLP) tasks. They can only incorporate new knowledge through training or supervised fine-tuning processes. This precise, up-to-date, and private information is typically stored in relational databases.
  arXiv  Detail & Related papers  (2024-07-21T06:19:10Z)
• Towards Completeness-Oriented Tool Retrieval for Large Language Models [60.733557487886635]
  Real-world systems often incorporate a wide array of tools, making it impractical to input all tools into Large Language Models. Existing tool retrieval methods primarily focus on semantic matching between user queries and tool descriptions. We propose a novel modelagnostic COllaborative Learning-based Tool Retrieval approach, COLT, which captures not only the semantic similarities between user queries and tool descriptions but also takes into account the collaborative information of tools.
  arXiv  Detail & Related papers  (2024-05-25T06:41:23Z)
• On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering [25.57202500348071]
  This study introduces a new long-form database question answering dataset designed to evaluate how Large Language Models interact with a database. The task requires LLMs to strategically generate multiplesql queries to retrieve sufficient data from a database, to reason with the acquired context, and to synthesize them into a comprehensive analytical narrative. We propose and evaluate two interaction strategies, and provide a fine-grained analysis of the individual stages within the interaction.
  arXiv  Detail & Related papers  (2023-11-16T09:55:07Z)
• LLM As DBA [25.92711955279298]
  Large language models (LLMs) have shown great potential to understand valuable documents and generate reasonable answers. This paper presents a revolutionary LLM-centric framework for database maintenance, including (i) database maintenance knowledge detection from documents and tools, (ii) tree of thought reasoning for root cause analysis, and (iii) collaborative diagnosis among multiple LLMs.
  arXiv  Detail & Related papers  (2023-08-10T10:12:43Z)
• BitE : Accelerating Learned Query Optimization in a Mixed-Workload Environment [0.36700088931938835]
  BitE is a novel ensemble learning model using database statistics and metadata to tune a learned query for enhancing performance. Our model achieves 19.6% more improved queries and 15.8% less regressed queries compared to the existing traditional methods.
  arXiv  Detail & Related papers  (2023-06-01T16:05:33Z)
• A Unified and Efficient Coordinating Framework for Autonomous DBMS Tuning [34.85351481228439]
  We propose a unified coordinating framework to efficiently utilize existing ML-based agents. We show that it can effectively utilize different ML-based agents and find better configurations with 1.414.1X speedups on the workload execution time.
  arXiv  Detail & Related papers  (2023-03-10T05:27:23Z)
• Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer [80.50327229467993]
  We show that a single model trained end-to-end can achieve both competitive retrieval and QA performance. We show that end-to-end adaptation significantly boosts its performance on out-of-domain datasets in both supervised and unsupervised settings.
  arXiv  Detail & Related papers  (2022-12-05T04:51:21Z)
• Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization [101.72755769194677]
  We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph. Our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks. Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks.
  arXiv  Detail & Related papers  (2022-05-25T10:44:25Z)
• Database Workload Characterization with Query Plan Encoders [32.941042348628606]
  We propose our query plan encoders that learn essential features and their correlations from query plans. Our pretrained encoders capture the em structural and the em computational performance of queries independently.
  arXiv  Detail & Related papers  (2021-05-26T01:17:27Z)
• A Unified Transferable Model for ML-Enhanced DBMS [53.46830627879208]
  We propose a unified model MTMLF that uses a multi-task training procedure to capture the transferable knowledge across tasks and a pretrain finetune procedure to distill the meta knowledge across DBs. We believe this paradigm is more suitable for cloud DB service, and has the potential to revolutionize the way how ML is used in the future.
  arXiv  Detail & Related papers  (2021-05-06T03:31:32Z)
• MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures [61.73533544385352]
  We propose a transferable perturbation, MetaPerturb, which is meta-learned to improve generalization performance on unseen data. As MetaPerturb is a set-function trained over diverse distributions across layers and tasks, it can generalize heterogeneous tasks and architectures.
  arXiv  Detail & Related papers  (2020-06-13T02:54:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.