Balsa: Learning a Query Optimizer Without Expert Demonstrations
- URL: http://arxiv.org/abs/2201.01441v1
- Date: Wed, 5 Jan 2022 03:59:29 GMT
- Title: Balsa: Learning a Query Optimizer Without Expert Demonstrations
- Authors: Zongheng Yang, Wei-Lin Chiang, Sifei Luan, Gautam Mittal, Michael Luo,
Ion Stoica
- Abstract summary: We present Balsa, a query built by deep reinforcement learning.
We demonstrate for first time that learning to optimize queries without learning from an expert is both possible and efficient.
- Score: 18.434140044005844
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Query optimizers are a performance-critical component in every database
system. Due to their complexity, optimizers take experts months to write and
years to refine. In this work, we demonstrate for the first time that learning
to optimize queries without learning from an expert optimizer is both possible
and efficient. We present Balsa, a query optimizer built by deep reinforcement
learning. Balsa first learns basic knowledge from a simple,
environment-agnostic simulator, followed by safe learning in real execution. On
the Join Order Benchmark, Balsa matches the performance of two expert query
optimizers, both open-source and commercial, with two hours of learning, and
outperforms them by up to 2.8$\times$ in workload runtime after a few more
hours. Balsa thus opens the possibility of automatically learning to optimize
in future compute environments where expert-designed optimizers do not exist.
Related papers
- FOSS: A Self-Learned Doctor for Query Optimizer [20.54782053709538]
Deep reinforcement learning (DRL) can be used to address the query optimization problem in database system.
We introduce FOSS, a novel DRL-based framework for query optimization.
We show that FOSS outperforms the state-of-the-art methods in terms of latency performance and optimization time.
arXiv Detail & Related papers (2023-12-11T13:05:51Z) - BitE : Accelerating Learned Query Optimization in a Mixed-Workload
Environment [0.36700088931938835]
BitE is a novel ensemble learning model using database statistics and metadata to tune a learned query for enhancing performance.
Our model achieves 19.6% more improved queries and 15.8% less regressed queries compared to the existing traditional methods.
arXiv Detail & Related papers (2023-06-01T16:05:33Z) - Lero: A Learning-to-Rank Query Optimizer [49.841082217997354]
We introduce a learning to rank query, called Lero, which builds on top of the native query and continuously learns to improve query optimization.
Rather than building a learned from scratch, Lero is designed to leverage decades of wisdom of databases and improve the native.
Lero achieves near optimal performance on several benchmarks.
arXiv Detail & Related papers (2023-02-14T07:31:11Z) - Learning to Optimize for Reinforcement Learning [58.01132862590378]
Reinforcement learning (RL) is essentially different from supervised learning, and in practice, these learneds do not work well even in simple RL tasks.
Agent-gradient distribution is non-independent and identically distributed, leading to inefficient meta-training.
We show that, although only trained in toy tasks, our learned can generalize unseen complex tasks in Brax.
arXiv Detail & Related papers (2023-02-03T00:11:02Z) - VeLO: Training Versatile Learned Optimizers by Scaling Up [67.90237498659397]
We leverage the same scaling approach behind the success of deep learning to learn versatiles.
We train an ingest for deep learning which is itself a small neural network that ingests and outputs parameter updates.
We open source our learned, meta-training code, the associated train test data, and an extensive benchmark suite with baselines at velo-code.io.
arXiv Detail & Related papers (2022-11-17T18:39:07Z) - Training Learned Optimizers with Randomly Initialized Learned Optimizers [49.67678615506608]
We show that a population of randomly learneds can be used to train themselves from scratch in an online fashion.
A form of population based training is used to orchestrate this self-training.
We believe feedback loops of this type will be important and powerful in the future of machine learning.
arXiv Detail & Related papers (2021-01-14T19:07:17Z) - Reverse engineering learned optimizers reveals known and novel
mechanisms [50.50540910474342]
Learneds are algorithms that can themselves be trained to solve optimization problems.
Our results help elucidate the previously murky understanding of how learneds work, and establish tools for interpreting future learneds.
arXiv Detail & Related papers (2020-11-04T07:12:43Z) - Tasks, stability, architecture, and compute: Training more effective
learned optimizers, and using them to train themselves [53.37905268850274]
We introduce a new, hierarchical, neural network parameterized, hierarchical with access to additional features such as validation loss to enable automatic regularization.
Most learneds have been trained on only a single task, or a small number of tasks.
We train ours on thousands of tasks, making use of orders of magnitude more compute, resulting in generalizes that perform better to unseen tasks.
arXiv Detail & Related papers (2020-09-23T16:35:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.