Deep learning based Auto Tuning for Database Management System

• URL: http://arxiv.org/abs/2304.12747v1
• Date: Tue, 25 Apr 2023 11:52:52 GMT
• Title: Deep learning based Auto Tuning for Database Management System
• Authors: Karthick Prasad Gunasekaran, Kajal Tiwari, Rachana Acharya
• Abstract summary: In this work, we extend an automated technique based on Ottertune to reuse data gathered from previous sessions to tune new deployments with the help of supervised and unsupervised machine learning methods to improve latency prediction. We use GMM clustering to prune metrics and combine ensemble models, such as RandomForest, with non-linear models, like neural networks, for prediction modeling.
• Score: 0.12891210250935148
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The management of database system configurations is a challenging task, as there are hundreds of configuration knobs that control every aspect of the system. This is complicated by the fact that these knobs are not standardized, independent, or universal, making it difficult to determine optimal settings. An automated approach to address this problem using supervised and unsupervised machine learning methods to select impactful knobs, map unseen workloads, and recommend knob settings was implemented in a new tool called OtterTune and is being evaluated on three DBMSs, with results demonstrating that it recommends configurations as good as or better than those generated by existing tools or a human expert.In this work, we extend an automated technique based on Ottertune [1] to reuse training data gathered from previous sessions to tune new DBMS deployments with the help of supervised and unsupervised machine learning methods to improve latency prediction. Our approach involves the expansion of the methods proposed in the original paper. We use GMM clustering to prune metrics and combine ensemble models, such as RandomForest, with non-linear models, like neural networks, for prediction modeling.

Related papers

• Attribute-to-Delete: Machine Unlearning via Datamodel Matching [65.13151619119782]
  Machine unlearning -- efficiently removing a small "forget set" training data on a pre-divertrained machine learning model -- has recently attracted interest. Recent research shows that machine unlearning techniques do not hold up in such a challenging setting.
  arXiv  Detail & Related papers  (2024-10-30T17:20:10Z)
• Dropout MPC: An Ensemble Neural MPC Approach for Systems with Learned Dynamics [0.0]
  We propose a novel sampling-based ensemble neural MPC algorithm that employs the Monte-Carlo dropout technique on the learned system model. The method aims in general at uncertain systems with complex dynamics, where models derived from first principles are hard to infer.
  arXiv  Detail & Related papers  (2024-06-04T17:15:25Z)
• Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining [50.00291020618743]
  This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU) Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems.
  arXiv  Detail & Related papers  (2024-04-08T20:02:19Z)
• An Integrated Data Processing Framework for Pretraining Foundation Models [57.47845148721817]
  Researchers and practitioners often have to manually curate datasets from difference sources. We propose a data processing framework that integrates a Processing Module and an Analyzing Module. The proposed framework is easy to use and highly flexible.
  arXiv  Detail & Related papers  (2024-02-26T07:22:51Z)
• Utilizing deep learning for automated tuning of database management systems [0.12891210250935148]
  OtterTune identifies influential knobs, analyze previously unseen workloads, and provide recommendations for knob settings. The effectiveness of this approach is demonstrated through the evaluation of a new tool called OtterTune on three different database management systems (DBMSs)
  arXiv  Detail & Related papers  (2023-06-25T21:50:14Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• On the Generalizability and Predictability of Recommender Systems [33.46314108814183]
  We give the first large-scale study of recommender system approaches. We create Reczilla, a meta-learning approach to recommender systems.
  arXiv  Detail & Related papers  (2022-06-23T17:51:42Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Knodle: Modular Weakly Supervised Learning with PyTorch [5.874587993411972]
  Knodle is a software framework for separating weak data annotations, powerful deep learning models, and methods for improving weakly supervised training. This modularization gives the training process access to fine-grained information such as data set characteristics, matches of rules, or elements of the deep learning model ultimately used for prediction.
  arXiv  Detail & Related papers  (2021-04-23T12:33:25Z)
• Amazon SageMaker Automatic Model Tuning: Scalable Black-box Optimization [23.52446054521187]
  Amazon SageMaker Automatic Model Tuning (AMT) is a fully managed system for black-box optimization at scale. AMT finds the best version of a machine learning model by repeatedly training it with different hyperparameter configurations. It can be used with built-in algorithms, custom algorithms, and Amazon SageMaker pre-built containers for machine learning frameworks.
  arXiv  Detail & Related papers  (2020-12-15T18:34:34Z)
• Data-driven Koopman Operators for Model-based Shared Control of Human-Machine Systems [66.65503164312705]
  We present a data-driven shared control algorithm that can be used to improve a human operator's control of complex machines. Both the dynamics and information about the user's interaction are learned from observation through the use of a Koopman operator. We find that model-based shared control significantly improves task and control metrics when compared to a natural learning, or user only, control paradigm.
  arXiv  Detail & Related papers  (2020-06-12T14:14:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.