MANGO: A Python Library for Parallel Hyperparameter Tuning

• URL: http://arxiv.org/abs/2005.11394v1
• Date: Fri, 22 May 2020 20:58:26 GMT
• Title: MANGO: A Python Library for Parallel Hyperparameter Tuning
• Authors: Sandeep Singh Sandha, Mohit Aggarwal, Igor Fedorov, Mani Srivastava
• Abstract summary: We present Mango, a Python library for parallel hyperparameter tuning. Mango enables the use of any distributed scheduling framework. It implements intelligent parallel search strategies, and provides rich abstractions.
• Score: 4.728291880913813
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tuning hyperparameters for machine learning algorithms is a tedious task, one that is typically done manually. To enable automated hyperparameter tuning, recent works have started to use techniques based on Bayesian optimization. However, to practically enable automated tuning for large scale machine learning training pipelines, significant gaps remain in existing libraries, including lack of abstractions, fault tolerance, and flexibility to support scheduling on any distributed computing framework. To address these challenges, we present Mango, a Python library for parallel hyperparameter tuning. Mango enables the use of any distributed scheduling framework, implements intelligent parallel search strategies, and provides rich abstractions for defining complex hyperparameter search spaces that are compatible with scikit-learn. Mango is comparable in performance to Hyperopt, another widely used library. Mango is available open-source and is currently used in production at Arm Research to provide state-of-art hyperparameter tuning capabilities.

Related papers

• Hyperparameter Optimization in Machine Learning [34.356747514732966]
  Hyperparameters are configuration variables controlling the behavior of machine learning algorithms. The choice of their values determine the effectiveness of systems based on these technologies. We present a unified treatment of hyperparameter optimization, providing the reader with examples and insights into the state-of-the-art.
  arXiv  Detail & Related papers  (2024-10-30T09:39:22Z)
• Tune As You Scale: Hyperparameter Optimization For Compute Efficient Training [0.0]
  We propose a practical method for robustly tuning large models. CarBS performs local search around the performance-cost frontier. Among our results, we effectively solve the entire ProcGen benchmark just by tuning a simple baseline.
  arXiv  Detail & Related papers  (2023-06-13T18:22:24Z)
• Parameter-efficient Tuning of Large-scale Multimodal Foundation Model [68.24510810095802]
  We propose A graceful prompt framework for cross-modal transfer (Aurora) to overcome these challenges. Considering the redundancy in existing architectures, we first utilize the mode approximation to generate 0.1M trainable parameters to implement the multimodal prompt tuning. A thorough evaluation on six cross-modal benchmarks shows that it not only outperforms the state-of-the-art but even outperforms the full fine-tuning approach.
  arXiv  Detail & Related papers  (2023-05-15T06:40:56Z)
• Online Continuous Hyperparameter Optimization for Generalized Linear Contextual Bandits [55.03293214439741]
  In contextual bandits, an agent sequentially makes actions from a time-dependent action set based on past experience. We propose the first online continuous hyperparameter tuning framework for contextual bandits. We show that it could achieve a sublinear regret in theory and performs consistently better than all existing methods on both synthetic and real datasets.
  arXiv  Detail & Related papers  (2023-02-18T23:31:20Z)
• Automating DBSCAN via Deep Reinforcement Learning [73.82740568765279]
  We propose a novel Deep Reinforcement Learning guided automatic DBSCAN parameters search framework, namely DRL-DBSCAN. The framework models the process of adjusting the parameter search direction by perceiving the clustering environment as a Markov decision process. The framework consistently improves DBSCAN clustering accuracy by up to 26% and 25% respectively.
  arXiv  Detail & Related papers  (2022-08-09T04:40:11Z)
• AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuning [72.54359545547904]
  We propose a gradient-based subset selection framework for hyper- parameter tuning. We show that using gradient-based data subsets for hyper- parameter tuning achieves significantly faster turnaround times and speedups of 3$times$-30$times$.
  arXiv  Detail & Related papers  (2022-03-15T19:25:01Z)
• Towards Robust and Automatic Hyper-Parameter Tunning [39.04604349338802]
  We introduce a new class of HPO method and explore how the low-rank factorization of intermediate layers of a convolutional network can be used to define an analytical response surface. We quantify how this surface behaves as a surrogate to model performance and can be solved using a trust-region search algorithm, which we call autoHyper.
  arXiv  Detail & Related papers  (2021-11-28T05:27:34Z)
• High-Dimensional Bayesian Optimization with Multi-Task Learning for RocksDB [0.0]
  RocksDB is a general-purpose embedded key-value store. This paper investigates maximizing the throughput of RocksDB IO operations by auto-tuning ten parameters.
  arXiv  Detail & Related papers  (2021-03-30T11:38:52Z)
• Sherpa: Robust Hyperparameter Optimization for Machine Learning [6.156647008180291]
  Sherpa is a hyperparameter optimization library for machine learning models. It is specifically designed for problems with computationally expensive, iterative function evaluations. Sherpa can be run on either a single machine or in parallel on a cluster.
  arXiv  Detail & Related papers  (2020-05-08T13:52:49Z)
• Joint Parameter-and-Bandwidth Allocation for Improving the Efficiency of Partitioned Edge Learning [73.82875010696849]
  Machine learning algorithms are deployed at the network edge for training artificial intelligence (AI) models. This paper focuses on the novel joint design of parameter (computation load) allocation and bandwidth allocation.
  arXiv  Detail & Related papers  (2020-03-10T05:52:15Z)
• Dynamic Parameter Allocation in Parameter Servers [74.250687861348]
  We propose to integrate dynamic parameter allocation into parameter servers, describe an efficient implementation of such a parameter server called Lapse. We found that Lapse provides near-linear scaling and can be orders of magnitude faster than existing parameter servers.
  arXiv  Detail & Related papers  (2020-02-03T11:37:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.