A Light-Weight Multi-Objective Asynchronous Hyper-Parameter Optimizer

• URL: http://arxiv.org/abs/2202.07735v1
• Date: Tue, 15 Feb 2022 21:30:38 GMT
• Title: A Light-Weight Multi-Objective Asynchronous Hyper-Parameter Optimizer
• Authors: Gabriel Maher, Stephen Boyd, Mykel Kochenderfer, Cristian Matache, Alex Ulitsky, Slava Yukhymuk, Leonid Kopman
• Abstract summary: We describe a light-weight yet performant system for hyper- parameter optimization. It approximately minimizes an overall scalar cost function that is obtained by combining multiple performance objectives. It also supports a trade-off mode, where the goal is to find an appropriate trade-off among objectives by interacting with the user.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We describe a light-weight yet performant system for hyper-parameter optimization that approximately minimizes an overall scalar cost function that is obtained by combining multiple performance objectives using a target-priority-limit scalarizer. It also supports a trade-off mode, where the goal is to find an appropriate trade-off among objectives by interacting with the user. We focus on the common scenario where there are on the order of tens of hyper-parameters, each with various attributes such as a range of continuous values, or a finite list of values, and whether it should be treated on a linear or logarithmic scale. The system supports multiple asynchronous simulations and is robust to simulation stragglers and failures.

Related papers

• Semantic Codebook Learning for Dynamic Recommendation Models [55.98259490159084]
  Dynamic sequential recommendation (DSR) can generate model parameters based on user behavior to improve personalization of sequential recommendation. It faces the challenges of large parameter search space and sparse and noisy user-item interactions, which reduces the applicability of the generated model parameters. The Semantic Codebook Learning for Dynamic Recommendation Models (SOLID) framework presents a significant advancement in DSR by effectively tackling these challenges.
  arXiv  Detail & Related papers  (2024-07-31T19:25:25Z)
• Hyperparameter Importance Analysis for Multi-Objective AutoML [14.336028105614824]
  In this paper, we propose the first method for assessing the importance of hyperparameters in the context of multi-objective hyperparameter optimization. Specifically, we compute the a-priori scalarization of the objectives and determine the importance of the hyperparameters for different objective tradeoffs. Our findings offer valuable guidance for hyperparameter tuning in MOO tasks but also contribute to advancing the understanding of HPI in complex optimization scenarios.
  arXiv  Detail & Related papers  (2024-05-13T11:00:25Z)
• Parallel Multi-Objective Hyperparameter Optimization with Uniform Normalization and Bounded Objectives [5.94867851915494]
  We propose a multi-objective Bayesian optimization (MoBO) algorithm that addresses these problems. We increase the efficiency of our approach by imposing constraints on the objective to avoid exploring unnecessary configurations. Finally, we leverage an approach to parallelize the MoBO which results in a 5x speed-up when using 16x more workers.
  arXiv  Detail & Related papers  (2023-09-26T13:48:04Z)
• Parameter-efficient Tuning of Large-scale Multimodal Foundation Model [68.24510810095802]
  We propose A graceful prompt framework for cross-modal transfer (Aurora) to overcome these challenges. Considering the redundancy in existing architectures, we first utilize the mode approximation to generate 0.1M trainable parameters to implement the multimodal prompt tuning. A thorough evaluation on six cross-modal benchmarks shows that it not only outperforms the state-of-the-art but even outperforms the full fine-tuning approach.
  arXiv  Detail & Related papers  (2023-05-15T06:40:56Z)
• Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures [68.91874045918112]
  adapter-ALBERT is an efficient model optimization for maximal data reuse across different tasks. We demonstrate the advantage of mapping the model to a heterogeneous on-chip memory architecture by performing simulations on a validated NLP edge accelerator.
  arXiv  Detail & Related papers  (2023-03-25T14:40:59Z)
• Efficiently Controlling Multiple Risks with Pareto Testing [34.83506056862348]
  We propose a two-stage process which combines multi-objective optimization with multiple hypothesis testing. We demonstrate the effectiveness of our approach to reliably accelerate the execution of large-scale Transformer models in natural language processing (NLP) applications.
  arXiv  Detail & Related papers  (2022-10-14T15:54:39Z)
• Multi-objective and multi-fidelity Bayesian optimization of laser-plasma acceleration [0.0]
  We present first results on multi-objective optimization of a simulated laser-plasma accelerator. We find that multi-objective optimization is equal or even superior in performance to its single-objective counterparts. We significantly reduce the computational costs of the optimization by choosing the resolution and box size of the simulations dynamically.
  arXiv  Detail & Related papers  (2022-10-07T12:09:09Z)
• AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuning [72.54359545547904]
  We propose a gradient-based subset selection framework for hyper- parameter tuning. We show that using gradient-based data subsets for hyper- parameter tuning achieves significantly faster turnaround times and speedups of 3$times$-30$times$.
  arXiv  Detail & Related papers  (2022-03-15T19:25:01Z)
• A multi-objective perspective on jointly tuning hardware and hyperparameters [10.605719154114357]
  A full AutoML solution requires selecting appropriate hardware automatically. We adopt a multi-objective approach that selects and adapts the hardware configuration automatically. We show in extensive NAS and HPO experiments that both ingredients bring significant speed-ups and cost savings.
  arXiv  Detail & Related papers  (2021-06-10T11:52:55Z)
• DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution [136.7261709896713]
  We propose a data-driven approach that generates the appropriate convolution kernels to apply in response to the nature of the instances. The proposed method achieves promising results on both ScanetNetV2 and S3DIS. It also improves inference speed by more than 25% over the current state-of-the-art.
  arXiv  Detail & Related papers  (2020-11-26T14:56:57Z)
• Highly Efficient Salient Object Detection with 100K Parameters [137.74898755102387]
  We propose a flexible convolutional module, namely generalized OctConv (gOctConv), to efficiently utilize both in-stage and cross-stages multi-scale features. We build an extremely light-weighted model, namely CSNet, which achieves comparable performance with about 0.2% (100k) of large models on popular object detection benchmarks.
  arXiv  Detail & Related papers  (2020-03-12T07:00:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.