Guided Hyperparameter Tuning Through Visualization and Inference

• URL: http://arxiv.org/abs/2105.11516v1
• Date: Mon, 24 May 2021 19:55:24 GMT
• Title: Guided Hyperparameter Tuning Through Visualization and Inference
• Authors: Hyekang Joo, Calvin Bao, Ishan Sen, Furong Huang, and Leilani Battle
• Abstract summary: We present a streamlined visualization system enabling deep learning practitioners to more efficiently explore, tune, and optimize hyper parameters. A key idea is to directly suggest more optimal hyper parameters using a predictive mechanism. We evaluate the tool with a user study on deep learning model builders, finding that our participants have little issue adopting the tool and working with it as part of their workflow.
• Score: 12.035299005299306
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For deep learning practitioners, hyperparameter tuning for optimizing model performance can be a computationally expensive task. Though visualization can help practitioners relate hyperparameter settings to overall model performance, significant manual inspection is still required to guide the hyperparameter settings in the next batch of experiments. In response, we present a streamlined visualization system enabling deep learning practitioners to more efficiently explore, tune, and optimize hyperparameters in a batch of experiments. A key idea is to directly suggest more optimal hyperparameter values using a predictive mechanism. We then integrate this mechanism with current visualization practices for deep learning. Moreover, an analysis on the variance in a selected performance metric in the context of the model hyperparameters shows the impact that certain hyperparameters have on the performance metric. We evaluate the tool with a user study on deep learning model builders, finding that our participants have little issue adopting the tool and working with it as part of their workflow.

Related papers

• Hyperparameter Optimization in Machine Learning [34.356747514732966]
  Hyperparameters are configuration variables controlling the behavior of machine learning algorithms. The choice of their values determine the effectiveness of systems based on these technologies. We present a unified treatment of hyperparameter optimization, providing the reader with examples and insights into the state-of-the-art.
  arXiv  Detail & Related papers  (2024-10-30T09:39:22Z)
• Efficient Hyperparameter Importance Assessment for CNNs [1.7778609937758323]
  This paper aims to quantify the importance weights of some hyperparameters in Convolutional Neural Networks (CNNs) with an algorithm called N-RReliefF. We conduct an extensive study by training over ten thousand CNN models across ten popular image classification datasets.
  arXiv  Detail & Related papers  (2024-10-11T15:47:46Z)
• Scaling Exponents Across Parameterizations and Optimizers [94.54718325264218]
  We propose a new perspective on parameterization by investigating a key assumption in prior work. Our empirical investigation includes tens of thousands of models trained with all combinations of threes. We find that the best learning rate scaling prescription would often have been excluded by the assumptions in prior work.
  arXiv  Detail & Related papers  (2024-07-08T12:32:51Z)
• Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis [51.14136878142034]
  Point cloud analysis has achieved outstanding performance by transferring point cloud pre-trained models. Existing methods for model adaptation usually update all model parameters, which is inefficient as it relies on high computational costs. In this paper, we aim to study parameter-efficient transfer learning for point cloud analysis with an ideal trade-off between task performance and parameter efficiency.
  arXiv  Detail & Related papers  (2024-03-03T08:25:04Z)
• E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning [55.50908600818483]
  Fine-tuning large-scale pretrained vision models for new tasks has become increasingly parameter-intensive. We propose an Effective and Efficient Visual Prompt Tuning (E2VPT) approach for large-scale transformer-based model adaptation. Our approach outperforms several state-of-the-art baselines on two benchmarks.
  arXiv  Detail & Related papers  (2023-07-25T19:03:21Z)
• Towards Learning Universal Hyperparameter Optimizers with Transformers [57.35920571605559]
  We introduce the OptFormer, the first text-based Transformer HPO framework that provides a universal end-to-end interface for jointly learning policy and function prediction. Our experiments demonstrate that the OptFormer can imitate at least 7 different HPO algorithms, which can be further improved via its function uncertainty estimates.
  arXiv  Detail & Related papers  (2022-05-26T12:51:32Z)
• AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuning [72.54359545547904]
  We propose a gradient-based subset selection framework for hyper- parameter tuning. We show that using gradient-based data subsets for hyper- parameter tuning achieves significantly faster turnaround times and speedups of 3$times$-30$times$.
  arXiv  Detail & Related papers  (2022-03-15T19:25:01Z)
• The Role of Adaptive Optimizers for Honest Private Hyperparameter Selection [12.38071940409141]
  We show that standard composition tools outperform more advanced techniques in many settings. We draw upon limiting behaviour of Adam in the DP setting to design a new and more efficient tool.
  arXiv  Detail & Related papers  (2021-11-09T01:56:56Z)
• To tune or not to tune? An Approach for Recommending Important Hyperparameters [2.121963121603413]
  We consider building the relationship between the performance of the machine learning models and their hyperparameters to discover the trend and gain insights. Our results enable users to decide whether it is worth conducting a possibly time-consuming tuning strategy.
  arXiv  Detail & Related papers  (2021-08-30T08:54:58Z)
• VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization [4.237343083490243]
  During the training phase of machine learning (ML) models, it is usually necessary to configure several hyper parameters. We present VisEvol, a visual analytics tool that supports interactive exploration of hyper parameters and intervention in this evolutionary procedure. The utility and applicability of VisEvol are demonstrated with two use cases and interviews with ML experts who evaluated the effectiveness of the tool.
  arXiv  Detail & Related papers  (2020-12-02T13:43:37Z)
• Rethinking the Hyperparameters for Fine-tuning [78.15505286781293]
  Fine-tuning from pre-trained ImageNet models has become the de-facto standard for various computer vision tasks. Current practices for fine-tuning typically involve selecting an ad-hoc choice of hyper parameters. This paper re-examines several common practices of setting hyper parameters for fine-tuning.
  arXiv  Detail & Related papers  (2020-02-19T18:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.