HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimization

• URL: http://arxiv.org/abs/2502.01276v2
• Date: Mon, 10 Nov 2025 16:43:45 GMT
• Title: HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimization
• Authors: Marcel Wever, Maximilian Muschalik, Fabian Fumagalli, Marius Lindauer,
• Abstract summary: Black-box HPO methods undermine user trust and discourages adoption.<n>We propose a game-theoretic explainability framework for HPO based on Shapley values and interactions.<n>We demonstrate HyperSHAP's capabilities on various HPO benchmarks to analyze the interaction structure of the corresponding HPO problems.
• Score: 17.646452580419545
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Hyperparameter optimization (HPO) is a crucial step in achieving strong predictive performance. Yet, the impact of individual hyperparameters on model generalization is highly context-dependent, prohibiting a one-size-fits-all solution and requiring opaque HPO methods to find optimal configurations. However, the black-box nature of most HPO methods undermines user trust and discourages adoption. To address this, we propose a game-theoretic explainability framework for HPO based on Shapley values and interactions. Our approach provides an additive decomposition of a performance measure across hyperparameters, enabling local and global explanations of hyperparameters' contributions and their interactions. The framework, named HyperSHAP, offers insights into ablation studies, the tunability of learning algorithms, and optimizer behavior across different hyperparameter spaces. We demonstrate HyperSHAP's capabilities on various HPO benchmarks to analyze the interaction structure of the corresponding HPO problems, demonstrating its broad applicability and actionable insights for improving HPO.

Related papers

• Dynamic Hyperparameter Importance for Efficient Multi-Objective Optimization [10.530794046739619]
  Multi-objective optimization (MOO) is used to trade off objectives in choosing a suitable ML model.<n>We propose a novel dynamic optimization approach that prioritizes the most influential hyperparameters based on varying objective trade-offs.
  arXiv  Detail & Related papers  (2026-01-06T16:37:44Z)
• POCAII: Parameter Optimization with Conscious Allocation using Iterative Intelligence [4.478575931884855]
  POCAII is a flexible scheme for managing a hyper parameter optimization budget.<n>It shows superior performance in low-budget hyper parameter optimization regimes.<n>It has wide applications to real-world problems.
  arXiv  Detail & Related papers  (2025-05-16T23:05:07Z)
• Modeling All Response Surfaces in One for Conditional Search Spaces [69.90317997694218]
  This paper proposes a novel approach to model the response surfaces of all subspaces in one.<n>We introduce an attention-based deep feature extractor, capable of projecting configurations with different structures from various subspaces into a unified feature space.
  arXiv  Detail & Related papers  (2025-01-08T03:56:06Z)
• HyperQ-Opt: Q-learning for Hyperparameter Optimization [0.0]
  This paper presents a novel perspective on HPO by formulating it as a sequential decision-making problem and leveraging Q-learning, a reinforcement learning technique.<n>The approaches are evaluated for their ability to find optimal or near-optimal configurations within a limited number of trials.<n>By shifting the paradigm toward policy-based optimization, this work contributes to advancing HPO methods for scalable and efficient machine learning applications.
  arXiv  Detail & Related papers  (2024-12-23T18:22:34Z)
• Hyperparameter Importance Analysis for Multi-Objective AutoML [14.336028105614824]
  In this paper, we propose the first method for assessing the importance of hyper parameters in multi-objective optimization tasks.<n>Specifically, we compute the a-priori scalarization of the objectives and determine the importance of the hyper parameters for different objective tradeoffs.
  arXiv  Detail & Related papers  (2024-05-13T11:00:25Z)
• Parameter Optimization with Conscious Allocation (POCA) [4.478575931884855]
  Hyperband-based approaches to machine learning are among the most effective. We present. the new. Optimization with Conscious Allocation (POCA), a hyperband-based algorithm that adaptively allocates the inputted. budget to the hyperparameter configurations it generates. POCA finds strong configurations faster in both settings.
  arXiv  Detail & Related papers  (2023-12-29T00:13:55Z)
• Hyperparameter Adaptive Search for Surrogate Optimization: A Self-Adjusting Approach [1.6317061277457001]
  Surrogate optimization (SO) algorithms have shown promise for optimizing expensive black-box functions. Our approach identifies and modifies the most influential hyper parameters specific to each problem and SO approach. Experimental results demonstrate the effectiveness of HASSO in enhancing the performance of various SO algorithms.
  arXiv  Detail & Related papers  (2023-10-12T01:26:05Z)
• Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learning [65.51668094117802]
  We propose a human-centered interactive HPO approach tailored towards multi-objective machine learning (ML) Instead of relying on the user guessing the most suitable indicator for their needs, our approach automatically learns an appropriate indicator.
  arXiv  Detail & Related papers  (2023-09-07T09:22:05Z)
• PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning [49.92394599459274]
  We propose PriorBand, an HPO algorithm tailored to Deep Learning (DL) pipelines. We show its robustness across a range of DL benchmarks and show its gains under informative expert input and against poor expert beliefs.
  arXiv  Detail & Related papers  (2023-06-21T16:26:14Z)
• Deep Ranking Ensembles for Hyperparameter Optimization [9.453554184019108]
  We present a novel method that meta-learns neural network surrogates optimized for ranking the configurations' performances while modeling their uncertainty via ensembling. In a large-scale experimental protocol comprising 12 baselines, 16 HPO search spaces and 86 datasets/tasks, we demonstrate that our method achieves new state-of-the-art results in HPO.
  arXiv  Detail & Related papers  (2023-03-27T13:52:40Z)
• A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization [57.450449884166346]
  We propose an adaptive HPO method to account for the privacy cost of HPO. We obtain state-of-the-art performance on 22 benchmark tasks, across computer vision and natural language processing, across pretraining and finetuning.
  arXiv  Detail & Related papers  (2022-12-08T18:56:37Z)
• Towards Learning Universal Hyperparameter Optimizers with Transformers [57.35920571605559]
  We introduce the OptFormer, the first text-based Transformer HPO framework that provides a universal end-to-end interface for jointly learning policy and function prediction. Our experiments demonstrate that the OptFormer can imitate at least 7 different HPO algorithms, which can be further improved via its function uncertainty estimates.
  arXiv  Detail & Related papers  (2022-05-26T12:51:32Z)
• AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuning [72.54359545547904]
  We propose a gradient-based subset selection framework for hyper- parameter tuning. We show that using gradient-based data subsets for hyper- parameter tuning achieves significantly faster turnaround times and speedups of 3$times$-30$times$.
  arXiv  Detail & Related papers  (2022-03-15T19:25:01Z)
• Towards Robust and Automatic Hyper-Parameter Tunning [39.04604349338802]
  We introduce a new class of HPO method and explore how the low-rank factorization of intermediate layers of a convolutional network can be used to define an analytical response surface. We quantify how this surface behaves as a surrogate to model performance and can be solved using a trust-region search algorithm, which we call autoHyper.
  arXiv  Detail & Related papers  (2021-11-28T05:27:34Z)
• Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges [5.139260825952818]
  This paper reviews important HPO methods such as grid or random search, evolutionary algorithms, Bayesian optimization, Hyperband and racing. It gives practical recommendations regarding important choices to be made when conducting HPO, including the HPO algorithms themselves, performance evaluation, how to combine HPO with ML pipelines, runtime improvements, and parallelization.
  arXiv  Detail & Related papers  (2021-07-13T04:55:47Z)
• HyperNP: Interactive Visual Exploration of Multidimensional Projection Hyperparameters [61.354362652006834]
  HyperNP is a scalable method that allows for real-time interactive exploration of projection methods by training neural network approximations. We evaluate the performance of the HyperNP across three datasets in terms of performance and speed.
  arXiv  Detail & Related papers  (2021-06-25T17:28:14Z)
• Optimizing Large-Scale Hyperparameters via Automated Learning Algorithm [97.66038345864095]
  We propose a new hyperparameter optimization method with zeroth-order hyper-gradients (HOZOG) Specifically, we first formulate hyperparameter optimization as an A-based constrained optimization problem. Then, we use the average zeroth-order hyper-gradients to update hyper parameters.
  arXiv  Detail & Related papers  (2021-02-17T21:03:05Z)
• HyperSTAR: Task-Aware Hyperparameters for Deep Networks [52.50861379908611]
  HyperSTAR is a task-aware method to warm-start HPO for deep neural networks. It learns a dataset (task) representation along with the performance predictor directly from raw images. It evaluates 50% less configurations to achieve the best performance compared to existing methods.
  arXiv  Detail & Related papers  (2020-05-21T08:56:50Z)
• Automatic Hyper-Parameter Optimization Based on Mapping Discovery from Data to Hyper-Parameters [3.37314595161109]
  We propose an efficient automatic parameter optimization approach, which is based on the mapping from data to the corresponding hyper- parameters. We show that the proposed approaches outperform the state-of-the-art apporaches significantly.
  arXiv  Detail & Related papers  (2020-03-03T19:26:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.