Which Hyperparameters Matter? A Game-Theoretic Framework for Interpretable Hyperparameter Sensitivity Analysis

• URL: http://arxiv.org/abs/2607.15884v1
• Date: Fri, 17 Jul 2026 11:58:35 GMT
• Title: Which Hyperparameters Matter? A Game-Theoretic Framework for Interpretable Hyperparameter Sensitivity Analysis
• Abstract summary: This work presents a game-theoretic framework for interpretable hyperparameter-objective interaction analysis.<n>The proposed framework provides interpretable insights into objective-aware hyperparameter interactions, enabling practitioners to guide subsequent optimization and reduce the search space.<n>The effectiveness of the proposed framework is demonstrated using three distinct neural network architectures across different problem domains under multi-objective settings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work presents a game-theoretic framework for interpretable hyperparameter-objective interaction analysis rather than proposing a new optimization algorithm. In the proposed framework, Shapley Effects are employed for global sensitivity analysis, while Pareto front sets are utilized to identify effective hyperparameter configurations and support early-stage model evaluation. The resulting analysis reveals which players (hyperparameters) are most influential with respect to different objectives in a given game (application). Consequently, the proposed framework provides interpretable insights into objective-aware hyperparameter interactions, enabling practitioners to guide subsequent optimization, reduce the search space, and perform early-stage model evaluation. The effectiveness of the proposed framework is demonstrated using three distinct neural network architectures across different problem domains under multi-objective settings.

Related papers

• Joint Surrogate Learning of Objectives, Constraints, and Sensitivities for Efficient Multi-objective Optimization of Neural Dynamical Systems [5.875902463027425]
  Biophysical neural system simulations are among the most computationally demanding scientific applications.<n>We introduce DMOSOPT, a scalable optimization framework that leverages a unified, jointly learned surrogate model.<n>While motivated by and demonstrated in the context of computational neuroscience, the framework is general and applicable to constrained multi-objective optimization problems.
  arXiv  Detail & Related papers  (2026-03-22T00:06:28Z)
• Structural bias in multi-objective optimisation [5.869782631289138]
  Structural bias (SB) refers to systematic preferences of an optimisation algorithm for particular regions of the search space.<n> SB is studied extensively in single-objective optimisation, but its role in multi-objective optimisation remains largely unexplored.<n>In this paper, we extend the concept of structural bias to the multi-objective setting and propose a methodology to study it in isolation from fitness-driven guidance.
  arXiv  Detail & Related papers  (2026-02-06T14:43:28Z)
• Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarios [54.58186816693791]
  environments constantly change over time and space, posing significant challenges for object detectors trained based on a closed-set assumption.<n>We propose a new mechanism, converting the fine-tuning process to a specific- parameter generation.<n>In particular, we first design a dual-path LoRA-based domain-aware adapter that disentangles features into domain-invariant and domain-specific components.
  arXiv  Detail & Related papers  (2025-06-30T17:14:12Z)
• Tuning for Trustworthiness -- Balancing Performance and Explanation Consistency in Neural Network Optimization [49.567092222782435]
  We introduce the novel concept of XAI consistency, defined as the agreement among different feature attribution methods.<n>We create a multi-objective optimization framework that balances predictive performance with explanation.<n>Our research provides a foundation for future investigations into whether models from the trade-off zone-balancing performance loss and XAI consistency-exhibit greater robustness.
  arXiv  Detail & Related papers  (2025-05-12T13:19:14Z)
• Scaling Exponents Across Parameterizations and Optimizers [94.54718325264218]
  We propose a new perspective on parameterization by investigating a key assumption in prior work. Our empirical investigation includes tens of thousands of models trained with all combinations of threes. We find that the best learning rate scaling prescription would often have been excluded by the assumptions in prior work.
  arXiv  Detail & Related papers  (2024-07-08T12:32:51Z)
• Hyperparameter Importance Analysis for Multi-Objective AutoML [14.336028105614824]
  In this paper, we propose the first method for assessing the importance of hyper parameters in multi-objective optimization tasks.<n>Specifically, we compute the a-priori scalarization of the objectives and determine the importance of the hyper parameters for different objective tradeoffs.
  arXiv  Detail & Related papers  (2024-05-13T11:00:25Z)
• Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learning [65.51668094117802]
  We propose a human-centered interactive HPO approach tailored towards multi-objective machine learning (ML) Instead of relying on the user guessing the most suitable indicator for their needs, our approach automatically learns an appropriate indicator.
  arXiv  Detail & Related papers  (2023-09-07T09:22:05Z)
• Interaction-Aware Sensitivity Analysis for Aerodynamic Optimization Results using Information Theory [0.07614628596146601]
  An important issue during an engineering design process is to develop an understanding which design parameters have the most influence on the performance. We propose to use recently introduced information-theoretic methods and estimation algorithms to find the most influential input parameters in optimization results.
  arXiv  Detail & Related papers  (2021-12-10T15:41:56Z)
• Guided Hyperparameter Tuning Through Visualization and Inference [12.035299005299306]
  We present a streamlined visualization system enabling deep learning practitioners to more efficiently explore, tune, and optimize hyper parameters. A key idea is to directly suggest more optimal hyper parameters using a predictive mechanism. We evaluate the tool with a user study on deep learning model builders, finding that our participants have little issue adopting the tool and working with it as part of their workflow.
  arXiv  Detail & Related papers  (2021-05-24T19:55:24Z)
• An Empirical Study of Assumptions in Bayesian Optimisation [61.19427472792523]
  In this work we rigorously analyse conventional and non-conventional assumptions inherent to Bayesian optimisation. We conclude that the majority of hyper- parameter tuning tasks exhibit heteroscedasticity and non-stationarity. We hope these findings may serve as guiding principles, both for practitioners and for further research in the field.
  arXiv  Detail & Related papers  (2020-12-07T16:21:12Z)
• An Asymptotically Optimal Multi-Armed Bandit Algorithm and Hyperparameter Optimization [48.5614138038673]
  We propose an efficient and robust bandit-based algorithm called Sub-Sampling (SS) in the scenario of hyper parameter search evaluation. We also develop a novel hyper parameter optimization algorithm called BOSS. Empirical studies validate our theoretical arguments of SS and demonstrate the superior performance of BOSS on a number of applications.
  arXiv  Detail & Related papers  (2020-07-11T03:15:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.