From Black-Box Tuning to Guided Optimization via Hyperparameters Interaction Analysis

• URL: http://arxiv.org/abs/2512.19246v1
• Date: Mon, 22 Dec 2025 10:28:22 GMT
• Title: From Black-Box Tuning to Guided Optimization via Hyperparameters Interaction Analysis
• Authors: Moncef Garouani, Ayah Barhrhouj,
• Abstract summary: We introduce MetaSHAP, a scalable semi-automated AI (XAI) method that uses meta-learning and Shapley values analysis to provide actionable tuning insights.<n>We empirically validate MetaSHAP on a diverse benchmark of 164 classification datasets and 14 classifiers.
• Score: 0.5371337604556311
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hyperparameters tuning is a fundamental, yet computationally expensive, step in optimizing machine learning models. Beyond optimization, understanding the relative importance and interaction of hyperparameters is critical to efficient model development. In this paper, we introduce MetaSHAP, a scalable semi-automated eXplainable AI (XAI) method, that uses meta-learning and Shapley values analysis to provide actionable and dataset-aware tuning insights. MetaSHAP operates over a vast benchmark of over 09 millions evaluated machine learning pipelines, allowing it to produce interpretable importance scores and actionable tuning insights that reveal how much each hyperparameter matters, how it interacts with others and in which value ranges its influence is concentrated. For a given algorithm and dataset, MetaSHAP learns a surrogate performance model from historical configurations, computes hyperparameters interactions using SHAP-based analysis, and derives interpretable tuning ranges from the most influential hyperparameters. This allows practitioners not only to prioritize which hyperparameters to tune, but also to understand their directionality and interactions. We empirically validate MetaSHAP on a diverse benchmark of 164 classification datasets and 14 classifiers, demonstrating that it produces reliable importance rankings and competitive performance when used to guide Bayesian optimization.

Related papers

• Dynamic Hyperparameter Importance for Efficient Multi-Objective Optimization [10.530794046739619]
  Multi-objective optimization (MOO) is used to trade off objectives in choosing a suitable ML model.<n>We propose a novel dynamic optimization approach that prioritizes the most influential hyperparameters based on varying objective trade-offs.
  arXiv  Detail & Related papers  (2026-01-06T16:37:44Z)
• Predictable Scale: Part I, Step Law -- Optimal Hyperparameter Scaling Law in Large Language Model Pretraining [59.369484219304866]
  We conduct an unprecedented empirical investigation training over 3,700 Large Language Models (LLMs) from scratch across 100 trillion tokens.<n>We establish a universal Scaling Law for hyperparameter optimization in LLM Pre-training, called Step Law.<n>Our estimated optima deviates from the global best performance found via exhaustive search by merely 0.094% on the test set.
  arXiv  Detail & Related papers  (2025-03-06T18:58:29Z)
• HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimization [17.646452580419545]
  Black-box HPO methods undermine user trust and discourages adoption.<n>We propose a game-theoretic explainability framework for HPO based on Shapley values and interactions.<n>We demonstrate HyperSHAP's capabilities on various HPO benchmarks to analyze the interaction structure of the corresponding HPO problems.
  arXiv  Detail & Related papers  (2025-02-03T11:47:52Z)
• Efficient Hyperparameter Importance Assessment for CNNs [1.7778609937758323]
  This paper aims to quantify the importance weights of some hyperparameters in Convolutional Neural Networks (CNNs) with an algorithm called N-RReliefF. We conduct an extensive study by training over ten thousand CNN models across ten popular image classification datasets.
  arXiv  Detail & Related papers  (2024-10-11T15:47:46Z)
• End-to-End Learning for Fair Multiobjective Optimization Under Uncertainty [55.04219793298687]
  The Predict-Then-Forecast (PtO) paradigm in machine learning aims to maximize downstream decision quality. This paper extends the PtO methodology to optimization problems with nondifferentiable Ordered Weighted Averaging (OWA) objectives. It shows how optimization of OWA functions can be effectively integrated with parametric prediction for fair and robust optimization under uncertainty.
  arXiv  Detail & Related papers  (2024-02-12T16:33:35Z)
• Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learning [65.51668094117802]
  We propose a human-centered interactive HPO approach tailored towards multi-objective machine learning (ML) Instead of relying on the user guessing the most suitable indicator for their needs, our approach automatically learns an appropriate indicator.
  arXiv  Detail & Related papers  (2023-09-07T09:22:05Z)
• HyperTuner: A Cross-Layer Multi-Objective Hyperparameter Auto-Tuning Framework for Data Analytic Services [25.889791254011794]
  We propose HyperTuner to execute cross-layer multi-objective hyperparameter auto-tuning. We show that HyperTuner is superior in both convergence and diversity compared with the other four baseline algorithms. experiments with different training datasets, different optimization objectives and different machine learning platforms verify that HyperTuner can well adapt to various data analytic service scenarios.
  arXiv  Detail & Related papers  (2023-04-20T02:19:10Z)
• Pre-training helps Bayesian optimization too [49.28382118032923]
  We seek an alternative practice for setting functional priors. In particular, we consider the scenario where we have data from similar functions that allow us to pre-train a tighter distribution a priori. Our results show that our method is able to locate good hyper parameters at least 3 times more efficiently than the best competing methods.
  arXiv  Detail & Related papers  (2022-07-07T04:42:54Z)
• Towards Learning Universal Hyperparameter Optimizers with Transformers [57.35920571605559]
  We introduce the OptFormer, the first text-based Transformer HPO framework that provides a universal end-to-end interface for jointly learning policy and function prediction. Our experiments demonstrate that the OptFormer can imitate at least 7 different HPO algorithms, which can be further improved via its function uncertainty estimates.
  arXiv  Detail & Related papers  (2022-05-26T12:51:32Z)
• AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuning [72.54359545547904]
  We propose a gradient-based subset selection framework for hyper- parameter tuning. We show that using gradient-based data subsets for hyper- parameter tuning achieves significantly faster turnaround times and speedups of 3$times$-30$times$.
  arXiv  Detail & Related papers  (2022-03-15T19:25:01Z)
• To tune or not to tune? An Approach for Recommending Important Hyperparameters [2.121963121603413]
  We consider building the relationship between the performance of the machine learning models and their hyperparameters to discover the trend and gain insights. Our results enable users to decide whether it is worth conducting a possibly time-consuming tuning strategy.
  arXiv  Detail & Related papers  (2021-08-30T08:54:58Z)
• Guided Hyperparameter Tuning Through Visualization and Inference [12.035299005299306]
  We present a streamlined visualization system enabling deep learning practitioners to more efficiently explore, tune, and optimize hyper parameters. A key idea is to directly suggest more optimal hyper parameters using a predictive mechanism. We evaluate the tool with a user study on deep learning model builders, finding that our participants have little issue adopting the tool and working with it as part of their workflow.
  arXiv  Detail & Related papers  (2021-05-24T19:55:24Z)
• Rethinking the Hyperparameters for Fine-tuning [78.15505286781293]
  Fine-tuning from pre-trained ImageNet models has become the de-facto standard for various computer vision tasks. Current practices for fine-tuning typically involve selecting an ad-hoc choice of hyper parameters. This paper re-examines several common practices of setting hyper parameters for fine-tuning.
  arXiv  Detail & Related papers  (2020-02-19T18:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.