HINNPerf: Hierarchical Interaction Neural Network for Performance Prediction of Configurable Systems

• URL: http://arxiv.org/abs/2204.03931v1
• Date: Fri, 8 Apr 2022 08:52:23 GMT
• Title: HINNPerf: Hierarchical Interaction Neural Network for Performance Prediction of Configurable Systems
• Authors: Jiezhu Cheng, Cuiyun Gao and Zibin Zheng
• Abstract summary: HINNPerf is a novel hierarchical interaction neural network for performance prediction. HINNPerf employs the embedding method and hierarchic network blocks to model the complicated interplay between configuration options. Empirical results on 10 real-world systems show that our method statistically significantly outperforms state-of-the-art approaches.
• Score: 22.380061796355616
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern software systems are usually highly configurable, providing users with customized functionality through various configuration options. Understanding how system performance varies with different option combinations is important to determine optimal configurations that meet specific requirements. Due to the complex interactions among multiple options and the high cost of performance measurement under a huge configuration space, it is challenging to study how different configurations influence the system performance. To address these challenges, we propose HINNPerf, a novel hierarchical interaction neural network for performance prediction of configurable systems. HINNPerf employs the embedding method and hierarchic network blocks to model the complicated interplay between configuration options, which improves the prediction accuracy of the method. Besides, we devise a hierarchical regularization strategy to enhance the model robustness. Empirical results on 10 real-world configurable systems show that our method statistically significantly outperforms state-of-the-art approaches by achieving average 22.67% improvement in prediction accuracy. In addition, combined with the Integrated Gradients method, the designed hierarchical architecture provides some insights about the interaction complexity and the significance of configuration options, which might help users and developers better understand how the configurable system works and efficiently identify significant options affecting the performance.

Related papers

• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Landscape-Aware Automated Algorithm Configuration using Multi-output Mixed Regression and Classification [0.01649298969786889]
  We investigate the potential of randomly generated functions (RGF) for the model training. We focus on automated algorithm configuration (AAC) We analyze the performance of dense neural network (NN) models in handling the mixed regression and classification tasks.
  arXiv  Detail & Related papers  (2024-09-02T20:04:41Z)
• Principled Architecture-aware Scaling of Hyperparameters [69.98414153320894]
  Training a high-quality deep neural network requires choosing suitable hyperparameters, which is a non-trivial and expensive process. In this work, we precisely characterize the dependence of initializations and maximal learning rates on the network architecture. We demonstrate that network rankings can be easily changed by better training networks in benchmarks.
  arXiv  Detail & Related papers  (2024-02-27T11:52:49Z)
• CURE: Simulation-Augmented Auto-Tuning in Robotics [15.943773140929856]
  This paper proposes CURE -- a method that identifies causally relevant configuration options. CURE abstracts the causal relationships between various configuration options and robot performance objectives. We demonstrate the effectiveness and transferability of CURE by conducting experiments in both physical robots and simulation.
  arXiv  Detail & Related papers  (2024-02-08T04:27:14Z)
• CAMEO: A Causal Transfer Learning Approach for Performance Optimization of Configurable Computer Systems [16.75106122540052]
  We propose CAMEO, a method that identifies invariant causal predictors under environmental changes. We demonstrate significant performance improvements over state-of-the-art optimization methods in MLperf deep learning systems, a video analytics pipeline, and a database system.
  arXiv  Detail & Related papers  (2023-06-13T16:28:37Z)
• AdaEnsemble: Learning Adaptively Sparse Structured Ensemble Network for Click-Through Rate Prediction [0.0]
  We propose AdaEnsemble: a Sparsely-Gated Mixture-of-Experts architecture that can leverage the strengths of heterogeneous feature interaction experts. AdaEnsemble can adaptively choose the feature interaction depth and find the corresponding SparseMoE stacking layer to exit and compute prediction from. We implement the proposed AdaEnsemble and evaluate its performance on real-world datasets.
  arXiv  Detail & Related papers  (2023-01-06T12:08:15Z)
• Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spaces [54.58348769621782]
  Tree ensembles can be well-suited for black-box optimization tasks such as algorithm tuning and neural architecture search. Two well-known challenges in using tree ensembles for black-box optimization are (i) effectively quantifying model uncertainty for exploration and (ii) optimizing over the piece-wise constant acquisition function. Our framework performs as well as state-of-the-art methods for unconstrained black-box optimization over continuous/discrete features and outperforms competing methods for problems combining mixed-variable feature spaces and known input constraints.
  arXiv  Detail & Related papers  (2022-07-02T16:59:37Z)
• Unicorn: Reasoning about Configurable System Performance through the lens of Causality [12.877523121932114]
  We propose a new method, called Unicorn, which captures intricate interactions between configuration options across the software- hardware stack. Experiments indicate that Unicorn outperforms state-of-the-art performance optimization and debug methods. Unlike the existing methods, the learned causal performance models reliably predict performance for new environments.
  arXiv  Detail & Related papers  (2022-01-20T19:16:50Z)
• Rethinking Architecture Selection in Differentiable NAS [74.61723678821049]
  Differentiable Neural Architecture Search is one of the most popular NAS methods for its search efficiency and simplicity. We propose an alternative perturbation-based architecture selection that directly measures each operation's influence on the supernet. We find that several failure modes of DARTS can be greatly alleviated with the proposed selection method.
  arXiv  Detail & Related papers  (2021-08-10T00:53:39Z)
• Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components [71.03032589756434]
  We investigate the effect of different variation operators in a complex domain, that of multi-network heterogeneous neural models. We characterize both the variation operators, according to their effect on the complexity and performance of the model; and the models, relying on diverse metrics which estimate the quality of the different parts composing it.
  arXiv  Detail & Related papers  (2021-06-16T17:12:26Z)
• Deep Multi-Task Learning for Cooperative NOMA: System Design and Principles [52.79089414630366]
  We develop a novel deep cooperative NOMA scheme, drawing upon the recent advances in deep learning (DL) We develop a novel hybrid-cascaded deep neural network (DNN) architecture such that the entire system can be optimized in a holistic manner.
  arXiv  Detail & Related papers  (2020-07-27T12:38:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.