A Deep Dive into Effects of Structural Bias on CMA-ES Performance along Affine Trajectories

• URL: http://arxiv.org/abs/2404.17323v1
• Date: Fri, 26 Apr 2024 11:07:09 GMT
• Title: A Deep Dive into Effects of Structural Bias on CMA-ES Performance along Affine Trajectories
• Abstract summary: This study explores the impact of structural bias in the modular Comodvariance Matrix Adaptation Evolution Strategy (modCMA) Through an investigation involving 435,456 configurations of modCMA, we identified key modules that significantly influence structural bias of various classes. Our results demonstrate an interplay between module-induced structural bias and algorithm performance across different landscape characteristics.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To guide the design of better iterative optimisation heuristics, it is imperative to understand how inherent structural biases within algorithm components affect the performance on a wide variety of search landscapes. This study explores the impact of structural bias in the modular Covariance Matrix Adaptation Evolution Strategy (modCMA), focusing on the roles of various modulars within the algorithm. Through an extensive investigation involving 435,456 configurations of modCMA, we identified key modules that significantly influence structural bias of various classes. Our analysis utilized the Deep-BIAS toolbox for structural bias detection and classification, complemented by SHAP analysis for quantifying module contributions. The performance of these configurations was tested on a sequence of affine-recombined functions, maintaining fixed optimum locations while gradually varying the landscape features. Our results demonstrate an interplay between module-induced structural bias and algorithm performance across different landscape characteristics.

Related papers

• Evolution-Aware MSA Reasoning for Subsampling via Factor Graphs [55.715754117956514]
  Multiple Sequence Alignments provide protein language models with explicit evolutionary context.<n>MSA subsampling is unavoidable under limited token budgets.<n>We introduce AP-REASONER, which infers a fixed-budget MSA subset using factor-graph reasoning.
  arXiv  Detail & Related papers  (2026-07-24T13:55:21Z)
• RooflineBench: A Benchmarking Framework for On-Device LLMs via Roofline Analysis [53.90240071275054]
  The transition toward localized intelligence through Small Language Models (SLMs) has intensified the need for rigorous performance characterization on resource-constrained edge hardware.<n>We propose a systematic framework that unifies architectural primitives and hardware constraints through the lens of operational intensity (OI)<n>By defining an inference-potential region, we introduce the Relative Inference Potential as a novel metric to compare efficiency differences between Large Language Models (LLMs) on the same hardware substrate.
  arXiv  Detail & Related papers  (2026-02-12T03:02:22Z)
• Structural bias in multi-objective optimisation [5.869782631289138]
  Structural bias (SB) refers to systematic preferences of an optimisation algorithm for particular regions of the search space.<n> SB is studied extensively in single-objective optimisation, but its role in multi-objective optimisation remains largely unexplored.<n>In this paper, we extend the concept of structural bias to the multi-objective setting and propose a methodology to study it in isolation from fitness-driven guidance.
  arXiv  Detail & Related papers  (2026-02-06T14:43:28Z)
• An Integrated Fusion Framework for Ensemble Learning Leveraging Gradient Boosting and Fuzzy Rule-Based Models [59.13182819190547]
  Fuzzy rule-based models excel in interpretability and have seen widespread application across diverse fields.<n>They face challenges such as complex design specifications and scalability issues with large datasets.<n>This paper proposes an Integrated Fusion Framework that merges the strengths of both paradigms to enhance model performance and interpretability.
  arXiv  Detail & Related papers  (2025-11-11T10:28:23Z)
• Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules [23.89617465228557]
  Mixture-of-Experts (MoE) benefits from a dynamic routing mechanism among their specialized experts.<n>We investigate whether adaptation modules themselves should incorporate routing mechanisms to align with MoE's multi-expert architecture.
  arXiv  Detail & Related papers  (2025-08-04T16:43:09Z)
• Tracing the Interactions of Modular CMA-ES Configurations Across Problem Landscapes [2.732041684677653]
  Performance footprints are calculated for six modular variants of the CMA-ES algorithm (modCMA), evaluated on 24 benchmark problems.<n>Our analysis uncovers shared behavioral patterns across configurations due to common interactions with problem properties, as well as distinct behaviors on the same problem driven by differing problem features.
  arXiv  Detail & Related papers  (2025-07-03T05:48:58Z)
• AlphaFold Database Debiasing for Robust Inverse Folding [58.792020809180336]
  We introduce a Debiasing Structure AutoEncoder (DeSAE) that learns to reconstruct native-like conformations from intentionally corrupted backbone geometries.<n>At inference, applying DeSAE to AFDB structures produces debiased structures that significantly improve inverse folding performance.
  arXiv  Detail & Related papers  (2025-06-10T02:25:31Z)
• Model Hemorrhage and the Robustness Limits of Large Language Models [119.46442117681147]
  Large language models (LLMs) demonstrate strong performance across natural language processing tasks, yet undergo significant performance degradation when modified for deployment. We define this phenomenon as model hemorrhage - performance decline caused by parameter alterations and architectural changes.
  arXiv  Detail & Related papers  (2025-03-31T10:16:03Z)
• Cliqueformer: Model-Based Optimization with Structured Transformers [102.55764949282906]
  Large neural networks excel at prediction tasks, but their application to design problems, such as protein engineering or materials discovery, requires solving offline model-based optimization (MBO) problems. We present Cliqueformer, a transformer-based architecture that learns the black-box function's structure through functional graphical models (FGM) Across various domains, including chemical and genetic design tasks, Cliqueformer demonstrates superior performance compared to existing methods.
  arXiv  Detail & Related papers  (2024-10-17T00:35:47Z)
• Generative Sentiment Analysis via Latent Category Distribution and Constrained Decoding [30.05158520307257]
  This study introduces a generative sentiment analysis model. By reconstructing the input of a variational autoencoder, the model learns the intensity of the relationship between categories and text. Experimental results on the Restaurant-ACOS and Laptop-ACOS datasets demonstrate a significant performance improvement.
  arXiv  Detail & Related papers  (2024-07-31T12:29:17Z)
• Representing Topological Self-Similarity Using Fractal Feature Maps for Accurate Segmentation of Tubular Structures [12.038095281876071]
  In this study, we incorporate fractal features into a deep learning model by extending FD to the pixel-level using a sliding window technique. The resulting fractal feature maps (FFMs) are then incorporated as additional input to the model and additional weight in the loss function. Experiments on five tubular structure datasets validate the effectiveness and robustness of our approach.
  arXiv  Detail & Related papers  (2024-07-20T05:22:59Z)
• Embedded feature selection in LSTM networks with multi-objective evolutionary ensemble learning for time series forecasting [49.1574468325115]
  We present a novel feature selection method embedded in Long Short-Term Memory networks. Our approach optimize the weights and biases of the LSTM in a partitioned manner. Experimental evaluations on air quality time series data from Italy and southeast Spain demonstrate that our method substantially improves the ability generalization of conventional LSTMs.
  arXiv  Detail & Related papers  (2023-12-29T08:42:10Z)
• AbCD: A Component-wise Adjustable Framework for Dynamic Optimization Problems [49.1574468325115]
  Dynamic Optimization Problems (DOPs) are characterized by changes in the fitness landscape that can occur at any time and are common in real world applications. We develop a component-oriented framework for DOPs called Adjustable Components for Dynamic Problems (AbCD) Our results highlight existing problems in the DOP field that need to be addressed in the future development of algorithms and components.
  arXiv  Detail & Related papers  (2023-10-09T08:11:31Z)
• Disentangling Structured Components: Towards Adaptive, Interpretable and Scalable Time Series Forecasting [52.47493322446537]
  We develop a adaptive, interpretable and scalable forecasting framework, which seeks to individually model each component of the spatial-temporal patterns. SCNN works with a pre-defined generative process of MTS, which arithmetically characterizes the latent structure of the spatial-temporal patterns. Extensive experiments are conducted to demonstrate that SCNN can achieve superior performance over state-of-the-art models on three real-world datasets.
  arXiv  Detail & Related papers  (2023-05-22T13:39:44Z)
• Automatic Modulation Classification with Deep Neural Networks [0.0]
  We show that a combination of dilated convolutions, statistics pooling, and squeeze-and-excitation units results in the strongest performing classifier. We further investigate this best performer according to various other criteria, including short signal bursts, common misclassifications, and performance across differing modulation categories and modes.
  arXiv  Detail & Related papers  (2023-01-27T15:16:06Z)
• Demystify Transformers & Convolutions in Modern Image Deep Networks [82.32018252867277]
  This paper aims to identify the real gains of popular convolution and attention operators through a detailed study. We find that the key difference among these feature transformation modules, such as attention or convolution, lies in their spatial feature aggregation approach. Our experiments on various tasks and an analysis of inductive bias show a significant performance boost due to advanced network-level and block-level designs.
  arXiv  Detail & Related papers  (2022-11-10T18:59:43Z)
• A Pareto-optimal compositional energy-based model for sampling and optimization of protein sequences [55.25331349436895]
  Deep generative models have emerged as a popular machine learning-based approach for inverse problems in the life sciences. These problems often require sampling new designs that satisfy multiple properties of interest in addition to learning the data distribution.
  arXiv  Detail & Related papers  (2022-10-19T19:04:45Z)
• The Importance of Landscape Features for Performance Prediction of Modular CMA-ES Variants [2.3823600586675724]
  Recent studies show that supervised machine learning methods can predict algorithm performance using landscape features extracted from the problem instances. We consider the modular CMA-ES framework and estimate how much each landscape feature contributes to the best algorithm performance regression models.
  arXiv  Detail & Related papers  (2022-04-15T11:55:28Z)
• Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components [71.03032589756434]
  We investigate the effect of different variation operators in a complex domain, that of multi-network heterogeneous neural models. We characterize both the variation operators, according to their effect on the complexity and performance of the model; and the models, relying on diverse metrics which estimate the quality of the different parts composing it.
  arXiv  Detail & Related papers  (2021-06-16T17:12:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.