An Evolutionary Algorithm Assisted by an Ensemble of Pareto-Optimal Surrogate Models

• URL: http://arxiv.org/abs/2608.01777v1
• Date: Mon, 03 Aug 2026 06:49:00 GMT
• Title: An Evolutionary Algorithm Assisted by an Ensemble of Pareto-Optimal Surrogate Models
• Abstract summary: An ensemble of surrogate models helps improve the prediction quality and robustness of surrogate models, and in turn, the search performance of surrogate-assisted evolutionary algorithms (SAEAs)<n>This study proposes an adaptive ensemble SAEA, which automatically constructs plausible ensemble models by optimizing their parameter settings.<n>The experimental results demonstrated the statistical superiority of our algorithm over state-of-the-art SAEAs on a single-objective benchmark and real-world problem sets.
• Score: 21.207177069711857
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: An ensemble of surrogate models helps improve the prediction quality and robustness of surrogate models, and in turn, the search performance of surrogate-assisted evolutionary algorithms (SAEAs). Although different degrees of smoothness of the approximated fitness landscapes need to be carefully designed for an effective ensemble, little attention has been paid to the explicit tuning of the degree of smoothness derived by surrogate models. This study proposes an adaptive ensemble SAEA, which automatically constructs plausible ensemble models by optimizing their parameter settings. Unlike existing adaptive/ensemble SAEAs, which consider prediction accuracy alone, the proposed algorithm optimizes the structure of radial basis function networks (RBFNs) by solving bi-objective minimization problems of approximation error and model complexity, resulting in robust ensemble models of accurate surrogate models with different degrees of smoothness of the approximated fitness landscapes. As a result, the over/under-fittings are reduced. Additionally, an infill criterion is designed so that surrogate models with different degrees of smoothness can contribute to the solution prescreening. The experimental results demonstrated the statistical superiority of our algorithm over state-of-the-art SAEAs on a single-objective benchmark and real-world problem sets under an expensive optimization scenario. The source code of the proposed algorithm is available at https://github.com/haranychan/EPOS

Related papers

• Opt-Verifier: Unleashing the Power of LLMs for Optimization Modeling via Dual-Side Verification [53.763212981479455]
  We propose a novel framework with Dual-side Verification (Opt-Verifier) from both structure and solution perspectives.<n> Experiments on popular benchmarks demonstrate that our approach achieves over 20% improvement in accuracy.
  arXiv  Detail & Related papers  (2026-05-28T08:09:52Z)
• New Hybrid Fine-Tuning Paradigm for LLMs: Algorithm Design and Convergence Analysis Framework [52.69678300445233]
  Fine-tuning Large Language Models (LLMs) typically involves either full fine-tuning, which updates all model parameters, or.<n>Efficient Fine-Tuning (PEFT) adjusts a small subset of parameters.<n>We propose a novel hybrid fine-tuning approach that jointly updates both LLMs and PEFT modules using a combination of zeroth-order and first-order optimization methods.
  arXiv  Detail & Related papers  (2026-04-10T22:39:38Z)
• Simulating classification models to evaluate Predict-Then-Optimize methods [0.0]
  Uncertainty in optimization is often represented as parameters in the optimization model.<n>In Predict-Then-trivial approaches, predictions of a machine learning model are used as values for such parameters.<n>We show that the relationship between prediction error and how close solutions are to the actual is non-experimental.
  arXiv  Detail & Related papers  (2025-09-02T11:05:27Z)
• Composition and Alignment of Diffusion Models using Constrained Learning [79.36736636241564]
  Diffusion models have become prevalent in generative modeling due to their ability to sample from complex distributions.<n>Two commonly used methods are: (i) alignment, which involves fine-tuning a diffusion model to align it with a reward; and (ii) composition, which combines several pre-trained diffusion models, each emphasizing a desirable attribute in the generated outputs.<n>We propose a constrained optimization framework that unifies alignment and composition of diffusion models by enforcing that the aligned model satisfies reward constraints and/or remains close to (potentially multiple) pre-trained models.
  arXiv  Detail & Related papers  (2025-08-26T15:06:30Z)
• Divergence Minimization Preference Optimization for Diffusion Model Alignment [66.31417479052774]
  Divergence Minimization Preference Optimization (DMPO) is a principled method for aligning diffusion models by minimizing reverse KL divergence.<n>DMPO can consistently outperform or match existing techniques across different base models and test sets.
  arXiv  Detail & Related papers  (2025-07-10T07:57:30Z)
• A Comparison-Relationship-Surrogate Evolutionary Algorithm for Multi-Objective Optimization [0.0]
  We propose a new evolutionary algorithm "CRSEA" which uses the comparison-relationship model.<n>We find that CRSEA finds better converged solutions than the tested SAEAs on many medium-scale, biobjective problems.
  arXiv  Detail & Related papers  (2025-04-28T01:39:38Z)
• Comparative study of regression vs pairwise models for surrogate-based heuristic optimisation [1.2535250082638645]
  This paper addresses the formulation of surrogate problems as both regression models that approximate fitness (surface surrogate models) and a novel way to connect classification models (pairwise surrogate models) The performance of the overall search, when using online machine learning-based surrogate models, depends not only on the accuracy of the predictive model but also on the kind of bias towards positive or negative cases.
  arXiv  Detail & Related papers  (2024-10-04T13:19:06Z)
• Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference [47.460898983429374]
  We introduce an ensemble Kalman filter (EnKF) into the non-mean-field (NMF) variational inference framework to approximate the posterior distribution of the latent states. This novel marriage between EnKF and GPSSM not only eliminates the need for extensive parameterization in learning variational distributions, but also enables an interpretable, closed-form approximation of the evidence lower bound (ELBO) We demonstrate that the resulting EnKF-aided online algorithm embodies a principled objective function by ensuring data-fitting accuracy while incorporating model regularizations to mitigate overfitting.
  arXiv  Detail & Related papers  (2023-12-10T15:22:30Z)
• Comparative Evaluation of Metaheuristic Algorithms for Hyperparameter Selection in Short-Term Weather Forecasting [0.0]
  This paper explores the application of metaheuristic algorithms, namely Genetic Algorithm (GA), Differential Evolution (DE) and Particle Swarm Optimization (PSO) We evaluate their performance in weather forecasting based on metrics such as Mean Squared Error (MSE) and Mean Absolute Percentage Error (MAPE)
  arXiv  Detail & Related papers  (2023-09-05T22:13:35Z)
• Modeling the Second Player in Distributionally Robust Optimization [90.25995710696425]
  We argue for the use of neural generative models to characterize the worst-case distribution. This approach poses a number of implementation and optimization challenges. We find that the proposed approach yields models that are more robust than comparable baselines.
  arXiv  Detail & Related papers  (2021-03-18T14:26:26Z)
• Automatically Learning Compact Quality-aware Surrogates for Optimization Problems [55.94450542785096]
  Solving optimization problems with unknown parameters requires learning a predictive model to predict the values of the unknown parameters and then solving the problem using these values. Recent work has shown that including the optimization problem as a layer in a complex training model pipeline results in predictions of iteration of unobserved decision making. We show that we can improve solution quality by learning a low-dimensional surrogate model of a large optimization problem.
  arXiv  Detail & Related papers  (2020-06-18T19:11:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.