Analyzing design principles for competitive evolution strategies in constrained search spaces

• URL: http://arxiv.org/abs/2405.05005v1
• Date: Wed, 8 May 2024 12:20:10 GMT
• Title: Analyzing design principles for competitive evolution strategies in constrained search spaces
• Authors: Michael Hellwig, Hans-Georg Beyer,
• Abstract summary: In the context of the 2018 IEEE Congress of Evolutionary Computation, the Matrix Adaptation Evolution Strategy for constrained optimization was notably successful. The $epsilon$MAg-ES algorithm can be considered to be the most successful participant in high dimensions. This paper presents an empirical analysis of the $epsilon$MAg-ES working principles that is expected to provide insights about the performance contribution of specific algorithmic components.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the context of the 2018 IEEE Congress of Evolutionary Computation, the Matrix Adaptation Evolution Strategy for constrained optimization turned out to be notably successful in the competition on constrained single objective real-parameter optimization. Across all considered instances the so-called $\epsilon$MAg-ES achieved the second rank. However, it can be considered to be the most successful participant in high dimensions. Unfortunately, the competition result does not provide any information about the modus operandi of a successful algorithm or its suitability for problems of a particular shape. To this end, the present paper is concerned with an extensive empirical analysis of the $\epsilon$MAg-ES working principles that is expected to provide insights about the performance contribution of specific algorithmic components. To avoid rankings with respect to insignificant differences within the algorithm realizations, the paper additionally introduces significance testing into the ranking process.

Related papers

• Optimization is Not Enough: Why Problem Formulation Deserves Equal Attention [1.6516446394328081]
  Black-box optimization is increasingly used in engineering design problems where simulation-based evaluations are costly and gradients are unavailable.<n>We show that context-agnostic strategies consistently lead to suboptimal or non-physical designs.<n>We motivate the development of new black-box benchmarks that reward physically informed and context-aware optimization strategies.
  arXiv  Detail & Related papers  (2026-02-05T09:15:19Z)
• Principled Algorithms for Optimizing Generalized Metrics in Binary Classification [53.604375124674796]
  We introduce principled algorithms for optimizing generalized metrics, supported by $H$-consistency and finite-sample generalization bounds.<n>Our approach reformulates metric optimization as a generalized cost-sensitive learning problem.<n>We develop new algorithms, METRO, with strong theoretical performance guarantees.
  arXiv  Detail & Related papers  (2025-12-29T01:33:42Z)
• NDCG-Consistent Softmax Approximation with Accelerated Convergence [67.10365329542365]
  We propose novel loss formulations that align directly with ranking metrics.<n>We integrate the proposed RG losses with the highly efficient Alternating Least Squares (ALS) optimization method.<n> Empirical evaluations on real-world datasets demonstrate that our approach achieves comparable or superior ranking performance.
  arXiv  Detail & Related papers  (2025-06-11T06:59:17Z)
• Learning-Augmented Online Bipartite Fractional Matching [7.721815606607016]
  We study online bipartite fractional matching when the algorithm is given advice in the form of a suggested matching in each iteration.<n>Our algorithms dominate the naive "coin flip" strategy of randomly choosing between the advice-following and advice-free algorithms.<n>We establish a hardness bound on the robustness-consistency tradeoff that is attainable by any algorithm.
  arXiv  Detail & Related papers  (2025-05-25T18:15:29Z)
• Composite Indicator-Guided Infilling Sampling for Expensive Multi-Objective Optimization [17.290389254217565]
  We propose a composite indicator-based evolutionary algorithm (CI-EMO) for expensive multi-objective optimization. We design a novel composite performance indicator to guide the selection of candidates for real fitness evaluation. The proposed algorithm outperforms five state-of-the-art expensive multi-objective optimization algorithms.
  arXiv  Detail & Related papers  (2025-03-28T08:15:58Z)
• Understanding Inverse Reinforcement Learning under Overparameterization: Non-Asymptotic Analysis and Global Optimality [52.906438147288256]
  We show that our algorithm can identify the globally optimal reward and policy under certain neural network structures. This is the first IRL algorithm with a non-asymptotic convergence guarantee that provably achieves global optimality.
  arXiv  Detail & Related papers  (2025-03-22T21:16:08Z)
• A Quantum Genetic Algorithm Framework for the MaxCut Problem [49.59986385400411]
  The proposed method introduces a Quantum Genetic Algorithm (QGA) using a Grover-based evolutionary framework and divide-and-conquer principles. On complete graphs, the proposed method consistently achieves the true optimal MaxCut values, outperforming the Semidefinite Programming (SDP) approach. On ErdHos-R'enyi random graphs, the QGA demonstrates competitive performance, achieving median solutions within 92-96% of the SDP results.
  arXiv  Detail & Related papers  (2025-01-02T05:06:16Z)
• Overcoming Brittleness in Pareto-Optimal Learning-Augmented Algorithms [6.131022957085439]
  We propose a new framework in which the smoothness in the performance of the algorithm is enforced by means of a user-specified profile. We apply this new approach to a well-studied online problem, namely the one-way trading problem.
  arXiv  Detail & Related papers  (2024-08-07T23:15:21Z)
• e-COP : Episodic Constrained Optimization of Policies [12.854752753529151]
  We present the first policy optimization algorithm for constrained Reinforcement Learning (RL) in episodic (finite horizon) settings.<n>We show that our algorithm has similar or better performance than SoTA (non-episodic) algorithms adapted for the episodic setting.
  arXiv  Detail & Related papers  (2024-06-13T20:12:09Z)
• A Novel Ranking Scheme for the Performance Analysis of Stochastic Optimization Algorithms using the Principles of Severity [9.310464457958844]
  We provide a novel ranking scheme to rank the algorithms over multiple single-objective optimization problems. The results of the algorithms are compared using a robust bootstrapping-based hypothesis testing procedure.
  arXiv  Detail & Related papers  (2024-05-31T19:35:34Z)
• Analyzing and Overcoming Local Optima in Complex Multi-Objective Optimization by Decomposition-Based Evolutionary Algorithms [5.153202024713228]
  Multi-objective Evolutionary Algorithms (MOEADs) often converge to local optima, limiting solution diversity. We introduce an innovative RP selection strategy, the Vector-Guided Weight-Hybrid method, designed to overcome the local optima issue. Our research comprises two main experimental components: an ablation involving 14 algorithms within the MOEADs framework from 2014 to 2022 to validate our theoretical framework, and a series empirical tests to evaluate the effectiveness of our proposed method against both traditional and cutting-edge alternatives.
  arXiv  Detail & Related papers  (2024-04-12T14:29:45Z)
• An Invariant Information Geometric Method for High-Dimensional Online Optimization [9.538618632613714]
  We introduce a full invariance oriented evolution strategies algorithm, derived from its corresponding framework. We benchmark SynCMA against leading algorithms in Bayesian optimization and evolution strategies. In all scenarios, SynCMA demonstrates great competence, if not dominance, over other algorithms in sample efficiency.
  arXiv  Detail & Related papers  (2024-01-03T07:06:26Z)
• Backpropagation of Unrolled Solvers with Folded Optimization [55.04219793298687]
  The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. One typical strategy is algorithm unrolling, which relies on automatic differentiation through the operations of an iterative solver. This paper provides theoretical insights into the backward pass of unrolled optimization, leading to a system for generating efficiently solvable analytical models of backpropagation.
  arXiv  Detail & Related papers  (2023-01-28T01:50:42Z)
• A Depth-Progressive Initialization Strategy for Quantum Approximate Optimization Algorithm [0.0]
  We first discuss the patterns of optimal parameters in QAOA in two directions. We then discuss on the symmetries and periodicity of the expectation that is used to determine the bounds of the search space. We propose a strategy which predicts the new initial parameters by taking the difference between previous optimal parameters.
  arXiv  Detail & Related papers  (2022-09-22T23:49:11Z)
• Evolving Pareto-Optimal Actor-Critic Algorithms for Generalizability and Stability [67.8426046908398]
  Generalizability and stability are two key objectives for operating reinforcement learning (RL) agents in the real world. This paper presents MetaPG, an evolutionary method for automated design of actor-critic loss functions.
  arXiv  Detail & Related papers  (2022-04-08T20:46:16Z)
• Contextual Model Aggregation for Fast and Robust Federated Learning in Edge Computing [88.76112371510999]
  Federated learning is a prime candidate for distributed machine learning at the network edge. Existing algorithms face issues with slow convergence and/or robustness of performance. We propose a contextual aggregation scheme that achieves the optimal context-dependent bound on loss reduction.
  arXiv  Detail & Related papers  (2022-03-23T21:42:31Z)
• Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator [75.05106948314956]
  We show that an increasing large momentum parameter for the first-order moment is sufficient for adaptive scaling. We also give insights for increasing the momentum in a stagewise manner in accordance with stagewise decreasing step size.
  arXiv  Detail & Related papers  (2021-04-30T08:50:24Z)
• Bilevel Optimization: Convergence Analysis and Enhanced Design [63.64636047748605]
  Bilevel optimization is a tool for many machine learning problems. We propose a novel stoc-efficientgradient estimator named stoc-BiO.
  arXiv  Detail & Related papers  (2020-10-15T18:09:48Z)
• Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization [71.03797261151605]
  Adaptivity is an important yet under-studied property in modern optimization theory. Our algorithm is proved to achieve the best-available convergence for non-PL objectives simultaneously while outperforming existing algorithms for PL objectives.
  arXiv  Detail & Related papers  (2020-02-13T05:42:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.