Distributed Evolution Strategies with Multi-Level Learning for
Large-Scale Black-Box Optimization
- URL: http://arxiv.org/abs/2310.05377v3
- Date: Thu, 2 Nov 2023 08:56:02 GMT
- Title: Distributed Evolution Strategies with Multi-Level Learning for
Large-Scale Black-Box Optimization
- Authors: Qiqi Duan and Chang Shao and Guochen Zhou and Minghan Zhang and Qi
Zhao and Yuhui Shi
- Abstract summary: We propose to parallelize the well-established covariance matrix adaptation evolution strategy (CMA-ES) and in particular its one latest LSO variant called limited-memory CMA-ES (LM-CMA)
We present a multilevel learning-based meta-framework for distributed LM-CMA. Owing to its hierarchically organized structure, Meta-ES is well-suited to implement our distributed meta-framework.
- Score: 14.570608891347446
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In the post-Moore era, main performance gains of black-box optimizers are
increasingly depending on parallelism, especially for large-scale optimization
(LSO). Here we propose to parallelize the well-established covariance matrix
adaptation evolution strategy (CMA-ES) and in particular its one latest LSO
variant called limited-memory CMA-ES (LM-CMA). To achieve efficiency while
approximating its powerful invariance property, we present a multilevel
learning-based meta-framework for distributed LM-CMA. Owing to its
hierarchically organized structure, Meta-ES is well-suited to implement our
distributed meta-framework, wherein the outer-ES controls strategy parameters
while all parallel inner-ESs run the serial LM-CMA with different settings. For
the distribution mean update of the outer-ES, both the elitist and
multi-recombination strategy are used in parallel to avoid stagnation and
regression, respectively. To exploit spatiotemporal information, the global
step-size adaptation combines Meta-ES with the parallel cumulative step-size
adaptation. After each isolation time, our meta-framework employs both the
structure and parameter learning strategy to combine aligned evolution paths
for CMA reconstruction. Experiments on a set of large-scale benchmarking
functions with memory-intensive evaluations, arguably reflecting many
data-driven optimization problems, validate the benefits (e.g., effectiveness
w.r.t. solution quality, and adaptability w.r.t. second-order learning) and
costs of our meta-framework.
Related papers
- Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark [166.40879020706151]
This paper proposes a shift towards BP-free, zeroth-order (ZO) optimization as a solution for reducing memory costs during fine-tuning.
Unlike traditional ZO-SGD methods, our work expands the exploration to a wider array of ZO optimization techniques.
Our study unveils previously overlooked optimization principles, highlighting the importance of task alignment, the role of the forward gradient method, and the balance between algorithm complexity and fine-tuning performance.
arXiv Detail & Related papers (2024-02-18T14:08:48Z) - An Invariant Information Geometric Method for High-Dimensional Online
Optimization [9.538618632613714]
We introduce a full invariance oriented evolution strategies algorithm, derived from its corresponding framework.
We benchmark SynCMA against leading algorithms in Bayesian optimization and evolution strategies.
In all scenarios, SynCMA demonstrates great competence, if not dominance, over other algorithms in sample efficiency.
arXiv Detail & Related papers (2024-01-03T07:06:26Z) - BiERL: A Meta Evolutionary Reinforcement Learning Framework via Bilevel
Optimization [34.24884427152513]
We propose a general meta ERL framework via bilevel optimization (BiERL)
We design an elegant meta-level architecture that embeds the inner-level's evolving experience into an informative population representation.
We perform extensive experiments in MuJoCo and Box2D tasks to verify that as a general framework, BiERL outperforms various baselines and consistently improves the learning performance for a diversity of ERL algorithms.
arXiv Detail & Related papers (2023-08-01T09:31:51Z) - Late Fusion Multi-view Clustering via Global and Local Alignment
Maximization [61.89218392703043]
Multi-view clustering (MVC) optimally integrates complementary information from different views to improve clustering performance.
Most of existing approaches directly fuse multiple pre-specified similarities to learn an optimal similarity matrix for clustering.
We propose late fusion MVC via alignment to address these issues.
arXiv Detail & Related papers (2022-08-02T01:49:31Z) - Deep Reinforcement Learning for Exact Combinatorial Optimization:
Learning to Branch [13.024115985194932]
We propose a new approach for solving the data labeling and inference issues in optimization based on the use of the reinforcement learning (RL) paradigm.
We use imitation learning to bootstrap an RL agent and then use Proximal Policy (PPO) to further explore global optimal actions.
arXiv Detail & Related papers (2022-06-14T16:35:58Z) - Evolving Pareto-Optimal Actor-Critic Algorithms for Generalizability and
Stability [67.8426046908398]
Generalizability and stability are two key objectives for operating reinforcement learning (RL) agents in the real world.
This paper presents MetaPG, an evolutionary method for automated design of actor-critic loss functions.
arXiv Detail & Related papers (2022-04-08T20:46:16Z) - Meta-Learning with Neural Tangent Kernels [58.06951624702086]
We propose the first meta-learning paradigm in the Reproducing Kernel Hilbert Space (RKHS) induced by the meta-model's Neural Tangent Kernel (NTK)
Within this paradigm, we introduce two meta-learning algorithms, which no longer need a sub-optimal iterative inner-loop adaptation as in the MAML framework.
We achieve this goal by 1) replacing the adaptation with a fast-adaptive regularizer in the RKHS; and 2) solving the adaptation analytically based on the NTK theory.
arXiv Detail & Related papers (2021-02-07T20:53:23Z) - A Nested Bi-level Optimization Framework for Robust Few Shot Learning [10.147225934340877]
NestedMAML learns to assign weights to training tasks or instances.
Experiments on synthetic and real-world datasets demonstrate that NestedMAML efficiently mitigates the effects of "unwanted" tasks or instances.
arXiv Detail & Related papers (2020-11-13T06:41:22Z) - On the Global Optimality of Model-Agnostic Meta-Learning [133.16370011229776]
Model-a meta-learning (MAML) formulates meta-learning as a bilevel optimization problem, where the inner level solves each subtask based on a shared prior.
We characterize optimality of the stationary points attained by MAML for both learning and supervised learning, where the inner-level outer-level problems are solved via first-order optimization methods.
arXiv Detail & Related papers (2020-06-23T17:33:14Z) - Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning [63.64636047748605]
We develop a new theoretical framework to provide convergence guarantee for the general multi-step MAML algorithm.
In particular, our results suggest that an inner-stage step needs to be chosen inversely proportional to $N$ of inner-stage steps in order for $N$ MAML to have guaranteed convergence.
arXiv Detail & Related papers (2020-02-18T19:17:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.