Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2404.08239v1
• Date: Fri, 12 Apr 2024 04:48:32 GMT
• Title: Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learning
• Authors: Zeyuan Ma, Jiacheng Chen, Hongshu Guo, Yining Ma, Yue-Jiao Gong,
• Abstract summary: Evolutionary computation (EC) algorithms leverage a group of individuals to cooperatively search for the optimum. We propose a deep reinforcement learning-based framework that autonomously configures and adapts the exploration tradeoff (EET) throughout the EC search process. Our proposed framework is characterized by its simplicity, effectiveness, and generalizability, with the potential to enhance numerous existing EC algorithms.
• Score: 14.217528205889296
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Evolutionary computation (EC) algorithms, renowned as powerful black-box optimizers, leverage a group of individuals to cooperatively search for the optimum. The exploration-exploitation tradeoff (EET) plays a crucial role in EC, which, however, has traditionally been governed by manually designed rules. In this paper, we propose a deep reinforcement learning-based framework that autonomously configures and adapts the EET throughout the EC search process. The framework allows different individuals of the population to selectively attend to the global and local exemplars based on the current search state, maximizing the cooperative search outcome. Our proposed framework is characterized by its simplicity, effectiveness, and generalizability, with the potential to enhance numerous existing EC algorithms. To validate its capabilities, we apply our framework to several representative EC algorithms and conduct extensive experiments on the augmented CEC2021 benchmark. The results demonstrate significant improvements in the performance of the backbone algorithms, as well as favorable generalization across diverse problem classes, dimensions, and population sizes. Additionally, we provide an in-depth analysis of the EET issue by interpreting the learned behaviors of EC.

Related papers

• Integrating Chaotic Evolutionary and Local Search Techniques in Decision Space for Enhanced Evolutionary Multi-Objective Optimization [1.8130068086063336]
  This paper focuses on both Single-Objective Multi-Modal Optimization (SOMMOP) and Multi-Objective Optimization (MOO) In SOMMOP, we integrate chaotic evolution with niching techniques, as well as Persistence-Based Clustering combined with Gaussian mutation. For MOO, we extend these methods into a comprehensive framework that incorporates Uncertainty-Based Selection, Adaptive Tuning, and introduces a radius ( R ) concept in deterministic crowding.
  arXiv  Detail & Related papers  (2024-11-12T15:18:48Z)
• Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization [16.49696895887536]
  Dynamic Algorithm configuration (DAC) addresses the challenge of dynamically setting hyperparameters of an algorithm for a diverse set of instances. Agents trained with Deep Reinforcement Learning (RL) offer a pathway to solve such settings. We take a step towards mitigating this by selecting a representative subset of training instances to overcome overrepresentation and then retraining the agent on this subset to improve its generalization performance.
  arXiv  Detail & Related papers  (2024-07-18T13:44:43Z)
• Evaluating Ensemble Methods for News Recommender Systems [50.90330146667386]
  This paper demonstrates how ensemble methods can be used to combine many diverse state-of-the-art algorithms to achieve superior results on the Microsoft News dataset (MIND) Our findings demonstrate that a combination of NRS algorithms can outperform individual algorithms, provided that the base learners are sufficiently diverse.
  arXiv  Detail & Related papers  (2024-06-23T13:40:50Z)
• Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control [1.1404490220482764]
  BRO is a model-free algorithm to achieve near-optimal policies in the Dog and Humanoid tasks. BRO achieves state-of-the-art results, significantly outperforming the leading model-based and model-free algorithms. BRO is the first model-free algorithm to achieve near-optimal policies in the notoriously challenging Dog and Humanoid tasks.
  arXiv  Detail & Related papers  (2024-05-25T09:53:25Z)
• Constrained Multi-objective Optimization with Deep Reinforcement Learning Assisted Operator Selection [28.088046969822543]
  This work proposes an online operator selection framework assisted by Deep Reinforcement Learning. The proposed approach can adaptively select an operator that maximizes the improvement of the population according to the current state. The framework is embedded into four popular CMOEAs and assessed on 42 benchmark problems.
  arXiv  Detail & Related papers  (2024-01-15T09:51:19Z)
• Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design [54.39859618450935]
  We show that it is possible to meta-learn update rules, with the hope of discovering algorithms that can perform well on a wide range of RL tasks. Despite impressive initial results from algorithms such as Learned Policy Gradient (LPG), there remains a gap when these algorithms are applied to unseen environments. In this work, we examine how characteristics of the meta-supervised-training distribution impact the performance of these algorithms.
  arXiv  Detail & Related papers  (2023-10-04T12:52:56Z)
• Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities [63.258517066104446]
  Reinforcement learning integrated as a component in the evolutionary algorithm has demonstrated superior performance in recent years. We discuss the RL-EA integration method, the RL-assisted strategy adopted by RL-EA, and its applications according to the existing literature. In the applications of RL-EA section, we also demonstrate the excellent performance of RL-EA on several benchmarks and a range of public datasets.
  arXiv  Detail & Related papers  (2023-08-25T15:06:05Z)
• On the Importance of Exploration for Generalization in Reinforcement Learning [89.63074327328765]
  We propose EDE: Exploration via Distributional Ensemble, a method that encourages exploration of states with high uncertainty. Our algorithm is the first value-based approach to achieve state-of-the-art on both Procgen and Crafter.
  arXiv  Detail & Related papers  (2023-06-08T18:07:02Z)
• Hierarchical Deep Counterfactual Regret Minimization [53.86223883060367]
  In this paper, we introduce the first hierarchical version of Deep CFR, an innovative method that boosts learning efficiency in tasks involving extensively large state spaces and deep game trees. A notable advantage of HDCFR over previous works is its ability to facilitate learning with predefined (human) expertise and foster the acquisition of skills that can be transferred to similar tasks.
  arXiv  Detail & Related papers  (2023-05-27T02:05:41Z)
• Improved Algorithms for Neural Active Learning [74.89097665112621]
  We improve the theoretical and empirical performance of neural-network(NN)-based active learning algorithms for the non-parametric streaming setting. We introduce two regret metrics by minimizing the population loss that are more suitable in active learning than the one used in state-of-the-art (SOTA) related work.
  arXiv  Detail & Related papers  (2022-10-02T05:03:38Z)
• Interactive Evolutionary Multi-Objective Optimization via Learning-to-Rank [8.421614560290609]
  This paper develops a framework for designing preference-based EMO algorithms to find solution(s) of interest (SOI) in an interactive manner. Its core idea is to involve human in the loop of EMO. After every several iterations, the DM is invited to elicit her feedback with regard to a couple of incumbent candidates. By collecting such information, her preference is progressively learned by a learning-to-rank neural network and then applied to guide the baseline EMO algorithm.
  arXiv  Detail & Related papers  (2022-04-06T06:34:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.