Scaling MAP-Elites to Deep Neuroevolution
- URL: http://arxiv.org/abs/2003.01825v3
- Date: Fri, 5 Jun 2020 15:59:15 GMT
- Title: Scaling MAP-Elites to Deep Neuroevolution
- Authors: C\'edric Colas, Joost Huizinga, Vashisht Madhavan, Jeff Clune
- Abstract summary: We propose to leverage the efficiency of Evolution Strategies (ES) to scale MAP-Elites to high-dimensional controllers parameterized by large neural networks.
We show that ME-ES performs efficient exploration, on par with state-of-the-art exploration algorithms in high-dimensional control tasks with strongly deceptive rewards.
- Score: 5.332714036560255
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Quality-Diversity (QD) algorithms, and MAP-Elites (ME) in particular, have
proven very useful for a broad range of applications including enabling real
robots to recover quickly from joint damage, solving strongly deceptive maze
tasks or evolving robot morphologies to discover new gaits. However, present
implementations of MAP-Elites and other QD algorithms seem to be limited to
low-dimensional controllers with far fewer parameters than modern deep neural
network models. In this paper, we propose to leverage the efficiency of
Evolution Strategies (ES) to scale MAP-Elites to high-dimensional controllers
parameterized by large neural networks. We design and evaluate a new hybrid
algorithm called MAP-Elites with Evolution Strategies (ME-ES) for post-damage
recovery in a difficult high-dimensional control task where traditional ME
fails. Additionally, we show that ME-ES performs efficient exploration, on par
with state-of-the-art exploration algorithms in high-dimensional control tasks
with strongly deceptive rewards.
Related papers
- Robust Analysis of Multi-Task Learning Efficiency: New Benchmarks on Light-Weighed Backbones and Effective Measurement of Multi-Task Learning Challenges by Feature Disentanglement [69.51496713076253]
In this paper, we focus on the aforementioned efficiency aspects of existing MTL methods.
We first carry out large-scale experiments of the methods with smaller backbones and on a the MetaGraspNet dataset as a new test ground.
We also propose Feature Disentanglement measure as a novel and efficient identifier of the challenges in MTL.
arXiv Detail & Related papers (2024-02-05T22:15:55Z) - tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs) [31.69308712027795]
We study how to harness large language models to automatically discover new tensor network structure search (TN-SS) algorithms.
By observing how human experts innovate in research, we propose an automatic algorithm discovery framework called tnGPS.
The proposed framework is an elaborate prompting pipeline that instruct LLMs to generate new TN-SS algorithms through iterative refinement and enhancement.
arXiv Detail & Related papers (2024-02-04T12:06:13Z) - EPIM: Efficient Processing-In-Memory Accelerators based on Epitome [78.79382890789607]
We introduce the Epitome, a lightweight neural operator offering convolution-like functionality.
On the software side, we evaluate epitomes' latency and energy on PIM accelerators.
We introduce a PIM-aware layer-wise design method to enhance their hardware efficiency.
arXiv Detail & Related papers (2023-11-12T17:56:39Z) - Evolving Populations of Diverse RL Agents with MAP-Elites [1.5575376673936223]
We introduce a flexible framework that allows the use of any Reinforcement Learning (RL) algorithm instead of just policies.
We demonstrate the benefits brought about by our framework through extensive numerical experiments on a number of robotics control problems.
arXiv Detail & Related papers (2023-03-09T19:05:45Z) - Training Diverse High-Dimensional Controllers by Scaling Covariance
Matrix Adaptation MAP-Annealing [12.90845054806193]
Pre-training a diverse set of neural network controllers in simulation has enabled robots to adapt online to damage in robot locomotion tasks.
CMA-MAE, an evolution strategies (ES)-based quality diversity algorithm, does not have these limitations and has achieved state-of-the-art performance on standard QD benchmarks.
We propose three new CMA-MAE variants that scale to high dimensions.
arXiv Detail & Related papers (2022-10-06T01:03:01Z) - PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive
Information Representations [32.37414300338581]
Evolution Strategy (ES) algorithms have shown promising results in training complex robotic control policies.
PI-ARS combines a gradient-based representation learning technique, Predictive Information (PI), with a gradient-free ES algorithm, Augmented Random Search (ARS)
We show PI-ARS demonstrates significantly better learning efficiency and performance compared to the ARS baseline.
arXiv Detail & Related papers (2022-07-27T00:26:15Z) - Collaborative Intelligent Reflecting Surface Networks with Multi-Agent
Reinforcement Learning [63.83425382922157]
Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks.
In this paper, we investigate a multi-user communication system assisted by cooperative IRS devices with the capability of energy harvesting.
arXiv Detail & Related papers (2022-03-26T20:37:14Z) - MS-RANAS: Multi-Scale Resource-Aware Neural Architecture Search [94.80212602202518]
We propose Multi-Scale Resource-Aware Neural Architecture Search (MS-RANAS)
We employ a one-shot architecture search approach in order to obtain a reduced search cost.
We achieve state-of-the-art results in terms of accuracy-speed trade-off.
arXiv Detail & Related papers (2020-09-29T11:56:01Z) - Rectified Linear Postsynaptic Potential Function for Backpropagation in
Deep Spiking Neural Networks [55.0627904986664]
Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation.
This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
arXiv Detail & Related papers (2020-03-26T11:13:07Z) - AutoML-Zero: Evolving Machine Learning Algorithms From Scratch [76.83052807776276]
We show that it is possible to automatically discover complete machine learning algorithms just using basic mathematical operations as building blocks.
We demonstrate this by introducing a novel framework that significantly reduces human bias through a generic search space.
We believe these preliminary successes in discovering machine learning algorithms from scratch indicate a promising new direction in the field.
arXiv Detail & Related papers (2020-03-06T19:00:04Z) - Distributionally Robust Deep Learning using Hardness Weighted Sampling [5.277562268045534]
We propose a principled and efficient algorithm for DRO in machine learning that is particularly suited in the context of deep learning.
Our experiments on fetal brain 3D MRI segmentation and brain tumor segmentation in MRI demonstrate the feasibility and the usefulness of our approach.
arXiv Detail & Related papers (2020-01-08T18:02:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.