Quality-Diversity Meta-Evolution: customising behaviour spaces to a meta-objective

• URL: http://arxiv.org/abs/2109.03918v1
• Date: Wed, 8 Sep 2021 20:27:24 GMT
• Title: Quality-Diversity Meta-Evolution: customising behaviour spaces to a meta-objective
• Abstract summary: Quality-Diversity (QD) algorithms evolve behaviourally diverse and high-performing solutions. This paper performs an empirical study of QD-Meta on function and multilegged robot locomotion benchmarks.
• Score: 1.0152838128195467
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quality-Diversity (QD) algorithms evolve behaviourally diverse and high-performing solutions. To illuminate the elite solutions for a space of behaviours, QD algorithms require the definition of a suitable behaviour space. If the behaviour space is high-dimensional, a suitable dimensionality reduction technique is required to maintain a limited number of behavioural niches. While current methodologies for automated behaviour spaces focus on changing the geometry or on unsupervised learning, there remains a need for customising behavioural diversity to a particular meta-objective specified by the end-user. In the newly emerging framework of QD Meta-Evolution, or QD-Meta for short, one evolves a population of QD algorithms, each with different algorithmic and representational characteristics, to optimise the algorithms and their resulting archives to a user-defined meta-objective. Despite promising results compared to traditional QD algorithms, QD-Meta has yet to be compared to state-of-the-art behaviour space automation methods such as Centroidal Voronoi Tessellations Multi-dimensional Archive of Phenotypic Elites Algorithm (CVT-MAP-Elites) and Autonomous Robots Realising their Abilities (AURORA). This paper performs an empirical study of QD-Meta on function optimisation and multilegged robot locomotion benchmarks. Results demonstrate that QD-Meta archives provide improved average performance and faster adaptation to a priori unknown changes to the environment when compared to CVT-MAP-Elites and AURORA. A qualitative analysis shows how the resulting archives are tailored to the meta-objectives provided by the end-user.

Related papers

• Beyond Binary Success: A Diagnostic Meta-Evaluation Framework for Fine-Grained Manipulation [98.79811866787263]
  We introduce MetaFine, a diagnostic meta-evaluation framework that disentangles manipulation competency along three axes.<n>We identify the visual encoder's ability to preserve local spatial structure as a key bottleneck for fine-grained precision.<n>By shifting evaluation from ranking to diagnosis, MetaFine turns benchmarking into an actionable compass for repairing the layered capacities underlying genuine physical dexterity.
  arXiv  Detail & Related papers  (2026-05-19T15:25:13Z)
• AutoQD: Automatic Discovery of Diverse Behaviors with Quality-Diversity Optimization [6.781039677915376]
  Quality-Diversity (QD) algorithms have shown remarkable success in discovering diverse, high-performing solutions.<n>However, they rely heavily on hand-crafted behavioral descriptors that constrain exploration to predefined notions of diversity.<n>We present a theoretically grounded approach to automatically generate behavioral descriptors by embedding the occupancy measures of policies in Markov Decision Processes.
  arXiv  Detail & Related papers  (2025-06-05T23:34:53Z)
• Multi-Objective Covariance Matrix Adaptation MAP-Annealing [7.103319934188755]
  Quality-Diversity (QD) optimization is an emerging field that focuses on finding a set of behaviorally diverse and high-quality solutions.<n>Recent work on Multi-Objective Quality-Diversity (MOQD) extends QD optimization to simultaneously optimize multiple objective functions.<n>This opens up multi-objective applications for QD, such as generating a diverse set of game maps that maximize difficulty, realism, or other properties.
  arXiv  Detail & Related papers  (2025-05-27T04:39:28Z)
• Efficient Quality-Diversity Optimization through Diverse Quality Species [3.428706362109921]
  We show that a diverse population of solutions can be found without the limitation of needing an archive or defining the range of behaviors in advance. We propose Diverse Quality Species (DQS) as an alternative to archive-based Quality-Diversity (QD) algorithms.
  arXiv  Detail & Related papers  (2023-04-14T23:15:51Z)
• Don't Bet on Luck Alone: Enhancing Behavioral Reproducibility of Quality-Diversity Solutions in Uncertain Domains [2.639902239625779]
  We introduce Archive Reproducibility Improvement Algorithm (ARIA) ARIA is a plug-and-play approach that improves the quality of solutions present in an archive. We show that our algorithm enhances the quality and descriptor space coverage of any given archive by at least 50%.
  arXiv  Detail & Related papers  (2023-04-07T14:45:14Z)
• Enhancing MAP-Elites with Multiple Parallel Evolution Strategies [8.585387103144825]
  We propose a novel Quality-Diversity (QD) algorithm based on Evolution Strategies (ES) MEMES maintains multiple (up to 100) simultaneous ES processes, each with its own independent objective and reset mechanism designed for QD optimisation. We show that MEMES outperforms both gradient-based and mutation-based QD algorithms on black-box optimisation and QD-Reinforcement-Learning tasks.
  arXiv  Detail & Related papers  (2023-03-10T18:55:02Z)
• Goal-Conditioned Q-Learning as Knowledge Distillation [136.79415677706612]
  We explore a connection between off-policy reinforcement learning in goal-conditioned settings and knowledge distillation. We empirically show that this can improve the performance of goal-conditioned off-policy reinforcement learning when the space of goals is high-dimensional. We also show that this technique can be adapted to allow for efficient learning in the case of multiple simultaneous sparse goals.
  arXiv  Detail & Related papers  (2022-08-28T22:01:10Z)
• Evolving Pareto-Optimal Actor-Critic Algorithms for Generalizability and Stability [67.8426046908398]
  Generalizability and stability are two key objectives for operating reinforcement learning (RL) agents in the real world. This paper presents MetaPG, an evolutionary method for automated design of actor-critic loss functions.
  arXiv  Detail & Related papers  (2022-04-08T20:46:16Z)
• Learning to Walk Autonomously via Reset-Free Quality-Diversity [73.08073762433376]
  Quality-Diversity algorithms can discover large and complex behavioural repertoires consisting of both diverse and high-performing skills. Existing QD algorithms need large numbers of evaluations as well as episodic resets, which require manual human supervision and interventions. This paper proposes Reset-Free Quality-Diversity optimization (RF-QD) as a step towards autonomous learning for robotics in open-ended environments.
  arXiv  Detail & Related papers  (2022-04-07T14:07:51Z)
• Approximating Gradients for Differentiable Quality Diversity in Reinforcement Learning [8.591356221688773]
  Differentiable quality diversity (DQD) algorithms greatly accelerate QD optimization when exact gradients are available for the objective and measures. We develop two variants of the DQD algorithm CMA-MEGA, each with different gradient approximations, and evaluate them on four simulated walking tasks. One variant achieves comparable performance (QD score) with the state-of-the-art PGA-MAP-Elites in two tasks. The other variant performs comparably in all tasks but is less efficient than PGA-MAP-Elites in two tasks.
  arXiv  Detail & Related papers  (2022-02-08T05:53:55Z)
• A survey on multi-objective hyperparameter optimization algorithms for Machine Learning [62.997667081978825]
  This article presents a systematic survey of the literature published between 2014 and 2020 on multi-objective HPO algorithms. We distinguish between metaheuristic-based algorithms, metamodel-based algorithms, and approaches using a mixture of both. We also discuss the quality metrics used to compare multi-objective HPO procedures and present future research directions.
  arXiv  Detail & Related papers  (2021-11-23T10:22:30Z)
• Few-shot Quality-Diversity Optimization [50.337225556491774]
  Quality-Diversity (QD) optimization has been shown to be effective tools in dealing with deceptive minima and sparse rewards in Reinforcement Learning. We show that, given examples from a task distribution, information about the paths taken by optimization in parameter space can be leveraged to build a prior population, which when used to initialize QD methods in unseen environments, allows for few-shot adaptation. Experiments carried in both sparse and dense reward settings using robotic manipulation and navigation benchmarks show that it considerably reduces the number of generations that are required for QD optimization in these environments.
  arXiv  Detail & Related papers  (2021-09-14T17:12:20Z)
• On the use of feature-maps and parameter control for improved quality-diversity meta-evolution [1.0152838128195467]
  Quality-Diversity (QD) algorithms evolve a behaviourally diverse archive of high-performing solutions. One evolves a population of QD algorithms to optimise the behaviour space based on an archive-level objective, the meta-fitness. Non-linear and feature-selection feature-maps yield a 15-fold and 3-fold improvement in meta-fitness.
  arXiv  Detail & Related papers  (2021-05-21T12:43:27Z)
• Meta-Learning with Neural Tangent Kernels [58.06951624702086]
  We propose the first meta-learning paradigm in the Reproducing Kernel Hilbert Space (RKHS) induced by the meta-model's Neural Tangent Kernel (NTK) Within this paradigm, we introduce two meta-learning algorithms, which no longer need a sub-optimal iterative inner-loop adaptation as in the MAML framework. We achieve this goal by 1) replacing the adaptation with a fast-adaptive regularizer in the RKHS; and 2) solving the adaptation analytically based on the NTK theory.
  arXiv  Detail & Related papers  (2021-02-07T20:53:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.