Uncertain Quality-Diversity: Evaluation methodology and new methods for
Quality-Diversity in Uncertain Domains
- URL: http://arxiv.org/abs/2302.00463v2
- Date: Mon, 27 Mar 2023 17:44:40 GMT
- Title: Uncertain Quality-Diversity: Evaluation methodology and new methods for
Quality-Diversity in Uncertain Domains
- Authors: Manon Flageat and Antoine Cully
- Abstract summary: Quality-Diversity optimisation (QD) has proven to yield promising results across a broad set of applications.
However, QD approaches struggle in the presence of uncertainty in the environment, as it impacts their ability to quantify the true performance and novelty of solutions.
We formalise a common framework for uncertain domains: the Uncertain QD setting.
Second, we propose a new methodology to evaluate Uncertain QD approaches, relying on a new per-generation sampling budget and a set of existing and new metrics specifically designed for Uncertain QD.
Third, we propose three new Uncertain QD algorithms: Archive-sampling, Parallel-Adaptive
- Score: 1.827510863075184
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Quality-Diversity optimisation (QD) has proven to yield promising results
across a broad set of applications. However, QD approaches struggle in the
presence of uncertainty in the environment, as it impacts their ability to
quantify the true performance and novelty of solutions. This problem has been
highlighted multiple times independently in previous literature. In this work,
we propose to uniformise the view on this problem through four main
contributions. First, we formalise a common framework for uncertain domains:
the Uncertain QD setting, a special case of QD in which fitness and descriptors
for each solution are no longer fixed values but distribution over possible
values. Second, we propose a new methodology to evaluate Uncertain QD
approaches, relying on a new per-generation sampling budget and a set of
existing and new metrics specifically designed for Uncertain QD. Third, we
propose three new Uncertain QD algorithms: Archive-sampling,
Parallel-Adaptive-sampling and Deep-Grid-sampling. We propose these approaches
taking into account recent advances in the QD community toward the use of
hardware acceleration that enable large numbers of parallel evaluations and
make sampling an affordable approach to uncertainty. Our final and fourth
contribution is to use this new framework and the associated comparison methods
to benchmark existing and novel approaches. We demonstrate once again the
limitation of MAP-Elites in uncertain domains and highlight the performance of
the existing Deep-Grid approach, and of our new algorithms. The goal of this
framework and methods is to become an instrumental benchmark for future works
considering Uncertain QD.
Related papers
- Exploring the Performance-Reproducibility Trade-off in Quality-Diversity [7.858994681440057]
Quality-Diversity (QD) algorithms have exhibited promising results across many domains and applications.
However, uncertainty in fitness and behaviour estimations of solutions remains a major challenge when QD is used in complex real-world applications.
We propose four new a-priori QD algorithms that find optimal solutions for given preferences over the trade-offs.
arXiv Detail & Related papers (2024-09-20T08:20:31Z) - Sound Heuristic Search Value Iteration for Undiscounted POMDPs with Reachability Objectives [16.101435842520473]
This paper studies the challenging yet important problem in POMDPs known as the (indefinite-horizon) Maximal Reachability Probability Problem.
Inspired by the success of point-based methods developed for discounted problems, we study their extensions to MRPP.
We present a novel algorithm that leverages the strengths of these techniques for efficient exploration of the belief space.
arXiv Detail & Related papers (2024-06-05T02:33:50Z) - Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs [44.890946409769924]
Neural Operators (NOs) have emerged as particularly promising quantification.
We show that ensembling several NOs can identify high-error regions and provide good uncertainty estimates.
We then introduce Operator-ProbConserv, a method that uses these well-calibrated UQ estimates within the ProbConserv framework to update the model.
arXiv Detail & Related papers (2024-03-15T19:21:27Z) - Q-REG: End-to-End Trainable Point Cloud Registration with Surface
Curvature [81.25511385257344]
We present a novel solution, Q-REG, which utilizes rich geometric information to estimate the rigid pose from a single correspondence.
Q-REG allows to formalize the robust estimation as an exhaustive search, hence enabling end-to-end training.
We demonstrate in the experiments that Q-REG is agnostic to the correspondence matching method and provides consistent improvement both when used only in inference and in end-to-end training.
arXiv Detail & Related papers (2023-09-27T20:58:53Z) - On Pitfalls of Test-Time Adaptation [82.8392232222119]
Test-Time Adaptation (TTA) has emerged as a promising approach for tackling the robustness challenge under distribution shifts.
We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols.
arXiv Detail & Related papers (2023-06-06T09:35:29Z) - Benchmark tasks for Quality-Diversity applied to Uncertain domains [1.5469452301122175]
We introduce a set of 8 easy-to-implement and lightweight tasks, split into 3 main categories.
We identify the key uncertainty properties to easily define UQD benchmark tasks.
All our tasks build on the Redundant Arm: a common QD environment that is lightweight and easily replicable.
arXiv Detail & Related papers (2023-04-24T21:23:26Z) - Toward Reliable Human Pose Forecasting with Uncertainty [51.628234388046195]
We develop an open-source library for human pose forecasting, including multiple models, supporting several datasets.
We devise two types of uncertainty in the problem to increase performance and convey better trust.
arXiv Detail & Related papers (2023-04-13T17:56:08Z) - Uncertainty-Driven Action Quality Assessment [67.20617610820857]
We propose a novel probabilistic model, named Uncertainty-Driven AQA (UD-AQA), to capture the diversity among multiple judge scores.
We generate the estimation of uncertainty for each prediction, which is employed to re-weight AQA regression loss.
Our proposed method achieves competitive results on three benchmarks including the Olympic events MTL-AQA and FineDiving, and the surgical skill JIGSAWS datasets.
arXiv Detail & Related papers (2022-07-29T07:21:15Z) - Towards QD-suite: developing a set of benchmarks for Quality-Diversity
algorithms [0.0]
Existing benchmarks are not standardized, and there is currently no MNIST equivalent for Quality-Diversity (QD)
We argue that the identification of challenges faced by QD methods and the development of targeted, challenging, scalable benchmarks is an important step.
arXiv Detail & Related papers (2022-05-06T13:33:50Z) - CertainNet: Sampling-free Uncertainty Estimation for Object Detection [65.28989536741658]
Estimating the uncertainty of a neural network plays a fundamental role in safety-critical settings.
In this work, we propose a novel sampling-free uncertainty estimation method for object detection.
We call it CertainNet, and it is the first to provide separate uncertainties for each output signal: objectness, class, location and size.
arXiv Detail & Related papers (2021-10-04T17:59:31Z) - Few-shot Quality-Diversity Optimization [50.337225556491774]
Quality-Diversity (QD) optimization has been shown to be effective tools in dealing with deceptive minima and sparse rewards in Reinforcement Learning.
We show that, given examples from a task distribution, information about the paths taken by optimization in parameter space can be leveraged to build a prior population, which when used to initialize QD methods in unseen environments, allows for few-shot adaptation.
Experiments carried in both sparse and dense reward settings using robotic manipulation and navigation benchmarks show that it considerably reduces the number of generations that are required for QD optimization in these environments.
arXiv Detail & Related papers (2021-09-14T17:12:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.