Geodesics, Non-linearities and the Archive of Novelty Search
- URL: http://arxiv.org/abs/2205.03162v1
- Date: Fri, 6 May 2022 12:03:40 GMT
- Title: Geodesics, Non-linearities and the Archive of Novelty Search
- Authors: Achkan Salehi, Alexandre Coninx, Stephane Doncieux
- Abstract summary: We show that a key effect of the archive is that it counterbalances the exploration biases that result from the use of inadequate behavior metrics.
Our observations seem to hint that attributing a more active role to the archive in sampling can be beneficial.
- Score: 69.6462706723023
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The Novelty Search (NS) algorithm was proposed more than a decade ago.
However, the mechanisms behind its empirical success are still not well
formalized/understood. This short note focuses on the effects of the archive on
exploration. Experimental evidence from a few application domains suggests that
archive-based NS performs in general better than when Novelty is solely
computed with respect to the population. An argument that is often encountered
in the literature is that the archive prevents exploration from backtracking or
cycling, i.e. from revisiting previously encountered areas in the behavior
space. We argue that this is not a complete or accurate explanation as
backtracking - beside often being desirable - can actually be enabled by the
archive. Through low-dimensional/analytical examples, we show that a key effect
of the archive is that it counterbalances the exploration biases that result,
among other factors, from the use of inadequate behavior metrics and the
non-linearities of the behavior mapping. Our observations seem to hint that
attributing a more active role to the archive in sampling can be beneficial.
Related papers
- Don't Miss Out on Novelty: Importance of Novel Features for Deep Anomaly
Detection [64.21963650519312]
Anomaly Detection (AD) is a critical task that involves identifying observations that do not conform to a learned model of normality.
We propose a novel approach to AD using explainability to capture such novel features as unexplained observations in the input space.
Our approach establishes a new state-of-the-art across multiple benchmarks, handling diverse anomaly types.
arXiv Detail & Related papers (2023-10-01T21:24:05Z) - Cyclophobic Reinforcement Learning [2.2940141855172036]
In environments with sparse rewards, finding a good inductive bias for exploration is crucial to the agent's success.
We propose a new intrinsic reward that is cyclophobic, i.e., it does not reward novelty, but punishes redundancy by avoiding cycles.
Augmenting the cyclophobic intrinsic reward with a sequence of hierarchical representations we are able to achieve excellent results in the MiniGrid and MiniHack environments.
arXiv Detail & Related papers (2023-08-30T09:38:44Z) - Latent Magic: An Investigation into Adversarial Examples Crafted in the
Semantic Latent Space [0.0]
Adrial attacks against Deep Neural Networks(DNN) have been a crutial topic ever since citegoodfellow purposed the vulnerability of DNNs.
Most prior works craft adversarial examples in the pixel space, following the $l_p$ norm constraint.
In this paper, we give intuitional explain about why crafting adversarial examples in the latent space is equally efficient and important.
arXiv Detail & Related papers (2023-05-22T10:39:54Z) - Look Beyond Bias with Entropic Adversarial Data Augmentation [4.893694715581673]
Deep neural networks do not discriminate between spurious and causal patterns, and will only learn the most predictive ones while ignoring the others.
Debiasing methods were developed to make networks robust to such spurious biases but require to know in advance if a dataset is biased.
In this paper, we argue that such samples should not be necessarily needed because the ''hidden'' causal information is often also contained in biased images.
arXiv Detail & Related papers (2023-01-10T08:25:24Z) - Guarantees for Epsilon-Greedy Reinforcement Learning with Function
Approximation [69.1524391595912]
Myopic exploration policies such as epsilon-greedy, softmax, or Gaussian noise fail to explore efficiently in some reinforcement learning tasks.
This paper presents a theoretical analysis of such policies and provides the first regret and sample-complexity bounds for reinforcement learning with myopic exploration.
arXiv Detail & Related papers (2022-06-19T14:44:40Z) - Deep Hierarchy in Bandits [51.22833900944146]
Mean rewards of actions are often correlated.
To maximize statistical efficiency, it is important to leverage these correlations when learning.
We formulate a bandit variant of this problem where the correlations of mean action rewards are represented by a hierarchical Bayesian model.
arXiv Detail & Related papers (2022-02-03T08:15:53Z) - BR-NS: an Archive-less Approach to Novelty Search [70.13948372218849]
We discuss an alternative approach to novelty estimation, dubbed Behavior Recognition based Novelty Search (BR-NS)
BR-NS does not require an archive, makes no assumption on the metrics that can be defined in the behavior space and does not rely on nearest neighbours search.
We conduct experiments to gain insight into its feasibility and dynamics as well as potential advantages over archive-based NS in terms of time complexity.
arXiv Detail & Related papers (2021-04-08T17:31:34Z) - Current Time Series Anomaly Detection Benchmarks are Flawed and are
Creating the Illusion of Progress [11.689905300531917]
We introduce the UCR Time Series Anomaly Archive.
This resource will perform a similar role as the UCR Time Series Classification Archive.
arXiv Detail & Related papers (2020-09-29T06:29:04Z) - DARTS-: Robustly Stepping out of Performance Collapse Without Indicators [74.21019737169675]
Differentiable architecture search suffers from long-standing performance instability.
indicators such as Hessian eigenvalues are proposed as a signal to stop searching before the performance collapses.
In this paper, we undertake a more subtle and direct approach to resolve the collapse.
arXiv Detail & Related papers (2020-09-02T12:54:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.