Assessing Evolutionary Terrain Generation Methods for Curriculum Reinforcement Learning

• URL: http://arxiv.org/abs/2203.15172v1
• Date: Tue, 29 Mar 2022 01:26:15 GMT
• Title: Assessing Evolutionary Terrain Generation Methods for Curriculum Reinforcement Learning
• Authors: David Howard, Josh Kannemeyer, Davide Dolcetti, Humphrey Munn and Nicole Robinson
• Abstract summary: We compare popular noise-based terrain generators to two indirect encodings, CPPN and GAN. We assess the impact of a range of representation-agnostic MAP-Elites feature descriptors that compute metrics directly from the generated terrain meshes. Results describe key differences between the generators that inform their use in curriculum learning, and present a range of useful feature descriptors for uptake by the community.
• Score: 3.1971316044104254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Curriculum learning allows complex tasks to be mastered via incremental progression over `stepping stone' goals towards a final desired behaviour. Typical implementations learn locomotion policies for challenging environments through gradual complexification of a terrain mesh generated through a parameterised noise function. To date, researchers have predominantly generated terrains from a limited range of noise functions, and the effect of the generator on the learning process is underrepresented in the literature. We compare popular noise-based terrain generators to two indirect encodings, CPPN and GAN. To allow direct comparison between both direct and indirect representations, we assess the impact of a range of representation-agnostic MAP-Elites feature descriptors that compute metrics directly from the generated terrain meshes. Next, performance and coverage are assessed when training a humanoid robot in a physics simulator using the PPO algorithm. Results describe key differences between the generators that inform their use in curriculum learning, and present a range of useful feature descriptors for uptake by the community.

Related papers

• Noise-conditioned Energy-based Annealed Rewards (NEAR): A Generative Framework for Imitation Learning from Observation [17.73467861849673]
  This paper introduces a new imitation learning framework based on energy-based generative models. We learn complex, physics-dependent, robot motion policies through state-only expert motion trajectories. Our framework sidesteps the optimisation challenges of adversarial imitation learning techniques.
  arXiv  Detail & Related papers  (2025-01-24T17:15:49Z)
• Enhance Vision-Language Alignment with Noise [59.2608298578913]
  We investigate whether the frozen model can be fine-tuned by customized noise. We propose Positive-incentive Noise (PiNI) which can fine-tune CLIP via injecting noise into both visual and text encoders.
  arXiv  Detail & Related papers  (2024-12-14T12:58:15Z)
• Simulating Articulatory Trajectories with Phonological Feature Interpolation [15.482738311360972]
  We investigate the forward mapping between pseudo-motor commands and articulatory trajectories. Two phonological feature sets, based respectively on generative and articulatory phonology, are used to encode a phonetic target sequence. We discuss the implications of our results for our understanding of the dynamics of biological motion.
  arXiv  Detail & Related papers  (2024-08-08T10:51:16Z)
• DIDA: Denoised Imitation Learning based on Domain Adaptation [28.36684781402964]
  We focus on the problem of Learning from Noisy Demonstrations (LND), where the imitator is required to learn from data with noise. We propose Denoised Imitation learning based on Domain Adaptation (DIDA), which designs two discriminators to distinguish the noise level and expertise level of data. Experiment results on MuJoCo demonstrate that DIDA can successfully handle challenging imitation tasks from demonstrations with various types of noise, outperforming most baseline methods.
  arXiv  Detail & Related papers  (2024-04-04T11:29:05Z)
• Learning with Noisy Foundation Models [95.50968225050012]
  This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets. We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
  arXiv  Detail & Related papers  (2024-03-11T16:22:41Z)
• Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Clutter [14.489086924126253]
  This work focuses on the task of referring grasp synthesis, which predicts a grasp pose for an object referred through natural language in cluttered scenes. Existing approaches often employ multi-stage pipelines that first segment the referred object and then propose a suitable grasp, and are evaluated in private datasets or simulators that do not capture the complexity of natural indoor scenes. We propose a novel end-to-end model (CROG) that leverages the visual grounding capabilities of CLIP to learn synthesis grasp directly from image-text pairs.
  arXiv  Detail & Related papers  (2023-11-09T22:55:10Z)
• Few-Shot Audio-Visual Learning of Environment Acoustics [89.16560042178523]
  Room impulse response (RIR) functions capture how the surrounding physical environment transforms the sounds heard by a listener. We explore how to infer RIRs based on a sparse set of images and echoes observed in the space. In experiments using a state-of-the-art audio-visual simulator for 3D environments, we demonstrate that our method successfully generates arbitrary RIRs.
  arXiv  Detail & Related papers  (2022-06-08T16:38:24Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Treatment Learning Causal Transformer for Noisy Image Classification [62.639851972495094]
  In this work, we incorporate this binary information of "existence of noise" as treatment into image classification tasks to improve prediction accuracy. Motivated from causal variational inference, we propose a transformer-based architecture, that uses a latent generative model to estimate robust feature representations for noise image classification. We also create new noisy image datasets incorporating a wide range of noise factors for performance benchmarking.
  arXiv  Detail & Related papers  (2022-03-29T13:07:53Z)
• Composable Learning with Sparse Kernel Representations [110.19179439773578]
  We present a reinforcement learning algorithm for learning sparse non-parametric controllers in a Reproducing Kernel Hilbert Space. We improve the sample complexity of this approach by imposing a structure of the state-action function through a normalized advantage function. We demonstrate the performance of this algorithm on learning obstacle-avoidance policies in multiple simulations of a robot equipped with a laser scanner while navigating in a 2D environment.
  arXiv  Detail & Related papers  (2021-03-26T13:58:23Z)
• Generating Human-Like Movement: A Comparison Between Two Approaches Based on Environmental Features [4.511923587827301]
  Two novel algorithms have been presented to generate human-like trajectories based on environmental features. The human-likeness aspect has been tested by a human expert judging the final generated trajectories as realistic. We show how, despite generating trajectories closer to the real one according to our predefined metrics, the Feature-Based A* algorithm fall short in time efficiency compared to the Attraction-Based A* algorithm.
  arXiv  Detail & Related papers  (2020-12-11T16:45:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.