L2Explorer: A Lifelong Reinforcement Learning Assessment Environment

• URL: http://arxiv.org/abs/2203.07454v1
• Date: Mon, 14 Mar 2022 19:20:26 GMT
• Title: L2Explorer: A Lifelong Reinforcement Learning Assessment Environment
• Authors: Erik C. Johnson, Eric Q. Nguyen, Blake Schreurs, Chigozie S. Ewulum, Chace Ashcraft, Neil M. Fendley, Megan M. Baker, Alexander New, Gautam K. Vallabha
• Abstract summary: Reinforcement learning solutions tend to generalize poorly when exposed to new tasks outside of the data distribution they are trained on. We introduce a framework for continual reinforcement-learning development and assessment using Lifelong Learning Explorer (L2Explorer) L2Explorer is a new, Unity-based, first-person 3D exploration environment that can be continuously reconfigured to generate a range of tasks and task variants structured into complex evaluation curricula.
• Score: 49.40779372040652
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite groundbreaking progress in reinforcement learning for robotics, gameplay, and other complex domains, major challenges remain in applying reinforcement learning to the evolving, open-world problems often found in critical application spaces. Reinforcement learning solutions tend to generalize poorly when exposed to new tasks outside of the data distribution they are trained on, prompting an interest in continual learning algorithms. In tandem with research on continual learning algorithms, there is a need for challenge environments, carefully designed experiments, and metrics to assess research progress. We address the latter need by introducing a framework for continual reinforcement-learning development and assessment using Lifelong Learning Explorer (L2Explorer), a new, Unity-based, first-person 3D exploration environment that can be continuously reconfigured to generate a range of tasks and task variants structured into complex and evolving evaluation curricula. In contrast to procedurally generated worlds with randomized components, we have developed a systematic approach to defining curricula in response to controlled changes with accompanying metrics to assess transfer, performance recovery, and data efficiency. Taken together, the L2Explorer environment and evaluation approach provides a framework for developing future evaluation methodologies in open-world settings and rigorously evaluating approaches to lifelong learning.

Related papers

• Robust Monocular Visual Odometry using Curriculum Learning [1.282543877006303]
  Our work applies innovative Curriculum Learning (CL) methodologies to address the challenging geometric problem of monocular Visual Odometry (VO) estimation. We enhance the end-to-end Deep-Patch-Visual Odometry framework through the integration of novel CL approaches.
  arXiv  Detail & Related papers  (2024-11-20T16:26:51Z)
• Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Approximate Unlearning Completeness [30.596695293390415]
  We introduce the task of Lifecycle Unlearning Commitment Management (LUCM) for approximate unlearning. We propose an efficient metric designed to assess the sample-level unlearning completeness. We show that this metric is able to serve as a tool for monitoring unlearning anomalies throughout the unlearning lifecycle.
  arXiv  Detail & Related papers  (2024-03-19T15:37:27Z)
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems [128.63953314853327]
  "Lifelong Learning" systems are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. We show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems.
  arXiv  Detail & Related papers  (2023-01-18T21:58:54Z)
• System Design for an Integrated Lifelong Reinforcement Learning Agent for Real-Time Strategy Games [34.3277278308442]
  Continual/lifelong learning (LL) involves minimizing forgetting of old tasks while maximizing a model's capability to learn new tasks. We introduce the Lifelong Reinforcement Learning Components Framework (L2RLCF), which standardizes L2RL systems and assimilates different continual learning components. We describe a case study that demonstrates how multiple independently-developed LL components can be integrated into a single realized system.
  arXiv  Detail & Related papers  (2022-12-08T23:32:57Z)
• Sample-Efficient Reinforcement Learning in the Presence of Exogenous Information [77.19830787312743]
  In real-world reinforcement learning applications the learner's observation space is ubiquitously high-dimensional with both relevant and irrelevant information about the task at hand. We introduce a new problem setting for reinforcement learning, the Exogenous Decision Process (ExoMDP), in which the state space admits an (unknown) factorization into a small controllable component and a large irrelevant component. We provide a new algorithm, ExoRL, which learns a near-optimal policy with sample complexity in the size of the endogenous component.
  arXiv  Detail & Related papers  (2022-06-09T05:19:32Z)
• Autonomous Reinforcement Learning: Formalism and Benchmarking [106.25788536376007]
  Real-world embodied learning, such as that performed by humans and animals, is situated in a continual, non-episodic world. Common benchmark tasks in RL are episodic, with the environment resetting between trials to provide the agent with multiple attempts. This discrepancy presents a major challenge when attempting to take RL algorithms developed for episodic simulated environments and run them on real-world platforms.
  arXiv  Detail & Related papers  (2021-12-17T16:28:06Z)
• Planning to Explore via Self-Supervised World Models [120.31359262226758]
  Plan2Explore is a self-supervised reinforcement learning agent. We present a new approach to self-supervised exploration and fast adaptation to new tasks. Without any training supervision or task-specific interaction, Plan2Explore outperforms prior self-supervised exploration methods.
  arXiv  Detail & Related papers  (2020-05-12T17:59:45Z)
• Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey [53.73359052511171]
  Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback. We present a framework for curriculum learning (CL) in RL, and use it to survey and classify existing CL methods in terms of their assumptions, capabilities, and goals.
  arXiv  Detail & Related papers  (2020-03-10T20:41:24Z)
• Human AI interaction loop training: New approach for interactive reinforcement learning [0.0]
  Reinforcement Learning (RL) in various decision-making tasks of machine learning provides effective results with an agent learning from a stand-alone reward function. RL presents unique challenges with large amounts of environment states and action spaces, as well as in the determination of rewards. Imitation Learning (IL) offers a promising solution for those challenges using a teacher.
  arXiv  Detail & Related papers  (2020-03-09T15:27:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.