Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions

• URL: http://arxiv.org/abs/2003.08536v2
• Date: Mon, 13 Apr 2020 07:18:27 GMT
• Title: Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions
• Authors: Rui Wang, Joel Lehman, Aditya Rawal, Jiale Zhi, Yulun Li, Jeff Clune, Kenneth O. Stanley
• Abstract summary: Paired Open-Ended Trailblazer (POET) is an algorithm that generates and solves its own challenges. POET was unable to demonstrate its full creative potential because of limitations of the algorithm itself. We introduce and empirically validate two new innovations to the original algorithm, as well as two external innovations designed to help elucidate its full potential.
• Score: 20.671903144896742
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Creating open-ended algorithms, which generate their own never-ending stream of novel and appropriately challenging learning opportunities, could help to automate and accelerate progress in machine learning. A recent step in this direction is the Paired Open-Ended Trailblazer (POET), an algorithm that generates and solves its own challenges, and allows solutions to goal-switch between challenges to avoid local optima. However, the original POET was unable to demonstrate its full creative potential because of limitations of the algorithm itself and because of external issues including a limited problem space and lack of a universal progress measure. Importantly, both limitations pose impediments not only for POET, but for the pursuit of open-endedness in general. Here we introduce and empirically validate two new innovations to the original algorithm, as well as two external innovations designed to help elucidate its full potential. Together, these four advances enable the most open-ended algorithmic demonstration to date. The algorithmic innovations are (1) a domain-general measure of how meaningfully novel new challenges are, enabling the system to potentially create and solve interesting challenges endlessly, and (2) an efficient heuristic for determining when agents should goal-switch from one problem to another (helping open-ended search better scale). Outside the algorithm itself, to enable a more definitive demonstration of open-endedness, we introduce (3) a novel, more flexible way to encode environmental challenges, and (4) a generic measure of the extent to which a system continues to exhibit open-ended innovation. Enhanced POET produces a diverse range of sophisticated behaviors that solve a wide range of environmental challenges, many of which cannot be solved through other means.

Related papers

• BloomWise: Enhancing Problem-Solving capabilities of Large Language Models using Bloom's-Taxonomy-Inspired Prompts [59.83547898874152]
  We introduce BloomWise, a new prompting technique, inspired by Bloom's taxonomy, to improve the performance of Large Language Models (LLMs) The decision regarding the need to employ more sophisticated cognitive skills is based on self-evaluation performed by the LLM. In extensive experiments across 4 popular math reasoning datasets, we have demonstrated the effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2024-10-05T09:27:52Z)
• Near-Optimal Solutions of Constrained Learning Problems [85.48853063302764]
  In machine learning systems, the need to curtail their behavior has become increasingly apparent. This is evidenced by recent advancements towards developing models that satisfy dual robustness variables. Our results show that rich parametrizations effectively mitigate non-dimensional, finite learning problems.
  arXiv  Detail & Related papers  (2024-03-18T14:55:45Z)
• Open-world Machine Learning: A Review and New Outlooks [83.6401132743407]
  This paper aims to provide a comprehensive introduction to the emerging open-world machine learning paradigm. It aims to help researchers build more powerful AI systems in their respective fields, and to promote the development of artificial general intelligence.
  arXiv  Detail & Related papers  (2024-03-04T06:25:26Z)
• Introduction to Algogens [0.0]
  Algogens is a promising integration of generative AI with traditional algorithms. The book explores the basics of Algogens, their development, applications, and advantages. It offers a balanced look at the prospects and obstacles facing Algogens.
  arXiv  Detail & Related papers  (2024-03-03T07:52:10Z)
• MacGyver: Are Large Language Models Creative Problem Solvers? [87.70522322728581]
  We explore the creative problem-solving capabilities of modern LLMs in a novel constrained setting. We create MACGYVER, an automatically generated dataset consisting of over 1,600 real-world problems. We present our collection to both LLMs and humans to compare and contrast their problem-solving abilities.
  arXiv  Detail & Related papers  (2023-11-16T08:52:27Z)
• Winning the CityLearn Challenge: Adaptive Optimization with Evolutionary Search under Trajectory-based Guidance [2.4476800587391234]
  We present a novel method using the solution function of optimization as policies to compute actions for sequential decision-making. Our agent ranked first in the latest 2021 CityLearn Challenge, being able to achieve superior performance in almost all metrics.
  arXiv  Detail & Related papers  (2022-12-04T22:18:38Z)
• UNIFY: a Unified Policy Designing Framework for Solving Constrained Optimization Problems with Machine Learning [18.183339583346005]
  We propose a unified framework to design a solution policy for complex decision-making problems. Our approach relies on a clever decomposition of the policy in two stages, namely an unconstrained ML model and a CO problem. We demonstrate the method effectiveness on two practical problems, namely an Energy Management System and the Set Multi-cover with coverage requirements.
  arXiv  Detail & Related papers  (2022-10-25T14:09:24Z)
• MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learning [65.52675802289775]
  We show that an uncertainty aware classifier can solve challenging reinforcement learning problems. We propose a novel method for computing the normalized maximum likelihood (NML) distribution. We show that the resulting algorithm has a number of intriguing connections to both count-based exploration methods and prior algorithms for learning reward functions.
  arXiv  Detail & Related papers  (2021-07-15T08:19:57Z)
• Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity [15.60659580411643]
  open-ended evolution and artificial life have much to contribute towards the understanding of open-ended AI. This paper argues that open-ended AI has its own safety challenges, whether the creativity of open-ended systems can be productively and predictably controlled.
  arXiv  Detail & Related papers  (2020-06-12T22:28:09Z)
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch [76.83052807776276]
  We show that it is possible to automatically discover complete machine learning algorithms just using basic mathematical operations as building blocks. We demonstrate this by introducing a novel framework that significantly reduces human bias through a generic search space. We believe these preliminary successes in discovering machine learning algorithms from scratch indicate a promising new direction in the field.
  arXiv  Detail & Related papers  (2020-03-06T19:00:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.