Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey

• URL: http://arxiv.org/abs/2003.04960v2
• Date: Thu, 17 Sep 2020 22:31:51 GMT
• Title: Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey
• Authors: Sanmit Narvekar and Bei Peng and Matteo Leonetti and Jivko Sinapov and Matthew E. Taylor and Peter Stone
• Abstract summary: Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback. We present a framework for curriculum learning (CL) in RL, and use it to survey and classify existing CL methods in terms of their assumptions, capabilities, and goals.
• Score: 53.73359052511171
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback. Despite many advances over the past three decades, learning in many domains still requires a large amount of interaction with the environment, which can be prohibitively expensive in realistic scenarios. To address this problem, transfer learning has been applied to reinforcement learning such that experience gained in one task can be leveraged when starting to learn the next, harder task. More recently, several lines of research have explored how tasks, or data samples themselves, can be sequenced into a curriculum for the purpose of learning a problem that may otherwise be too difficult to learn from scratch. In this article, we present a framework for curriculum learning (CL) in reinforcement learning, and use it to survey and classify existing CL methods in terms of their assumptions, capabilities, and goals. Finally, we use our framework to find open problems and suggest directions for future RL curriculum learning research.

Related papers

• Scalable Language Model with Generalized Continual Learning [58.700439919096155]
  The Joint Adaptive Re-ization (JARe) is integrated with Dynamic Task-related Knowledge Retrieval (DTKR) to enable adaptive adjustment of language models based on specific downstream tasks. Our method demonstrates state-of-the-art performance on diverse backbones and benchmarks, achieving effective continual learning in both full-set and few-shot scenarios with minimal forgetting.
  arXiv  Detail & Related papers  (2024-04-11T04:22:15Z)
• LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models [56.25156596019168]
  This paper introduces the LMRL-Gym benchmark for evaluating multi-turn RL for large language models (LLMs) Our benchmark consists of 8 different language tasks, which require multiple rounds of language interaction and cover a range of tasks in open-ended dialogue and text games.
  arXiv  Detail & Related papers  (2023-11-30T03:59:31Z)
• When Meta-Learning Meets Online and Continual Learning: A Survey [39.53836535326121]
  meta-learning is a data-driven approach to optimize the learning algorithm. Continual learning and online learning, both of which involve incrementally updating a model with streaming data. This paper organizes various problem settings using consistent terminology and formal descriptions.
  arXiv  Detail & Related papers  (2023-11-09T09:49:50Z)
• A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning [76.47138162283714]
  Forgetting refers to the loss or deterioration of previously acquired information or knowledge. Forgetting is a prevalent phenomenon observed in various other research domains within deep learning. Survey argues that forgetting is a double-edged sword and can be beneficial and desirable in certain cases.
  arXiv  Detail & Related papers  (2023-07-16T16:27:58Z)
• Teacher-student curriculum learning for reinforcement learning [1.7259824817932292]
  Reinforcement learning (rl) is a popular paradigm for sequential decision making problems. The sample inefficiency of deep reinforcement learning methods is a significant obstacle when applying rl to real-world problems. We propose a teacher-student curriculum learning setting where we simultaneously train a teacher that selects tasks for the student while the student learns how to solve the selected task.
  arXiv  Detail & Related papers  (2022-10-31T14:45:39Z)
• Toward Sustainable Continual Learning: Detection and Knowledge Repurposing of Similar Tasks [31.095642850920385]
  We introduce a paradigm where the continual learner gets a sequence of mixed similar and dissimilar tasks. We propose a new continual learning framework that uses a task similarity detection function that does not require additional learning. Our experiments show that the proposed framework performs competitively on widely used computer vision benchmarks.
  arXiv  Detail & Related papers  (2022-10-11T19:35:30Z)
• Curriculum Learning: A Survey [65.31516318260759]
  Curriculum learning strategies have been successfully employed in all areas of machine learning. We construct a taxonomy of curriculum learning approaches by hand, considering various classification criteria. We build a hierarchical tree of curriculum learning methods using an agglomerative clustering algorithm.
  arXiv  Detail & Related papers  (2021-01-25T20:08:32Z)
• Continual Lifelong Learning in Natural Language Processing: A Survey [3.9103337761169943]
  Continual learning (CL) aims to enable information systems to learn from a continuous data stream across time. It is difficult for existing deep learning architectures to learn a new task without largely forgetting previously acquired knowledge. We look at the problem of CL through the lens of various NLP tasks.
  arXiv  Detail & Related papers  (2020-12-17T18:44:36Z)
• Importance Weighted Policy Learning and Adaptation [89.46467771037054]
  We study a complementary approach which is conceptually simple, general, modular and built on top of recent improvements in off-policy learning. The framework is inspired by ideas from the probabilistic inference literature and combines robust off-policy learning with a behavior prior. Our approach achieves competitive adaptation performance on hold-out tasks compared to meta reinforcement learning baselines and can scale to complex sparse-reward scenarios.
  arXiv  Detail & Related papers  (2020-09-10T14:16:58Z)
• Learning Reusable Options for Multi-Task Reinforcement Learning [27.864350957396322]
  We propose a framework for exploiting existing experience by learning reusable options. We show that after an agent learns policies for solving a small number of problems, we are able to use the trajectories generated from those policies to learn reusable options.
  arXiv  Detail & Related papers  (2020-01-06T13:49:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.