A Survey of Exploration Methods in Reinforcement Learning

• URL: http://arxiv.org/abs/2109.00157v2
• Date: Thu, 2 Sep 2021 10:46:36 GMT
• Title: A Survey of Exploration Methods in Reinforcement Learning
• Authors: Susan Amin, Maziar Gomrokchi, Harsh Satija, Herke van Hoof, Doina Precup
• Abstract summary: Reinforcement learning agents depend crucially on exploration to obtain informative data for the learning process. In this article, we provide a survey of modern exploration methods in (Sequential) reinforcement learning, as well as a taxonomy of exploration methods.
• Score: 64.01676570654234
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Exploration is an essential component of reinforcement learning algorithms, where agents need to learn how to predict and control unknown and often stochastic environments. Reinforcement learning agents depend crucially on exploration to obtain informative data for the learning process as the lack of enough information could hinder effective learning. In this article, we provide a survey of modern exploration methods in (Sequential) reinforcement learning, as well as a taxonomy of exploration methods.

Related papers

• On the Importance of Exploration for Generalization in Reinforcement Learning [89.63074327328765]
  We propose EDE: Exploration via Distributional Ensemble, a method that encourages exploration of states with high uncertainty. Our algorithm is the first value-based approach to achieve state-of-the-art on both Procgen and Crafter.
  arXiv  Detail & Related papers  (2023-06-08T18:07:02Z)
• General Intelligence Requires Rethinking Exploration [24.980249597326985]
  We argue that exploration is essential to all learning systems, including supervised learning. Generalized exploration serves as a necessary objective for maintaining open-ended learning processes.
  arXiv  Detail & Related papers  (2022-11-15T00:46:15Z)
• The Role of Exploration for Task Transfer in Reinforcement Learning [8.817381809671804]
  We re-examine the exploration--exploitation trade-off in the context of transfer learning. In this work, we review reinforcement learning exploration methods, define a taxonomy with which to organize them, analyze these methods' differences in the context of task transfer, and suggest avenues for future investigation.
  arXiv  Detail & Related papers  (2022-10-11T01:23:21Z)
• Planning with RL and episodic-memory behavioral priors [0.20305676256390934]
  Learning from behavioral priors is a promising way to bootstrap agents with a better-than-random exploration policy. We present a planning-based approach that can use these behavioral priors for effective exploration and learning in a reinforcement learning environment.
  arXiv  Detail & Related papers  (2022-07-05T07:11:05Z)
• Sample-Efficient Reinforcement Learning in the Presence of Exogenous Information [77.19830787312743]
  In real-world reinforcement learning applications the learner's observation space is ubiquitously high-dimensional with both relevant and irrelevant information about the task at hand. We introduce a new problem setting for reinforcement learning, the Exogenous Decision Process (ExoMDP), in which the state space admits an (unknown) factorization into a small controllable component and a large irrelevant component. We provide a new algorithm, ExoRL, which learns a near-optimal policy with sample complexity in the size of the endogenous component.
  arXiv  Detail & Related papers  (2022-06-09T05:19:32Z)
• Lifelong Learning from Event-based Data [22.65311698505554]
  We investigate methods for learning from data produced by event cameras. We propose a model that is composed of both, feature extraction and continuous learning.
  arXiv  Detail & Related papers  (2021-11-11T17:59:41Z)
• Augmenting Policy Learning with Routines Discovered from a Demonstration [86.9307760606403]
  We propose routine-augmented policy learning (RAPL) RAPL discovers routines composed of primitive actions from a single demonstration. We show that RAPL improves the state-of-the-art imitation learning method SQIL and reinforcement learning method A2C.
  arXiv  Detail & Related papers  (2020-12-23T03:15:21Z)
• Reannealing of Decaying Exploration Based On Heuristic Measure in Deep Q-Network [82.20059754270302]
  We propose an algorithm based on the idea of reannealing, that aims at encouraging exploration only when it is needed. We perform an illustrative case study showing that it has potential to both accelerate training and obtain a better policy.
  arXiv  Detail & Related papers  (2020-09-29T20:40:00Z)
• Transfer Learning in Deep Reinforcement Learning: A Survey [64.36174156782333]
  Reinforcement learning is a learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in reinforcement learning upon the fast development of deep neural networks. transfer learning has arisen to tackle various challenges faced by reinforcement learning.
  arXiv  Detail & Related papers  (2020-09-16T18:38:54Z)
• A new Potential-Based Reward Shaping for Reinforcement Learning Agent [0.0]
  The proposed method extracts knowledge from episodes' cumulative rewards. The results indicate an improvement in the learning process in both the single-task and the multi-task reinforcement learner agents.
  arXiv  Detail & Related papers  (2019-02-17T10:34:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.