Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems

• URL: http://arxiv.org/abs/2005.01643v3
• Date: Sun, 1 Nov 2020 23:50:25 GMT
• Title: Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
• Authors: Sergey Levine, Aviral Kumar, George Tucker, Justin Fu
• Abstract summary: offline reinforcement learning algorithms hold tremendous promise for making it possible to turn large datasets into powerful decision making engines. We will aim to provide the reader with an understanding of these challenges, particularly in the context of modern deep reinforcement learning methods.
• Score: 108.81683598693539
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this tutorial article, we aim to provide the reader with the conceptual tools needed to get started on research on offline reinforcement learning algorithms: reinforcement learning algorithms that utilize previously collected data, without additional online data collection. Offline reinforcement learning algorithms hold tremendous promise for making it possible to turn large datasets into powerful decision making engines. Effective offline reinforcement learning methods would be able to extract policies with the maximum possible utility out of the available data, thereby allowing automation of a wide range of decision-making domains, from healthcare and education to robotics. However, the limitations of current algorithms make this difficult. We will aim to provide the reader with an understanding of these challenges, particularly in the context of modern deep reinforcement learning methods, and describe some potential solutions that have been explored in recent work to mitigate these challenges, along with recent applications, and a discussion of perspectives on open problems in the field.

Related papers

• On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems [36.608400817940236]
  Reinforcement learning serves as potent tool for modeling dynamic user interests within recommender systems. Recent strides in offline reinforcement learning present a new perspective. Despite being a burgeoning field, works centered on recommender systems utilizing offline reinforcement learning remain limited.
  arXiv  Detail & Related papers  (2023-08-22T10:28:02Z)
• Benchmarking Offline Reinforcement Learning on Real-Robot Hardware [35.29390454207064]
  Dexterous manipulation in particular remains an open problem in its general form. We propose a benchmark including a large collection of data for offline learning from a dexterous manipulation platform on two tasks. We evaluate prominent open-sourced offline reinforcement learning algorithms on the datasets and provide a reproducible experimental setup for offline reinforcement learning on real systems.
  arXiv  Detail & Related papers  (2023-07-28T17:29:49Z)
• A Survey of Label-Efficient Deep Learning for 3D Point Clouds [109.07889215814589]
  This paper presents the first comprehensive survey of label-efficient learning of point clouds. We propose a taxonomy that organizes label-efficient learning methods based on the data prerequisites provided by different types of labels. For each approach, we outline the problem setup and provide an extensive literature review that showcases relevant progress and challenges.
  arXiv  Detail & Related papers  (2023-05-31T12:54:51Z)
• Efficient Online Reinforcement Learning with Offline Data [78.92501185886569]
  We show that we can simply apply existing off-policy methods to leverage offline data when learning online. We extensively ablate these design choices, demonstrating the key factors that most affect performance. We see that correct application of these simple recommendations can provide a $mathbf2.5times$ improvement over existing approaches.
  arXiv  Detail & Related papers  (2023-02-06T17:30:22Z)
• Skill-based Meta-Reinforcement Learning [65.31995608339962]
  We devise a method that enables meta-learning on long-horizon, sparse-reward tasks. Our core idea is to leverage prior experience extracted from offline datasets during meta-learning.
  arXiv  Detail & Related papers  (2022-04-25T17:58:19Z)
• A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems [0.0]
  Reinforcement learning (RL) has experienced a dramatic increase in popularity. There is still a wide range of domains inaccessible to RL due to the high cost and danger of interacting with the environment. offline RL is a paradigm that learns exclusively from static datasets of previously collected interactions.
  arXiv  Detail & Related papers  (2022-03-02T20:05:11Z)
• Self-Supervised Representation Learning: Introduction, Advances and Challenges [125.38214493654534]
  Self-supervised representation learning methods aim to provide powerful deep feature learning without the requirement of large annotated datasets. This article introduces this vibrant area including key concepts, the four main families of approach and associated state of the art, and how self-supervised methods are applied to diverse modalities of data.
  arXiv  Detail & Related papers  (2021-10-18T13:51:22Z)
• A Survey of Deep Meta-Learning [1.2891210250935143]
  Deep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Deep Meta-Learning is one approach to address this issue, by enabling the network to learn how to learn.
  arXiv  Detail & Related papers  (2020-10-07T17:09:02Z)
• Continual Learning for Anomaly Detection in Surveillance Videos [36.24563211765782]
  We propose an online anomaly detection method for surveillance videos using transfer learning and continual learning. Our proposed algorithm leverages the feature extraction power of neural network-based models for transfer learning, and the continual learning capability of statistical detection methods.
  arXiv  Detail & Related papers  (2020-04-15T16:41:20Z)
• Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey [53.73359052511171]
  Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback. We present a framework for curriculum learning (CL) in RL, and use it to survey and classify existing CL methods in terms of their assumptions, capabilities, and goals.
  arXiv  Detail & Related papers  (2020-03-10T20:41:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.