Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Research

• URL: http://arxiv.org/abs/2011.14826v2
• Date: Fri, 21 May 2021 19:53:36 GMT
• Title: Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Research
• Authors: Johan S. Obando-Ceron and Pablo Samuel Castro
• Abstract summary: We argue that, despite the community's emphasis on large-scale environments, the traditional small-scale environments can still yield valuable scientific insights. We revisit the paper which introduced the Rainbow algorithm and present some new insights into the algorithms used by Rainbow.
• Score: 15.710674189908614
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Since the introduction of DQN, a vast majority of reinforcement learning research has focused on reinforcement learning with deep neural networks as function approximators. New methods are typically evaluated on a set of environments that have now become standard, such as Atari 2600 games. While these benchmarks help standardize evaluation, their computational cost has the unfortunate side effect of widening the gap between those with ample access to computational resources, and those without. In this work we argue that, despite the community's emphasis on large-scale environments, the traditional small-scale environments can still yield valuable scientific insights and can help reduce the barriers to entry for underprivileged communities. To substantiate our claims, we empirically revisit the paper which introduced the Rainbow algorithm [Hessel et al., 2018] and present some new insights into the algorithms used by Rainbow.

Related papers

• Efficient Exploration in Deep Reinforcement Learning: A Novel Bayesian Actor-Critic Algorithm [0.195804735329484]
  Reinforcement learning (RL) and Deep Reinforcement Learning (DRL) have the potential to disrupt and are already changing the way we interact with the world. One of the key indicators of their applicability is their ability to scale and work in real-world scenarios.
  arXiv  Detail & Related papers  (2024-08-19T14:50:48Z)
• ContainerGym: A Real-World Reinforcement Learning Benchmark for Resource Allocation [1.6058099298620425]
  ContainerGym is a benchmark for reinforcement learning inspired by a real-world industrial resource allocation task. The proposed benchmark encodes challenges commonly encountered in real-world sequential decision making problems. It can be configured to instantiate problems of varying degrees of difficulty.
  arXiv  Detail & Related papers  (2023-07-06T13:44:29Z)
• The Cascaded Forward Algorithm for Neural Network Training [61.06444586991505]
  We propose a new learning framework for neural networks, namely Cascaded Forward (CaFo) algorithm, which does not rely on BP optimization as that in FF. Unlike FF, our framework directly outputs label distributions at each cascaded block, which does not require generation of additional negative samples. In our framework each block can be trained independently, so it can be easily deployed into parallel acceleration systems.
  arXiv  Detail & Related papers  (2023-03-17T02:01:11Z)
• Interpretable Option Discovery using Deep Q-Learning and Variational Autoencoders [9.432068833600884]
  The DVQN algorithm is a promising approach for identifying initiation and termination conditions for option-based reinforcement learning. Experiments show that the DVQN algorithm, with automatic initiation and termination, has comparable performance to Rainbow.
  arXiv  Detail & Related papers  (2022-10-03T21:08:39Z)
• The Web Is Your Oyster -- Knowledge-Intensive NLP against a Very Large Web Corpus [76.9522248303716]
  We propose a new setup for evaluating existing KI-NLP tasks in which we generalize the background corpus to a universal web snapshot. We repurpose KILT, a standard KI-NLP benchmark initially developed for Wikipedia, and ask systems to use a subset of CCNet - the Sphere corpus. We find that despite potential gaps of coverage, challenges of scale, lack of structure and lower quality, retrieval from Sphere enables a state-of-the-art-and-read system to match and even outperform Wikipedia-based models.
  arXiv  Detail & Related papers  (2021-12-18T13:15:34Z)
• Neural Contextual Bandits with Deep Representation and Shallow Exploration [105.8099566651448]
  We propose a novel learning algorithm that transforms the raw feature vector using the last hidden layer of a deep ReLU neural network. Compared with existing neural contextual bandit algorithms, our approach is computationally much more efficient since it only needs to explore in the last layer of the deep neural network.
  arXiv  Detail & Related papers  (2020-12-03T09:17:55Z)
• Learning Intrinsic Symbolic Rewards in Reinforcement Learning [7.101885582663675]
  We present a method that discovers dense rewards in the form of low-dimensional symbolic trees. We show that the discovered dense rewards are an effective signal for an RL policy to solve the benchmark tasks.
  arXiv  Detail & Related papers  (2020-10-08T00:02:46Z)
• Munchausen Reinforcement Learning [50.396037940989146]
  bootstrapping is a core mechanism in Reinforcement Learning (RL) We show that slightly modifying Deep Q-Network (DQN) in that way provides an agent that is competitive with distributional methods on Atari games. We provide strong theoretical insights on what happens under the hood -- implicit Kullback-Leibler regularization and increase of the action-gap.
  arXiv  Detail & Related papers  (2020-07-28T18:30:23Z)
• Meta-Gradient Reinforcement Learning with an Objective Discovered Online [54.15180335046361]
  We propose an algorithm based on meta-gradient descent that discovers its own objective, flexibly parameterised by a deep neural network. Because the objective is discovered online, it can adapt to changes over time. On the Atari Learning Environment, the meta-gradient algorithm adapts over time to learn with greater efficiency.
  arXiv  Detail & Related papers  (2020-07-16T16:17:09Z)
• Importance of using appropriate baselines for evaluation of data-efficiency in deep reinforcement learning for Atari [0.0]
  We show that the actual improvement in the efficiency came from allowing the algorithm for more training updates for each data sample. We argue that the agent similar to the modified DQN that is presented in this paper should be used as a baseline for any future work aimed at improving sample efficiency of deep reinforcement learning.
  arXiv  Detail & Related papers  (2020-03-23T10:59:04Z)
• Robust Pruning at Initialization [61.30574156442608]
  A growing need for smaller, energy-efficient, neural networks to be able to use machine learning applications on devices with limited computational resources. For Deep NNs, such procedures remain unsatisfactory as the resulting pruned networks can be difficult to train and, for instance, they do not prevent one layer from being fully pruned.
  arXiv  Detail & Related papers  (2020-02-19T17:09:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.