Towards Deployable RL -- What's Broken with RL Research and a Potential Fix

• URL: http://arxiv.org/abs/2301.01320v1
• Date: Tue, 3 Jan 2023 19:21:10 GMT
• Title: Towards Deployable RL -- What's Broken with RL Research and a Potential Fix
• Authors: Shie Mannor and Aviv Tamar
• Abstract summary: We point to some difficulties with current research which we feel are endemic to the direction taken by the community. To us, the current direction is not likely to lead to "deployable" RL: RL that works in practice and can work in practical situations yet still is economically viable.
• Score: 82.34145109359442
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning (RL) has demonstrated great potential, but is currently full of overhyping and pipe dreams. We point to some difficulties with current research which we feel are endemic to the direction taken by the community. To us, the current direction is not likely to lead to "deployable" RL: RL that works in practice and can work in practical situations yet still is economically viable. We also propose a potential fix to some of the difficulties of the field.

Related papers

• Abstracted Trajectory Visualization for Explainability in Reinforcement Learning [2.1028463367241033]
  Explainable AI (XAI) has demonstrated the potential to help reinforcement learning (RL) practitioners to understand how RL models work. XAI for users who do not have RL expertise (non-RL experts) has not been studied sufficiently. We argue that abstracted trajectories, that depicts transitions between the major states of the RL model, will be useful for non-RL experts to build a mental model of the agents.
  arXiv  Detail & Related papers  (2024-02-05T21:17:44Z)
• Deploying Deep Reinforcement Learning Systems: A Taxonomy of Challenges [13.39623605590729]
  We propose an empirical study on Stack Overflow (SO), the most popular Q&A forum for developers, to uncover and understand the challenges practitioners faced when deploying DRL systems. After filtering and manual analysis, we examined 357 SO posts about DRL deployment, investigated the current state, and identified the challenges related to deploying DRL systems. Results show that the general interest in DRL deployment is growing, confirming the study's relevance and importance.
  arXiv  Detail & Related papers  (2023-08-23T21:44:09Z)
• Making Offline RL Online: Collaborative World Models for Offline Visual Reinforcement Learning [99.59728624855566]
  This paper tries to build more flexible constraints for value estimation without impeding the exploration of potential advantages. The key idea is to leverage off-the-shelf RL simulators, which can be easily interacted with in an online manner, as the "test bed" for offline policies. We introduce CoWorld, a model-based RL approach that mitigates cross-domain discrepancies in state and reward spaces.
  arXiv  Detail & Related papers  (2023-05-24T15:45:35Z)
• A Survey of Meta-Reinforcement Learning [83.95180398234238]
  We cast the development of better RL algorithms as a machine learning problem itself in a process called meta-RL. We discuss how, at a high level, meta-RL research can be clustered based on the presence of a task distribution and the learning budget available for each individual task. We conclude by presenting the open problems on the path to making meta-RL part of the standard toolbox for a deep RL practitioner.
  arXiv  Detail & Related papers  (2023-01-19T12:01:41Z)
• Beyond Tabula Rasa: Reincarnating Reinforcement Learning [37.201451908129386]
  Learning tabula rasa, that is without any prior knowledge, is the prevalent workflow in reinforcement learning (RL) research. We present reincarnating RL as an alternative workflow, where prior computational work is reused or transferred between design iterations of an RL agent. We find that existing approaches fail in this setting and propose a simple algorithm to address their limitations.
  arXiv  Detail & Related papers  (2022-06-03T15:11:10Z)
• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems [92.73407630874841]
  Automated Reinforcement Learning (AutoRL) involves not only standard applications of AutoML but also includes additional challenges unique to RL. We provide a common taxonomy, discuss each area in detail and pose open problems which would be of interest to researchers going forward.
  arXiv  Detail & Related papers  (2022-01-11T12:41:43Z)
• A Validation Tool for Designing Reinforcement Learning Environments [0.0]
  This study proposes a Markov-based feature analysis method to validate whether an MDP is well formulated. We believe an MDP suitable for applying RL should contain a set of state features that are both sensitive to actions and predictive in rewards.
  arXiv  Detail & Related papers  (2021-12-10T13:28:08Z)
• Generalization in Deep RL for TSP Problems via Equivariance and Local Search [21.07325126324399]
  We propose a simple deep learning architecture that learns with novel RL training techniques. We empirically evaluate our proposition on random and realistic TSP problems against relevant state-of-the-art deep RL methods.
  arXiv  Detail & Related papers  (2021-10-07T16:20:37Z)
• Explore and Control with Adversarial Surprise [78.41972292110967]
  Reinforcement learning (RL) provides a framework for learning goal-directed policies given user-specified rewards. We propose a new unsupervised RL technique based on an adversarial game which pits two policies against each other to compete over the amount of surprise an RL agent experiences. We show that our method leads to the emergence of complex skills by exhibiting clear phase transitions.
  arXiv  Detail & Related papers  (2021-07-12T17:58:40Z)
• The NetHack Learning Environment [79.06395964379107]
  We present the NetHack Learning Environment (NLE), a procedurally generated rogue-like environment for Reinforcement Learning research. We argue that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL. We demonstrate empirical success for early stages of the game using a distributed Deep RL baseline and Random Network Distillation exploration.
  arXiv  Detail & Related papers  (2020-06-24T14:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.