OpenRL: A Unified Reinforcement Learning Framework

• URL: http://arxiv.org/abs/2312.16189v1
• Date: Wed, 20 Dec 2023 12:04:06 GMT
• Title: OpenRL: A Unified Reinforcement Learning Framework
• Authors: Shiyu Huang, Wentse Chen, Yiwen Sun, Fuqing Bie, Wei-Wei Tu
• Abstract summary: We present OpenRL, an advanced reinforcement learning (RL) framework. It is designed to accommodate a diverse array of tasks, from single-agent challenges to complex multi-agent systems. It integrates Natural Language Processing (NLP) with RL, enabling researchers to address a combination of RL training and language-centric tasks effectively.
• Score: 19.12129820612253
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present OpenRL, an advanced reinforcement learning (RL) framework designed to accommodate a diverse array of tasks, from single-agent challenges to complex multi-agent systems. OpenRL's robust support for self-play training empowers agents to develop advanced strategies in competitive settings. Notably, OpenRL integrates Natural Language Processing (NLP) with RL, enabling researchers to address a combination of RL training and language-centric tasks effectively. Leveraging PyTorch's robust capabilities, OpenRL exemplifies modularity and a user-centric approach. It offers a universal interface that simplifies the user experience for beginners while maintaining the flexibility experts require for innovation and algorithm development. This equilibrium enhances the framework's practicality, adaptability, and scalability, establishing a new standard in RL research. To delve into OpenRL's features, we invite researchers and enthusiasts to explore our GitHub repository at https://github.com/OpenRL-Lab/openrl and access our comprehensive documentation at https://openrl-docs.readthedocs.io.

Related papers

• ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL [80.10358123795946]
  We develop a framework for building multi-turn RL algorithms for fine-tuning large language models. Our framework adopts a hierarchical RL approach and runs two RL algorithms in parallel. Empirically, we find that ArCHer significantly improves efficiency and performance on agent tasks.
  arXiv  Detail & Related papers  (2024-02-29T18:45:56Z)
• RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmark [69.19502244910632]
  Deep reinforcement learning (RL) has shown significant benefits in solving optimization (CO) problems. We introduce RL4CO, a unified benchmark with in-depth library coverage of 23 state-of-the-art methods and more than 20 CO problems. Built on efficient software libraries and best practices in implementation, RL4CO features modularized implementation and flexible configuration of diverse RL algorithms, neural network architectures, inference techniques, and environments.
  arXiv  Detail & Related papers  (2023-06-29T16:57:22Z)
• SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores [13.948640763797776]
  We present a novel abstraction on the dataflows of RL training, which unifies diverse RL training applications into a general framework. We develop a scalable, efficient, and distributed RL system called ReaLly scalableRL, which allows efficient and massively parallelized training. SRL is the first in the academic community to perform RL experiments at a large scale with over 15k CPU cores.
  arXiv  Detail & Related papers  (2023-06-29T05:16:25Z)
• Is Reinforcement Learning (Not) for Natural Language Processing?: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization [73.74371798168642]
  We introduce an open-source modular library, RL4LMs, for optimizing language generators with reinforcement learning. Next, we present the GRUE benchmark, a set of 6 language generation tasks which are supervised not by target strings, but by reward functions. Finally, we introduce an easy-to-use, performant RL algorithm, NLPO, that learns to effectively reduce the action space in language generation.
  arXiv  Detail & Related papers  (2022-10-03T21:38:29Z)
• JORLDY: a fully customizable open source framework for reinforcement learning [3.1864456096282696]
  Reinforcement Learning (RL) has been actively researched in both academic and industrial fields. JORLDY provides more than 20 widely used RL algorithms which are implemented with Pytorch. JORLDY supports multiple RL environments which include OpenAI gym, Unity ML-Agents, Mujoco, Super Mario Bros and Procgen.
  arXiv  Detail & Related papers  (2022-04-11T06:28:27Z)
• Jump-Start Reinforcement Learning [68.82380421479675]
  We present a meta algorithm that can use offline data, demonstrations, or a pre-existing policy to initialize an RL policy. In particular, we propose Jump-Start Reinforcement Learning (JSRL), an algorithm that employs two policies to solve tasks. We show via experiments that JSRL is able to significantly outperform existing imitation and reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-04-05T17:25:22Z)
• RL-DARTS: Differentiable Architecture Search for Reinforcement Learning [62.95469460505922]
  We introduce RL-DARTS, one of the first applications of Differentiable Architecture Search (DARTS) in reinforcement learning (RL) By replacing the image encoder with a DARTS supernet, our search method is sample-efficient, requires minimal extra compute resources, and is also compatible with off-policy and on-policy RL algorithms, needing only minor changes in preexisting code. We show that the supernet gradually learns better cells, leading to alternative architectures which can be highly competitive against manually designed policies, but also verify previous design choices for RL policies.
  arXiv  Detail & Related papers  (2021-06-04T03:08:43Z)
• Improving Reinforcement Learning with Human Assistance: An Argument for Human Subject Studies with HIPPO Gym [21.4215863934377]
  Reinforcement learning (RL) is a popular machine learning paradigm for game playing, robotics control, and other sequential decision tasks. This article introduces our new open-source RL framework, the Human Input Parsing Platform for Openai Gym (HIPPO Gym)
  arXiv  Detail & Related papers  (2021-02-02T12:56:02Z)
• EasyRL: A Simple and Extensible Reinforcement Learning Framework [3.2173369911280023]
  EasyRL provides an interactive graphical user interface for users to train and evaluate RL agents. EasyRL does not require programming knowledge for training and testing simple built-in RL agents. EasyRL also supports custom RL agents and environments, which can be highly beneficial for RL researchers in evaluating and comparing their RL models.
  arXiv  Detail & Related papers  (2020-08-04T17:02:56Z)
• Integrating Distributed Architectures in Highly Modular RL Libraries [4.297070083645049]
  Most popular reinforcement learning libraries advocate for highly modular agent composability. We propose a versatile approach that allows the definition of RL agents at different scales through independent reusable components.
  arXiv  Detail & Related papers  (2020-07-06T10:22:07Z)
• MushroomRL: Simplifying Reinforcement Learning Research [60.70556446270147]
  MushroomRL is an open-source Python library developed to simplify the process of implementing and running Reinforcement Learning (RL) experiments. Compared to other available libraries, MushroomRL has been created with the purpose of providing a comprehensive and flexible framework to minimize the effort in implementing and testing novel RL methodologies.
  arXiv  Detail & Related papers  (2020-01-04T17:23:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.