R2D-RL: A RoboCup 2D Soccer Environment for Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2606.18786v2
• Date: Thu, 25 Jun 2026 05:55:34 GMT
• Title: R2D-RL: A RoboCup 2D Soccer Environment for Multi-Agent Reinforcement Learning
• Abstract summary: RoboCup 2D Soccer Simulation (RCSS2D) provides a mature robot-soccer platform.<n>We introduce R2D-RL, a reinforcement learning environment that connects RCSS2D and HELIOS-based player clients to a Python MARL interface.<n>We provide front-goal scenarios and an 11-vs-11 full-field benchmark, together with baseline results.
• Score: 5.368091808102033
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Robot soccer is a challenging testbed for multi-agent reinforcement learning because it combines partial observability, cooperative and adversarial interaction, sparse rewards, and long-horizon tactical behavior. RoboCup 2D Soccer Simulation (RCSS2D) provides a mature robot-soccer platform, but its competition-oriented server-client architecture is difficult to use directly with modern Python-based MARL workflows. We introduce R2D-RL, a reinforcement learning environment that connects RCSS2D and HELIOS-based player clients to a Python MARL interface through shared-memory communication and cycle-level synchronization. R2D-RL supports full-field and scenario-based training with configurable opponents, Base discrete and Hybrid parameterized action spaces, action masks, expected possession value (EPV)-based reward shaping, and parallel execution. We provide front-goal scenarios and an 11-vs-11 full-field benchmark, together with baseline results.

Related papers

• ARC-RL: A Reinforcement Learning Playground Inspired by ARC Raiders [11.905134977931075]
  ARC-RL is a suite of four MuJoCo continuous-control environments featuring robotic morphologies inspired by the bestiary of ARC Raiders.<n>All four robots share a unified observation template, action convention, simulation cadence, and a single closed-form multi-component reward function.
  arXiv  Detail & Related papers  (2026-05-19T07:54:40Z)
• Seeing Together: Multi-Robot Cooperative Egocentric Spatial Reasoning with Multimodal Large Language Models [76.29781698910202]
  We propose a framework for fine-grained cooperative spatial reasoning.<n> SP-CoR combines dynamics-aware multi-robot frame sampling, spectral- and physics-guided view fusion, and physics-aligned prompt distillation.<n>It consistently improves cooperative reasoning, outperforming the strongest finetuned baseline by +3.87% on Habitat and +7.12% on iGibson.
  arXiv  Detail & Related papers  (2026-05-18T14:04:26Z)
• CRoSS: A Continual Robotic Simulation Suite for Scalable Reinforcement Learning with High Task Diversity and Realistic Physics Simulation [46.950823663585425]
  Continual reinforcement learning requires agents to learn from a sequence of tasks without forgetting previously acquired policies.<n>We introduce a novel benchmark suite for CRL based on realistically simulated robots in the Gazebo simulator.
  arXiv  Detail & Related papers  (2026-02-04T18:54:26Z)
• Part II: ROLL Flash -- Accelerating RLVR and Agentic Training with Asynchrony [78.70328630805041]
  ROLL Flash is a system that extends ROLL with native support for asynchronous RL post-training.<n>We show that ROLL Flash significantly improves resource utilization and scalability over synchronous RL post-training.
  arXiv  Detail & Related papers  (2025-10-13T12:41:27Z)
• Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation [65.3648667980258]
  Vision-language model (VLM) based GUI agents show promise for automating complex tasks, but face significant challenges in applying reinforcement learning (RL)<n>We propose DART, a Decoupled Agentic RL Training framework for GUI agents, which coordinates heterogeneous modules in a highly decoupled manner.<n>On the OSWorld benchmark, DART-GUI-7B achieves a 42.13% task success rate, a 14.61% absolute gain over the base model, and 7.34% higher than open-source SOTA.
  arXiv  Detail & Related papers  (2025-09-28T13:19:20Z)
• UI-S1: Advancing GUI Automation via Semi-online Reinforcement Learning [78.86567400365392]
  We present Semi-online Reinforcement Learning, a novel paradigm that simulates online RL on offline trajectories.<n>To capture long-term training signals, Semi-online RL introduces discounted future returns into the reward computation.<n>Experiments show that ours Semi-online RL achieves SOTA performance among 7B models across four dynamic benchmarks.
  arXiv  Detail & Related papers  (2025-09-15T03:24:08Z)
• RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics [67.11221574129937]
  Spatial referring is a fundamental capability of embodied robots to interact with the 3D physical world.<n>We propose RoboRefer, a 3D-aware VLM that can first achieve precise spatial understanding.<n>RoboRefer advances generalized multi-step spatial reasoning via reinforcement fine-tuning.
  arXiv  Detail & Related papers  (2025-06-04T17:59:27Z)
• Toward Real-World Cooperative and Competitive Soccer with Quadrupedal Robot Teams [18.20205072720915]
  We present a hierarchical multi-agent reinforcement learning (MARL) framework that enables fully autonomous and decentralized quadruped robot soccer.<n>First, a set of highly dynamic low-level skills is trained for legged locomotion and ball manipulation, such as walking, dribbling, and kicking.<n>On top of these, a high-level strategic planning policy is trained with Multi-Agent Proximal Policy Optimization (MAPPO) via Fictitious Self-Play.
  arXiv  Detail & Related papers  (2025-05-20T02:20:54Z)
• RobocupGym: A challenging continuous control benchmark in Robocup [7.926196208425107]
  We introduce a Robocup-based RL environment based on the open source rcssserver3d soccer server. In each task, an RL agent controls a simulated robot, and can interact with the ball or other agents.
  arXiv  Detail & Related papers  (2024-07-03T15:26:32Z)
• Robot Air Hockey: A Manipulation Testbed for Robot Learning with Reinforcement Learning [34.055177769808914]
  We introduce a dynamic, interactive RL testbed based on robot air hockey. Our testbed allows a varied assessment of RL capabilities. The robot air hockey testbed also supports sim-to-real transfer with three domains.
  arXiv  Detail & Related papers  (2024-05-06T02:13:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.