Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning

• URL: http://arxiv.org/abs/2607.16534v1
• Date: Fri, 17 Jul 2026 22:16:57 GMT
• Title: Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning
• Abstract summary: Building2Building is a large-scale suite of realistic Heating, Ventilation, and Air Conditioning control environments built on EnergyPlus.<n>We define benchmark tasks targeting key open challenges in RL, including goal adaptation, dynamics adaptation, action-space shifts, and cross-domain transfer.<n>By providing a large-scale, diverse, and physically grounded testbed, B2B enables systematic investigation of generalization and transfer in continuous control.
• Score: 10.870654605442427
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning (RL) has achieved strong results in control, yet learned policies remain brittle to changes in dynamics, action spaces, observation spaces, or goals, a critical limitation for real-world deployment. Existing benchmarks offer limited diversity and complexity, making it difficult to rigorously study transfer, multi-task learning, and meta-learning in RL. We introduce Building2Building (B2B), a large-scale suite of realistic Heating, Ventilation, and Air Conditioning (HVAC) control environments built on EnergyPlus, a state-of-the-art building simulator. B2B is fully compatible with the Gymnasium interface and features a parametric building generator, enabling the systematic generation of diverse building configurations with heterogeneous observation and action spaces. Based on this suite, we define benchmark tasks targeting key open challenges in RL, including goal adaptation, dynamics adaptation, action-space shifts, and cross-domain transfer. By providing a large-scale, diverse, and physically grounded testbed with standardized evaluation protocols, B2B enables systematic investigation of generalization and transfer in continuous control. Beyond advancing research on generalization in RL, this new benchmark also carries significant societal implications by enabling improved HVAC control at scale, one of the most energy-intensive systems in buildings.

Related papers

• LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoning [90.86828952599147]
  We present LaST-R1, a novel reinforcement learning framework designed to harness "latent reasoning-before-acting" policies.<n>LaST-R1 achieves a near-perfect 99.9% average success rate on the LIBERO benchmark.<n>In real-world deployments, LaST-R1 yields up to a 22.5% average improvement over SOTA supervised fine-tuning approach.
  arXiv  Detail & Related papers  (2026-04-30T17:59:52Z)
• Gym-V: A Unified Vision Environment System for Agentic Vision Research [51.62136909074774]
  Gym-V is a unified platform of 179 procedurally generated visual environments across 10 domains with controllable difficulty.<n>We find that observation scaffolding is more decisive for training success than the choice of RL algorithm.<n>Cross-domain transfer experiments show that training on diverse task generalizes broadly while narrow training can cause negative transfer.
  arXiv  Detail & Related papers  (2026-03-16T15:37:07Z)
• Sample-Efficient Neurosymbolic Deep Reinforcement Learning [49.60927398960061]
  We propose a neuro-symbolic Deep RL approach that integrates background symbolic knowledge to improve sample efficiency.<n>Online reasoning is performed to guide the training process through two mechanisms.<n>We show improved performance over a state-of-the-art reward machine baseline.
  arXiv  Detail & Related papers  (2026-01-06T09:28:53Z)
• MuFlex: A Scalable, Physics-based Platform for Multi-Building Flexibility Analysis and Coordination [17.55434238676]
  MuFlex is a scalable, open-source platform for benchmarking and testing control strategies for multi-building flexibility coordination.<n>The platform capabilities were demonstrated in a case study coordinating demand flexibility across four office buildings.
  arXiv  Detail & Related papers  (2025-08-19T05:44:06Z)
• Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation [69.30586607892842]
  We introduce Genie Envisioner (GE), a unified world foundation platform for robotic manipulation.<n>GE integrates policy learning, evaluation, and simulation within a single video-generative framework.
  arXiv  Detail & Related papers  (2025-08-07T17:59:44Z)
• BEAVER: Building Environments with Assessable Variation for Evaluating Multi-Objective Reinforcement Learning [4.342241136871849]
  We develop a novel benchmark to facilitate the evaluation of generalizable reinforcement learning algorithms in building control tasks.<n>Our results show that existing multi-objective RL methods are capable of achieving reasonable trade-offs between conflicting objectives.
  arXiv  Detail & Related papers  (2025-07-10T13:54:38Z)
• Continual Reinforcement Learning for HVAC Systems Control: Integrating Hypernetworks and Transfer Learning [0.0]
  Big data has enabled data-driven methods like Deep Reinforcement Learning (DRL)<n>We introduce a model-based reinforcement learning framework that uses a Hypernetwork to continuously learn environment dynamics across tasks with different action spaces.<n>Our approach demonstrates strong backward transfer in a continual learning setting after training on a second task, minimal fine-tuning on the first task allows rapid convergence within just 5 episodes.
  arXiv  Detail & Related papers  (2025-03-24T23:38:04Z)
• GRAM: Generalization in Deep RL with a Robust Adaptation Module [62.662894174616895]
  In this work, we present a framework for dynamics generalization in deep reinforcement learning.<n>We introduce a robust adaptation module that provides a mechanism for identifying and reacting to both in-distribution and out-of-distribution environment dynamics.<n>Our algorithm GRAM achieves strong generalization performance across in-distribution and out-of-distribution scenarios upon deployment.
  arXiv  Detail & Related papers  (2024-12-05T16:39:01Z)
• BEAR: Physics-Principled Building Environment for Control and Reinforcement Learning [9.66911049633598]
  "BEAR" is a physics-principled Building Environment for Control And Reinforcement Learning. It allows researchers to benchmark both model-based and model-free controllers using a broad collection of standard building models in Python without co-simulation using external building simulators. We demonstrate the compatibility and performance of BEAR with different controllers, including both model predictive control (MPC) and several state-of-the-art RL methods with two case studies.
  arXiv  Detail & Related papers  (2022-11-27T06:36:35Z)
• Low Emission Building Control with Zero-Shot Reinforcement Learning [70.70479436076238]
  Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency. We show it is possible to obtain emission-reducing policies without a priori--a paradigm we call zero-shot building control.
  arXiv  Detail & Related papers  (2022-08-12T17:13:25Z)
• Real-world challenges for reinforcement learning in building control [0.0]
  We propose a non-exhaustive nine real world challenges for reinforcement learning building controller. We argue that building control research should be expressed in this framework.
  arXiv  Detail & Related papers  (2021-11-25T02:59:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.