A Unified Python Framework for Direct PPO-based Control of AHUs with Economizer Logic and CO2-Constrained Ventilation

• URL: http://arxiv.org/abs/2605.24406v1
• Date: Sat, 23 May 2026 05:31:09 GMT
• Title: A Unified Python Framework for Direct PPO-based Control of AHUs with Economizer Logic and CO2-Constrained Ventilation
• Abstract summary: This paper presents a new approach to optimizing HVAC systems through the use of Deep Reinforcement Learning (DRL) algorithms.<n>The DRL system uses a second order-capacitor thermal model and an integrated dynamic mass balance of CO2 to replicate the complex physics associated with buildings.<n>An enthalpy-based economiser is used to create free cooling from the outdoor environment.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Optimizing HVAC (Heating, Ventilation and Air Conditioning) can enhance a building's energy efficiency while providing comfort levels for its occupants. Using conventional control systems to maintain HVAC functions is often difficult because of the nonlinear characteristics of a building envelope as it experiences stochastic load variations over time. This paper presents a new approach to optimizing HVAC systems through the use of Deep Reinforcement Learning (DRL) algorithms and the Proximal Policy Optimization (PPO) algorithm implemented in a custom Python performance environment. The DRL system uses a second order resistor-capacitor thermal model and an integrated dynamic mass balance of CO2 to replicate the complex physics associated with buildings. One major innovation of this study is a "Hierarchical Flow Logic," which provides the means to ensure that indoor air quality (IAQ) is maintained by overriding the accepted actions of the agent that cause CO2 to exceed 1000 ppm. In addition, an enthalpy-based economiser is used to create free cooling from the outdoor environment. The experimental data shows that compared to PID controllers tuned by GA or traditional On-Off controls, a PPO agent has better temperature stability and energy efficiency overall. An end-to-end pipeline provides an avenue for robust and generalized solutions to help implement smart building energy management within the context of real hardware implementation.

Related papers

• ADAPT: Physics-Aware Diffusion-based World Models for Adaptive Predictive Transferable HVAC Control [19.871939624912073]
  ADAPT is a physics-aware conditional diffusion indoor environmental world model for HVAC control.<n>It reduces HVAC energy consumption by 7.3% and occupant discomfort by 30.2% compared with state-of-the-art baselines under IID control.
  arXiv  Detail & Related papers  (2026-08-20T08:58:28Z)
• Joint Air Traffic Flow and Capacity Management via Answer Set Programming [64.47906587579985]
  We introduce a joint ATFCM model with an encoding in Answer Set Programming (ASP)<n>Our results indicate that the ASP model outperforms the MIP model.<n>While DAC has the largest improvement on solving performance compared to rerouting, unrestricted variants of DAC or rerouting lead to search space thrashing.
  arXiv  Detail & Related papers  (2026-06-22T07:58:54Z)
• Shielded Controller Units for RL with Operational Constraints Applied to Remote Microgrids [50.64533198075622]
  Reinforcement learning (RL) is a powerful framework for optimizing decision-making in complex systems under uncertainty.<n>In this paper, we introduce Shielded Controller Units (SCUs), a systematic and interpretable approach that leverages prior knowledge of system dynamics.<n>We demonstrate the effectiveness of SCUs on a remote microgrid optimization task with strict operational requirements.
  arXiv  Detail & Related papers  (2025-11-30T19:28:34Z)
• Mind Your Entropy: From Maximum Entropy to Trajectory Entropy-Constrained RL [56.085103402298905]
  We propose a trajectory entropy-constrained reinforcement learning (TECRL) framework to address these two challenges.<n>Within this framework, we first separately learn two Q-functions, one associated with reward and the other with entropy, ensuring clean and stable value targets unaffected by temperature updates.<n>We develop a practical off-policy algorithm, DSAC-E, by extending the state-of-the-art distributional soft actor-critic with three refinements.
  arXiv  Detail & Related papers  (2025-10-25T09:17:47Z)
• Optimizing Indoor Environmental Quality in Smart Buildings Using Deep Learning [0.0]
  This paper proposes a deep learning driven approach to proactively manage IEQ parameters specifically CO2 concentration, temperature, and humidity.<n>We benchmark three architectures--Long Short-Term Memory (LSTM), Gated Recurrent Units (GRU), and a hybrid Convolutional Neural Network LSTM (CNN-LSTM)--to forecast IEQ variables across various time horizons.<n>Our results show that GRU achieves the best short-term prediction accuracy with lower computational overhead, whereas CNN-LSTM excels in extracting dominant features for extended forecasting windows.
  arXiv  Detail & Related papers  (2025-09-30T12:42:34Z)
• Diffusion-Modeled Reinforcement Learning for Carbon and Risk-Aware Microgrid Optimization [48.70916202664808]
  DiffCarl is a diffusion-modeled carbon- and risk-aware reinforcement learning algorithm for intelligent operation of multi-microgrid systems.<n>It outperforms classic algorithms and state-of-the-art DRL solutions, with 2.3-30.1% lower operational cost.<n>It also achieves 28.7% lower carbon emissions than those of its carbon-unaware variant and reduces performance variability.
  arXiv  Detail & Related papers  (2025-07-22T03:27:07Z)
• Data-driven operator learning for energy-efficient building control [2.3326951882644553]
  We present a data-driven framework that combines the physical accuracy of CFD with the computational efficiency of machine learning to enable energy-efficient building ventilation control.<n>We train a neural operator transformer to learn the mapping from building control actions to airflow field distributions using high-resolution CFD data.<n> Experimental results demonstrate that our approach achieves substantial energy savings compared to maximum airflow rate control, rule-based control, and data-driven control based on regional average CO2 predictions.
  arXiv  Detail & Related papers  (2025-04-30T00:45:49Z)
• Global Transformer Architecture for Indoor Room Temperature Forecasting [49.32130498861987]
  This work presents a global Transformer architecture for indoor temperature forecasting in multi-room buildings. It aims at optimizing energy consumption and reducing greenhouse gas emissions associated with HVAC systems. Notably, this study is the first to apply a Transformer architecture for indoor temperature forecasting in multi-room buildings.
  arXiv  Detail & Related papers  (2023-10-31T14:09:32Z)
• Surrogate Empowered Sim2Real Transfer of Deep Reinforcement Learning for ORC Superheat Control [12.567922037611261]
  This paper proposes a Sim2Real transfer learning-based DRL control method for ORC superheat control. Experimental results show that the proposed method greatly improves the training speed of DRL in ORC control problems.
  arXiv  Detail & Related papers  (2023-08-05T01:59:44Z)
• Data-driven HVAC Control Using Symbolic Regression: Design and Implementation [0.0]
  This study proposes a design and implementation methodology of data-driven heating, ventilation, and air conditioning () control. Building thermodynamics is modeled using a symbolic regression model (SRM) built from the collected data. The proposed framework reduces the peak power by 16.1% compared to the widely used thermostat controller.
  arXiv  Detail & Related papers  (2023-04-06T13:57:50Z)
• Low Emission Building Control with Zero-Shot Reinforcement Learning [70.70479436076238]
  Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency. We show it is possible to obtain emission-reducing policies without a priori--a paradigm we call zero-shot building control.
  arXiv  Detail & Related papers  (2022-08-12T17:13:25Z)
• Development of a Soft Actor Critic Deep Reinforcement Learning Approach for Harnessing Energy Flexibility in a Large Office Building [0.0]
  This research is concerned with the novel application and investigation of Soft Actor Critic' (SAC) based Deep Reinforcement Learning (DRL) SAC is a model-free DRL technique that is able to handle continuous action spaces.
  arXiv  Detail & Related papers  (2021-04-25T10:33:35Z)
• NeurOpt: Neural network based optimization for building energy management and climate control [58.06411999767069]
  We propose a data-driven control algorithm based on neural networks to reduce this cost of model identification. We validate our learning and control algorithms on a two-story building with ten independently controlled zones, located in Italy.
  arXiv  Detail & Related papers  (2020-01-22T00:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.