LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers

• URL: http://arxiv.org/abs/2511.00116v1
• Date: Fri, 31 Oct 2025 03:04:14 GMT
• Title: LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers
• Abstract summary: We present LC-Opt, a benchmark environment for machine learning-based liquid cooling control strategies.<n>Built on the baseline of a high-fidelity digital twin of Oak Ridge National Lab's Frontier Supercomputer cooling system, LC-Opt provides detailed Modelica-based end-to-end models.<n>We benchmark centralized and decentralized multi-agent RL approaches, demonstrate policy distillation into decision and regression trees for interpretable control, and explore LLM-based methods that explain control actions in natural language.
• Score: 18.187205962508646
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Liquid cooling is critical for thermal management in high-density data centers with the rising AI workloads. However, machine learning-based controllers are essential to unlock greater energy efficiency and reliability, promoting sustainability. We present LC-Opt, a Sustainable Liquid Cooling (LC) benchmark environment, for reinforcement learning (RL) control strategies in energy-efficient liquid cooling of high-performance computing (HPC) systems. Built on the baseline of a high-fidelity digital twin of Oak Ridge National Lab's Frontier Supercomputer cooling system, LC-Opt provides detailed Modelica-based end-to-end models spanning site-level cooling towers to data center cabinets and server blade groups. RL agents optimize critical thermal controls like liquid supply temperature, flow rate, and granular valve actuation at the IT cabinet level, as well as cooling tower (CT) setpoints through a Gymnasium interface, with dynamic changes in workloads. This environment creates a multi-objective real-time optimization challenge balancing local thermal regulation and global energy efficiency, and also supports additional components like a heat recovery unit (HRU). We benchmark centralized and decentralized multi-agent RL approaches, demonstrate policy distillation into decision and regression trees for interpretable control, and explore LLM-based methods that explain control actions in natural language through an agentic mesh architecture designed to foster user trust and simplify system management. LC-Opt democratizes access to detailed, customizable liquid cooling models, enabling the ML community, operators, and vendors to develop sustainable data center liquid cooling control solutions.

Related papers

• ThermoLLM: Thermodynamics-Aware HVAC Control with Spatial-Semantic Knowledge Graph [11.363638628487912]
  This paper presents a thermodynamics-aware LLM control framework for a five-zone EnergyPlus building simulation.<n>At each control step, the model receives the current building state, graph-structured spatial context, and recent environment-controller history.<n>Results show that the proposed approach achieves the best overall energy-comfort trade-off and the lowest PMV violation.
  arXiv  Detail & Related papers  (2026-06-22T06:51:42Z)
• Shielded Controller Units for RL with Operational Constraints Applied to Remote Microgrids [50.64533198075622]
  Reinforcement learning (RL) is a powerful framework for optimizing decision-making in complex systems under uncertainty.<n>In this paper, we introduce Shielded Controller Units (SCUs), a systematic and interpretable approach that leverages prior knowledge of system dynamics.<n>We demonstrate the effectiveness of SCUs on a remote microgrid optimization task with strict operational requirements.
  arXiv  Detail & Related papers  (2025-11-30T19:28:34Z)
• ScaleWeaver: Weaving Efficient Controllable T2I Generation with Multi-Scale Reference Attention [86.93601565563954]
  ScaleWeaver is a framework designed to achieve high-fidelity, controllable generation upon advanced visual autoregressive( VAR) models.<n>The proposed Reference Attention module discards the unnecessary attention from image$rightarrow$condition, reducing computational cost.<n>Experiments show that ScaleWeaver delivers high-quality generation and precise control while attaining superior efficiency over diffusion-based methods.
  arXiv  Detail & Related papers  (2025-10-16T17:00:59Z)
• Multi-Agent Architecture in Distributed Environment Control Systems: vision, challenges, and opportunities [50.38638300332429]
  We propose a multi-agent architecture for distributed control of air-cooled chiller systems in data centers.<n>Our vision employs autonomous agents to monitor and regulate local operational parameters and optimize system-wide efficiency.
  arXiv  Detail & Related papers  (2025-02-21T18:41:03Z)
• Hierarchical Multi-Agent Framework for Carbon-Efficient Liquid-Cooled Data Center Clusters [5.335496791443277]
  This paper introduces Green-DCC, which proposes a Reinforcement Learning (RL) based hierarchical controller to optimize both workload and liquid cooling dynamically in a DCC.<n>We demonstrate how the system optimize multiple data centers synchronously, enabling the scope of digital twins, and compare the performance of various RL approaches based on carbon emissions and sustainability metrics.
  arXiv  Detail & Related papers  (2025-02-12T12:00:58Z)
• Sustainability of Data Center Digital Twins with Reinforcement Learning [2.4971633082970377]
  Machine learning (ML) has led to an increased demand for computational power, resulting in larger data centers (DCs) and higher energy consumption. To address this issue and reduce carbon emissions, intelligent design and control of DC components such as IT servers, cabinets, HVAC cooling, flexible load shifting, and battery energy storage are essential. DCRL-Green is a multi-agent RL environment that empowers the ML community to design data centers and research, develop, and refine RL controllers for carbon footprint reduction in DCs.
  arXiv  Detail & Related papers  (2024-04-16T18:22:30Z)
• An LLM-Based Digital Twin for Optimizing Human-in-the Loop Systems [13.388869442538399]
  We present a case study that employs large language models (LLMs) to mimic the behaviors and thermal preferences of various population groups in a shopping mall. The aggregated thermal preferences are integrated into an agent-in-the-loop based reinforcement learning algorithm AitL-RL. Our results show that LLMs are capable of simulating complex population movements within large open spaces.
  arXiv  Detail & Related papers  (2024-03-25T14:32:28Z)
• Autonomous Payload Thermal Control [55.2480439325792]
  In small satellites there is less room for heat control equipment, scientific instruments, and electronic components. An autonomous thermal control tool that uses deep reinforcement learning is proposed for learning the thermal control policy onboard. The proposed framework is able to learn to control the payload processing power to maintain the temperature under operational ranges.
  arXiv  Detail & Related papers  (2023-07-28T09:40:19Z)
• Laxity-Aware Scalable Reinforcement Learning for HVAC Control [2.0625936401496237]
  We tackle the curse of dimensionality issue in modeling and control by utilizing the concept of laxity to quantify the emergency level of each operation request. We propose a two-level approach to address energy optimization for a large population of HVAC systems.
  arXiv  Detail & Related papers  (2023-06-29T01:28:14Z)
• Low Emission Building Control with Zero-Shot Reinforcement Learning [70.70479436076238]
  Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency. We show it is possible to obtain emission-reducing policies without a priori--a paradigm we call zero-shot building control.
  arXiv  Detail & Related papers  (2022-08-12T17:13:25Z)
• Data-Driven Learning and Load Ensemble Control [1.647866856596524]
  This study aims to engage distributed small-scale flexible loads, such as thermostatically controllable loads (TCLs) to provide grid support services. The efficiency of this data-driven learning is demonstrated through simulations on Heating, Cooling & Ventilation units in a testbed neighborhood of residential houses.
  arXiv  Detail & Related papers  (2020-04-20T23:32:10Z)
• NeurOpt: Neural network based optimization for building energy management and climate control [58.06411999767069]
  We propose a data-driven control algorithm based on neural networks to reduce this cost of model identification. We validate our learning and control algorithms on a two-story building with ten independently controlled zones, located in Italy.
  arXiv  Detail & Related papers  (2020-01-22T00:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.