CiRL: Open-Source Environments for Reinforcement Learning in Circular Economy and Net Zero

• URL: http://arxiv.org/abs/2505.21536v1
• Date: Sat, 24 May 2025 08:26:14 GMT
• Title: CiRL: Open-Source Environments for Reinforcement Learning in Circular Economy and Net Zero
• Authors: Federico Zocco, Andrea Corti, Monica Malvezzi,
• Abstract summary: We introduce CiRL, a library of environments focused on the circularity of both solid and fluid materials.<n>Along with the focus on circularity, this library has three more features: the new CE-oriented environments are in the state-space form, which is typically used in dynamical systems analysis and control designs.
• Score: 4.062511856086843
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The demand of finite raw materials will keep increasing as they fuel modern society. Simultaneously, solutions for stopping carbon emissions in the short term are not available, thus making the net zero target extremely challenging to achieve at scale. The circular economy (CE) paradigm is gaining attention as a solution to address climate change and the uncertainties of supplies of critical materials. Hence, in this paper, we introduce CiRL, a deep reinforcement learning (DRL) library of environments focused on the circularity of both solid and fluid materials. The integration of DRL into the design of material circularity is possible thanks to the formalism of thermodynamical material networks, which is underpinned by compartmental dynamical thermodynamics. Along with the focus on circularity, this library has three more features: the new CE-oriented environments are in the state-space form, which is typically used in dynamical systems analysis and control designs; it is based on a state-of-the-art Python library of DRL algorithms, namely, Stable-Baselines3; and it is developed in Google Colaboratory to be accessible to researchers from different disciplines and backgrounds as is often the case for circular economy researchers and engineers. CiRL is publicly available.

Related papers

• Beyond Accuracy: EcoL2 Metric for Sustainable Neural PDE Solvers [11.268342044762463]
  This paper introduces a carbon emission measure for a range of PDE solvers.<n>Our proposed metric, EcoL2, balances model accuracy with emissions across data collection, model training, and deployment.<n>As such solvers grow in scale and deployment, EcoL2 represents a step toward building performant scientific machine learning systems with lower long-term environmental impact.
  arXiv  Detail & Related papers  (2025-05-18T22:05:11Z)
• Hierarchical Multi-Agent Framework for Carbon-Efficient Liquid-Cooled Data Center Clusters [5.335496791443277]
  This paper introduces Green-DCC, which proposes a Reinforcement Learning (RL) based hierarchical controller to optimize both workload and liquid cooling dynamically in a DCC.<n>We demonstrate how the system optimize multiple data centers synchronously, enabling the scope of digital twins, and compare the performance of various RL approaches based on carbon emissions and sustainability metrics.
  arXiv  Detail & Related papers  (2025-02-12T12:00:58Z)
• A Benchmark Environment for Offline Reinforcement Learning in Racing Games [54.83171948184851]
  Offline Reinforcement Learning (ORL) is a promising approach to reduce the high sample complexity of traditional Reinforcement Learning (RL) This paper introduces OfflineMania, a novel environment for ORL research. It is inspired by the iconic TrackMania series and developed using the Unity 3D game engine.
  arXiv  Detail & Related papers  (2024-07-12T16:44:03Z)
• Sustainability of Data Center Digital Twins with Reinforcement Learning [2.4971633082970377]
  Machine learning (ML) has led to an increased demand for computational power, resulting in larger data centers (DCs) and higher energy consumption. To address this issue and reduce carbon emissions, intelligent design and control of DC components such as IT servers, cabinets, HVAC cooling, flexible load shifting, and battery energy storage are essential. DCRL-Green is a multi-agent RL environment that empowers the ML community to design data centers and research, develop, and refine RL controllers for carbon footprint reduction in DCs.
  arXiv  Detail & Related papers  (2024-04-16T18:22:30Z)
• Confidence-Controlled Exploration: Efficient Sparse-Reward Policy Learning for Robot Navigation [72.24964965882783]
  Reinforcement learning (RL) is a promising approach for robotic navigation, allowing robots to learn through trial and error.<n>Real-world robotic tasks often suffer from sparse rewards, leading to inefficient exploration and suboptimal policies.<n>We introduce Confidence-Controlled Exploration (CCE), a novel method that improves sample efficiency in RL-based robotic navigation without modifying the reward function.
  arXiv  Detail & Related papers  (2023-06-09T18:45:15Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Low Emission Building Control with Zero-Shot Reinforcement Learning [70.70479436076238]
  Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency. We show it is possible to obtain emission-reducing policies without a priori--a paradigm we call zero-shot building control.
  arXiv  Detail & Related papers  (2022-08-12T17:13:25Z)
• GriddlyJS: A Web IDE for Reinforcement Learning [7.704064306361941]
  We introduce GriddlyJS, a web-based Integrated Development Environment (IDE) based on the Griddly engine. GriddlyJS allows researchers to visually design and debug arbitrary, complex PCG grid-world environments. By connecting the RL workflow to the advanced functionality enabled by modern web standards, GriddlyJS allows publishing interactive agent-environment demos.
  arXiv  Detail & Related papers  (2022-07-13T10:26:38Z)
• An Energy and Carbon Footprint Analysis of Distributed and Federated Learning [42.37180749113699]
  Classical and centralized Artificial Intelligence (AI) methods require moving data from producers (sensors, machines) to energy hungry data centers. Emerging alternatives to mitigate such high energy costs propose to efficiently distribute, or federate, the learning tasks across devices. This paper proposes a novel framework for the analysis of energy and carbon footprints in distributed and federated learning.
  arXiv  Detail & Related papers  (2022-06-21T13:28:49Z)
• Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence [76.96698721128406]
  Mobile edge computing (MEC) considered a novel paradigm for computation and delay-sensitive tasks in fifth generation (5G) networks and beyond. This paper provides a comprehensive research review on free-enabled RL and offers insight for development.
  arXiv  Detail & Related papers  (2022-01-27T10:02:54Z)
• A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learning [48.63610479916003]
  This article breaks down and analyzes the main factors that influence the environmental footprint of distributed learning policies. It models both vanilla and decentralized FL policies driven by consensus. Results show that FL allows remarkable end-to-end energy savings (30%-40%) for wireless systems characterized by low bit/Joule efficiency.
  arXiv  Detail & Related papers  (2021-03-18T16:04:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.