Towards Affordable Energy: A Gymnasium Environment for Electric Utility Demand-Response Programs

• URL: http://arxiv.org/abs/2605.12462v1
• Date: Tue, 12 May 2026 17:48:45 GMT
• Title: Towards Affordable Energy: A Gymnasium Environment for Electric Utility Demand-Response Programs
• Abstract summary: DR-Gym is an open-source, online Gymnasium-compatible environment designed to train and evaluate demand-response from the electric utility's perspective.<n>Our environment focuses on the market-level electric utility setting and provides a rich observational space relevant to the electric utility.<n>For our learning signal, we use a multi-objective reward function for specifying diverse learning objectives.
• Score: 15.560423769248766
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Extreme weather and volatile wholesale electricity markets expose residential consumers to catastrophic financial risks, yet demand response at the distribution level remains an underutilized tool for grid flexibility and energy affordability. While a demand-response program can shield consumers by issuing financial credits during high-price periods, optimizing this sequential decision-making process presents a unique challenge for reinforcement learning despite the plentiful offline historical smart meter and wholesale pricing data available publicly. Offline historical data fails to capture the dynamic, interactive feedback loop between an electric utility's pricing signals and customer acceptance and adaptation to a demand-response program. To address this, we introduce DR-Gym, an open-source, online Gymnasium-compatible environment designed to train and evaluate demand-response from the electric utility's perspective. Unlike existing device-level energy simulators, our environment focuses on the market-level electric utility setting and provides a rich observational space relevant to the electric utility. The simulator additionally features a regime-switching wholesale price model calibrated to real-world extreme events, alongside physics-based building demand profiles. For our learning signal, we use a configurable, multi-objective reward function for specifying diverse learning objectives. We demonstrate through baseline strategies and data snapshots the capability of our simulator to create realistic and learnable environments.

Related papers

• LoaDiff: Conditional Generation of Electricity Consumption Time Series for Energy Analytics [6.062089991172254]
  We introduce LoaDiff, a diffusion-based generative model for year-long, sub-hourly smart-meter load curves.<n>We evaluate the model against multiple generative baselines on three residential electricity-consumption datasets.
  arXiv  Detail & Related papers  (2026-09-10T14:46:30Z)
• An Automated Framework for Assessing Electric Vehicle Charging Impacts on a Campus Distribution Grid [0.6978367196609415]
  This paper introduces a unified and automated framework designed to assess the impact of electric vehicle (EV) charging on distribution feeders and transformers at California State University, Northridge (CSUN)<n>Our main contribution is the development of a flexible testbed that integrates Julia, a high-performance programming language for technical computing, with PowerWorld Simulator via the EasySimauto.jl package.<n>The proposed system offers a valuable tool for evaluating transformer loading, feeder utilization, and overall system stress.
  arXiv  Detail & Related papers  (2025-09-09T01:17:59Z)
• Estimating the Unobservable Components of Electricity Demand Response with Inverse Optimization [0.0]
  Understanding and predicting electricity demand responses to prices are critical activities for system operators, retailers, and regulators. The emergence of active consumers with flexible assets such as solar-plus-storage systems, and electric vehicles, introduces new challenges. We define the main components of net demand, as baseload, flexible, and self-generation, each having nonlinear responses to market price signals.
  arXiv  Detail & Related papers  (2024-09-17T14:17:31Z)
• A Deep Q-Learning based Smart Scheduling of EVs for Demand Response in Smart Grids [0.0]
  We propose a model-free solution, leveraging Deep Q-Learning to schedule the charging and discharging activities of EVs within a microgrid. We adapted the Bellman Equation to assess the value of a state based on specific rewards for EV scheduling actions and used a neural network to estimate Q-values for available actions and the epsilon-greedy algorithm to balance exploitation and exploration to meet the target energy profile.
  arXiv  Detail & Related papers  (2024-01-05T06:04:46Z)
• Optimal Scheduling of Electric Vehicle Charging with Deep Reinforcement Learning considering End Users Flexibility [1.3812010983144802]
  This work aims to identify households' EV cost-reducing charging policy under a Time-of-Use tariff scheme, with the use of Deep Reinforcement Learning, and more specifically Deep Q-Networks (DQN) A novel end users flexibility potential reward is inferred from historical data analysis, where households with solar power generation have been used to train and test the algorithm.
  arXiv  Detail & Related papers  (2023-10-13T12:07:36Z)
• ILB: Graph Neural Network Enabled Emergency Demand Response Program For Electricity [6.123324869194196]
  In times of crisis, an emergency Demand Response program is required to manage unexpected spikes in energy demand. We propose the Incentive-Driven Load Balancer (ILB), a program designed to efficiently manage demand and response during crisis situations. We introduce a two-step machine learning-based framework for participant selection, which employs a graph-based approach to identify households capable of easily adjusting their electricity consumption.
  arXiv  Detail & Related papers  (2023-09-29T20:38:04Z)
• Machine learning applications for electricity market agent-based models: A systematic literature review [68.8204255655161]
  Agent-based simulations are used to better understand the dynamics of the electricity market. Agent-based models provide the opportunity to integrate machine learning and artificial intelligence. We review 55 papers published between 2016 and 2021 which focus on machine learning applied to agent-based electricity market models.
  arXiv  Detail & Related papers  (2022-06-05T14:52:26Z)
• The impact of online machine-learning methods on long-term investment decisions and generator utilization in electricity markets [69.68068088508505]
  We investigate the impact of eleven offline and five online learning algorithms to predict the electricity demand profile over the next 24h. We show we can reduce the mean absolute error by 30% using an online algorithm when compared to the best offline algorithm. We also show that large errors in prediction accuracy have a disproportionate error on investments made over a 17-year time frame.
  arXiv  Detail & Related papers  (2021-03-07T11:28:54Z)
• Investigating Underlying Drivers of Variability in Residential Energy Usage Patterns with Daily Load Shape Clustering of Smart Meter Data [53.51471969978107]
  Large-scale deployment of smart meters has motivated increasing studies to explore disaggregated daily load patterns. This paper aims to shed light on the mechanisms by which electricity consumption patterns exhibit variability.
  arXiv  Detail & Related papers  (2021-02-16T16:56:27Z)
• To Talk or to Work: Flexible Communication Compression for Energy Efficient Federated Learning over Heterogeneous Mobile Edge Devices [78.38046945665538]
  federated learning (FL) over massive mobile edge devices opens new horizons for numerous intelligent mobile applications. FL imposes huge communication and computation burdens on participating devices due to periodical global synchronization and continuous local training. We develop a convergence-guaranteed FL algorithm enabling flexible communication compression.
  arXiv  Detail & Related papers  (2020-12-22T02:54:18Z)
• A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids [58.666456917115056]
  This paper presents a Reinforcement Learning based energy market for a prosumer dominated microgrid. The proposed market model facilitates a real-time and demanddependent dynamic pricing environment, which reduces grid costs and improves the economic benefits for prosumers.
  arXiv  Detail & Related papers  (2020-09-23T02:17:51Z)
• Demand-Side Scheduling Based on Multi-Agent Deep Actor-Critic Learning for Smart Grids [56.35173057183362]
  We consider the problem of demand-side energy management, where each household is equipped with a smart meter that is able to schedule home appliances online. The goal is to minimize the overall cost under a real-time pricing scheme. We propose the formulation of a smart grid environment as a Markov game.
  arXiv  Detail & Related papers  (2020-05-05T07:32:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.