A New Error Temporal Difference Algorithm for Deep Reinforcement Learning in Microgrid Optimization

• URL: http://arxiv.org/abs/2511.18093v1
• Date: Sat, 22 Nov 2025 15:29:18 GMT
• Title: A New Error Temporal Difference Algorithm for Deep Reinforcement Learning in Microgrid Optimization
• Authors: Fulong Yao, Wanqing Zhao, Matthew Forshaw,
• Abstract summary: This paper presents a new error temporal difference (ETD) algorithm for DRL to address the uncertainty in predictions.<n>A microgrid system integrated with renewable energy sources (RES) and energy storage systems (ESS) is modelled.<n>Second, a predictive control approach based on a deep Q network (DQN) is presented, in which a weighted average algorithm and a new ETD algorithm are designed to quantify and address the prediction uncertainty.
• Score: 5.605678552776343
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Predictive control approaches based on deep reinforcement learning (DRL) have gained significant attention in microgrid energy optimization. However, existing research often overlooks the issue of uncertainty stemming from imperfect prediction models, which can lead to suboptimal control strategies. This paper presents a new error temporal difference (ETD) algorithm for DRL to address the uncertainty in predictions,aiming to improve the performance of microgrid operations. First,a microgrid system integrated with renewable energy sources (RES) and energy storage systems (ESS), along with its Markov decision process (MDP), is modelled. Second, a predictive control approach based on a deep Q network (DQN) is presented, in which a weighted average algorithm and a new ETD algorithm are designed to quantify and address the prediction uncertainty, respectively. Finally, simulations on a realworld US dataset suggest that the developed ETD effectively improves the performance of DRL in optimizing microgrid operations.

Related papers

• Approximate Model Predictive Control for Microgrid Energy Management via Imitation Learning [9.044455355747482]
  This paper proposes an imitation learning-based framework to approximate mixed-integer Economic Model Predictive Control (EMPC) for microgrid energy management.<n>The proposed method trains a neural network to imitate expert EMPC control actions from offline trajectories, enabling fast, real-time decision making without solving optimization problems online.
  arXiv  Detail & Related papers  (2025-10-22T21:39:18Z)
• RL-finetuning LLMs from on- and off-policy data with a single algorithm [53.70731390624718]
  We introduce a novel reinforcement learning algorithm (AGRO) for fine-tuning large-language models.<n>AGRO leverages the concept of generation consistency, which states that the optimal policy satisfies the notion of consistency across any possible generation of the model.<n>We derive algorithms that find optimal solutions via the sample-based policy gradient and provide theoretical guarantees on their convergence.
  arXiv  Detail & Related papers  (2025-03-25T12:52:38Z)
• Enhancing Microgrid Performance Prediction with Attention-based Deep Learning Models [0.0]
  This research aims to address microgrid systems' operational challenges, characterized by power oscillations that contribute to grid instability. An integrated strategy is proposed, leveraging the strengths of convolutional and Gated Recurrent Unit (GRU) layers. The framework is anchored by a Multi-Layer Perceptron (MLP) model, which is tasked with comprehensive load forecasting.
  arXiv  Detail & Related papers  (2024-07-20T21:24:11Z)
• Active RIS-aided EH-NOMA Networks: A Deep Reinforcement Learning Approach [66.53364438507208]
  An active reconfigurable intelligent surface (RIS)-aided multi-user downlink communication system is investigated. Non-orthogonal multiple access (NOMA) is employed to improve spectral efficiency, and the active RIS is powered by energy harvesting (EH) An advanced LSTM based algorithm is developed to predict users' dynamic communication state. A DDPG based algorithm is proposed to joint control the amplification matrix and phase shift matrix RIS.
  arXiv  Detail & Related papers  (2023-04-11T13:16:28Z)
• UAV Path Planning Employing MPC- Reinforcement Learning Method for search and rescue mission [0.0]
  We tackle the problem of Unmanned Aerial (UA V) path planning in complex and uncertain environments. We design a Model Predictive Control (MPC) based on a Long-Short-Term Memory (LSTM) network integrated into the Deep Deterministic Policy Gradient algorithm.
  arXiv  Detail & Related papers  (2023-02-21T13:39:40Z)
• Coverage and Capacity Optimization in STAR-RISs Assisted Networks: A Machine Learning Approach [102.00221938474344]
  A novel model is proposed for the coverage and capacity optimization of simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs) assisted networks. A loss function-based update strategy is the core point, which is able to calculate weights for both loss functions of coverage and capacity by a min-norm solver at each update. The numerical results demonstrate that the investigated update strategy outperforms the fixed weight-based MO algorithms.
  arXiv  Detail & Related papers  (2022-04-13T13:52:22Z)
• Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning [63.83425382922157]
  Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks. In this paper, we investigate a multi-user communication system assisted by cooperative IRS devices with the capability of energy harvesting.
  arXiv  Detail & Related papers  (2022-03-26T20:37:14Z)
• Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks [50.42141893913188]
  We study a distributed variable for large-scale AUC for a neural network as with a deep neural network. Our model requires a much less number of communication rounds and still a number of communication rounds in theory. Our experiments on several datasets show the effectiveness of our theory and also confirm our theory.
  arXiv  Detail & Related papers  (2020-05-05T18:08:23Z)
• Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning [21.770491711632832]
  The reconfigurable intelligent surface (RIS) has been speculated as one of the key enabling technologies for the future six generation (6G) wireless communication systems. In this paper, we investigate the joint design of transmit beamforming matrix at the base station and the phase shift matrix at the RIS, by leveraging recent advances in deep reinforcement learning (DRL) The proposed algorithm is not only able to learn from the environment and gradually improve its behavior, but also obtains the comparable performance compared with two state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2020-02-24T04:28:44Z)
• RIS Enhanced Massive Non-orthogonal Multiple Access Networks: Deployment and Passive Beamforming Design [116.88396201197533]
  A novel framework is proposed for the deployment and passive beamforming design of a reconfigurable intelligent surface (RIS) The problem of joint deployment, phase shift design, as well as power allocation is formulated for maximizing the energy efficiency. A novel long short-term memory (LSTM) based echo state network (ESN) algorithm is proposed to predict users' tele-traffic demand by leveraging a real dataset. A decaying double deep Q-network (D3QN) based position-acquisition and phase-control algorithm is proposed to solve the joint problem of deployment and design of the RIS.
  arXiv  Detail & Related papers  (2020-01-28T14:37:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.