Attentive Convolutional Deep Reinforcement Learning for Optimizing Solar-Storage Systems in Real-Time Electricity Markets

• URL: http://arxiv.org/abs/2401.15853v1
• Date: Mon, 29 Jan 2024 03:04:43 GMT
• Title: Attentive Convolutional Deep Reinforcement Learning for Optimizing Solar-Storage Systems in Real-Time Electricity Markets
• Authors: Jinhao Li, Changlong Wang, Hao Wang
• Abstract summary: We study the synergy of solar-battery energy storage system (BESS) and develop a viable strategy for the BESS to unlock its economic potential. We develop a novel deep reinforcement learning (DRL) algorithm to solve the problem by leveraging attention mechanism (AC) and multi-grained feature convolution.
• Score: 5.1888966391612605
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper studies the synergy of solar-battery energy storage system (BESS) and develops a viable strategy for the BESS to unlock its economic potential by serving as a backup to reduce solar curtailments while also participating in the electricity market. We model the real-time bidding of the solar-battery system as two Markov decision processes for the solar farm and the BESS, respectively. We develop a novel deep reinforcement learning (DRL) algorithm to solve the problem by leveraging attention mechanism (AC) and multi-grained feature convolution to process DRL input for better bidding decisions. Simulation results demonstrate that our AC-DRL outperforms two optimization-based and one DRL-based benchmarks by generating 23%, 20%, and 11% higher revenue, as well as improving curtailment responses. The excess solar generation can effectively charge the BESS to bid in the market, significantly reducing solar curtailments by 76% and creating synergy for the solar-battery system to be more viable.

Related papers

• Learning a local trading strategy: deep reinforcement learning for grid-scale renewable energy integration [0.0]
  This paper explores the use of reinforcement learning for operating grid-scale batteries co-located with solar power. Our results show RL achieves an average of 61% (and up to 96%) of the approximate theoretical optimal (non-causal) operation.
  arXiv  Detail & Related papers  (2024-11-23T02:55:38Z)
• Deep Reinforcement Learning for Community Battery Scheduling under Uncertainties of Load, PV Generation, and Energy Prices [5.694872363688119]
  This paper presents a deep reinforcement learning (RL) strategy to schedule a community battery system in the presence of uncertainties. We position the community battery to play a versatile role, in integrating local PV energy, reducing peak load, and exploiting energy price fluctuations for arbitrage.
  arXiv  Detail & Related papers  (2023-12-04T13:45:17Z)
• Deep Reinforcement Learning for Wind and Energy Storage Coordination in Wholesale Energy and Ancillary Service Markets [5.1888966391612605]
  Wind curtailment can be reduced using battery energy storage systems (BESS) as onsite backup sources. We propose a novel deep reinforcement learning-based approach that decouples the system's market participation into two related Markov decision processes. Our results show that joint-market bidding can significantly improve the financial performance of wind-battery systems.
  arXiv  Detail & Related papers  (2022-12-27T05:51:54Z)
• Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
  This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems. In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
  arXiv  Detail & Related papers  (2022-11-29T01:18:58Z)
• Solar Power driven EV Charging Optimization with Deep Reinforcement Learning [6.936743119804558]
  Decentralized energy resources, such as Electric Vehicles (EV) and solar photovoltaic systems (PV), are continuously integrated in residential power systems. This paper aims to address the challenge of domestic EV charging while prioritizing clean, solar energy consumption.
  arXiv  Detail & Related papers  (2022-11-17T11:52:27Z)
• Low Emission Building Control with Zero-Shot Reinforcement Learning [70.70479436076238]
  Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency. We show it is possible to obtain emission-reducing policies without a priori--a paradigm we call zero-shot building control.
  arXiv  Detail & Related papers  (2022-08-12T17:13:25Z)
• Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training [71.7750435554693]
  We show that several state-of-the-art RL agents proposed for power system control are vulnerable to adversarial attacks. Specifically, we use an adversary Markov Decision Process to learn an attack policy, and demonstrate the potency of our attack. We propose to use adversarial training to increase the robustness of RL agent against attacks and avoid infeasible operational decisions.
  arXiv  Detail & Related papers  (2021-10-18T00:50:34Z)
• Optimizing a domestic battery and solar photovoltaic system with deep reinforcement learning [69.68068088508505]
  A lowering in the cost of batteries and solar PV systems has led to a high uptake of solar battery home systems. In this work, we use the deep deterministic policy algorithm to optimise the charging and discharging behaviour of a battery within such a system.
  arXiv  Detail & Related papers  (2021-09-10T10:59:14Z)
• A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids [58.666456917115056]
  This paper presents a Reinforcement Learning based energy market for a prosumer dominated microgrid. The proposed market model facilitates a real-time and demanddependent dynamic pricing environment, which reduces grid costs and improves the economic benefits for prosumers.
  arXiv  Detail & Related papers  (2020-09-23T02:17:51Z)
• Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems [87.4519172058185]
  An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
  arXiv  Detail & Related papers  (2020-02-20T04:58:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.