Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks

• URL: http://arxiv.org/abs/2308.08792v1
• Date: Thu, 17 Aug 2023 05:34:46 GMT
• Title: Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks
• Authors: Junkai Qian and Yuning Jiang and Xin Liu and Qing Wang and Ting Wang and Yuanming Shi and Wei Chen
• Abstract summary: Multi-agent deep reinforcement learning (MADRL) has proven its effectiveness in EV charging control. Existing MADRL-based approaches fail to consider the natural power flow of EV charging/discharging in the distribution network. This paper proposes a novel approach that combines multi-EV charging/discharging with a radial distribution network (RDN) operating under optimal power flow.
• Score: 42.04263644600909
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: With the growing popularity of electric vehicles (EVs), maintaining power grid stability has become a significant challenge. To address this issue, EV charging control strategies have been developed to manage the switch between vehicle-to-grid (V2G) and grid-to-vehicle (G2V) modes for EVs. In this context, multi-agent deep reinforcement learning (MADRL) has proven its effectiveness in EV charging control. However, existing MADRL-based approaches fail to consider the natural power flow of EV charging/discharging in the distribution network and ignore driver privacy. To deal with these problems, this paper proposes a novel approach that combines multi-EV charging/discharging with a radial distribution network (RDN) operating under optimal power flow (OPF) to distribute power flow in real time. A mathematical model is developed to describe the RDN load. The EV charging control problem is formulated as a Markov Decision Process (MDP) to find an optimal charging control strategy that balances V2G profits, RDN load, and driver anxiety. To effectively learn the optimal EV charging control strategy, a federated deep reinforcement learning algorithm named FedSAC is further proposed. Comprehensive simulation results demonstrate the effectiveness and superiority of our proposed algorithm in terms of the diversity of the charging control strategy, the power fluctuations on RDN, the convergence efficiency, and the generalization ability.

Related papers

• Centralized vs. Decentralized Multi-Agent Reinforcement Learning for Enhanced Control of Electric Vehicle Charging Networks [1.9188272016043582]
  We introduce a novel approach for distributed and cooperative charging strategy using a Multi-Agent Reinforcement Learning (MARL) framework. Our method is built upon the Deep Deterministic Policy Gradient (DDPG) algorithm for a group of EVs in a residential community. Our results indicate that, despite higher policy variances and training complexity, the CTDE-DDPG framework significantly improves charging efficiency by reducing total variation by approximately %36 and charging cost by around %9.1 on average.
  arXiv  Detail & Related papers  (2024-04-18T21:50:03Z)
• Safety-Aware Reinforcement Learning for Electric Vehicle Charging Station Management in Distribution Network [4.842172685255376]
  Electric vehicles (EVs) pose a significant risk to the distribution system operation in the absence of coordination. This paper presents a safety-aware reinforcement learning (RL) algorithm designed to manage EV charging stations. Our proposed algorithm does not rely on explicit penalties for constraint violations, eliminating the need for penalty tuning coefficient.
  arXiv  Detail & Related papers  (2024-03-20T01:57:38Z)
• Charge Manipulation Attacks Against Smart Electric Vehicle Charging Stations and Deep Learning-based Detection Mechanisms [49.37592437398933]
  "Smart" electric vehicle charging stations (EVCSs) will be a key step toward achieving green transportation. We investigate charge manipulation attacks (CMAs) against EV charging, in which an attacker manipulates the information exchanged during smart charging operations. We propose an unsupervised deep learning-based mechanism to detect CMAs by monitoring the parameters involved in EV charging.
  arXiv  Detail & Related papers  (2023-10-18T18:38:59Z)
• An Efficient Distributed Multi-Agent Reinforcement Learning for EV Charging Network Control [2.5477011559292175]
  We introduce a decentralized Multi-agent Reinforcement Learning (MARL) charging framework that prioritizes the preservation of privacy for EV owners. Our results demonstrate that the CTDE framework improves the performance of the charging network by reducing the network costs.
  arXiv  Detail & Related papers  (2023-08-24T16:53:52Z)
• Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
  This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems. In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
  arXiv  Detail & Related papers  (2022-11-29T01:18:58Z)
• Transfer Deep Reinforcement Learning-based Large-scale V2G Continuous Charging Coordination with Renewable Energy Sources [5.99526159525785]
  Vehicle-to-grid (V2G) technique and large-scale scheduling algorithms have been developed to achieve a high level of renewable energy and power grid stability. This paper proposes a deep reinforcement learning (DRL) method for the continuous charging/discharging coordination strategy.
  arXiv  Detail & Related papers  (2022-10-13T13:21:55Z)
• Federated Reinforcement Learning for Real-Time Electric Vehicle Charging and Discharging Control [42.17503767317918]
  This paper develops an optimal EV charging/discharging control strategy for different EV users under dynamic environments. A horizontal federated reinforcement learning (HFRL)-based method is proposed to fit various users' behaviors and dynamic environments. Simulation results illustrate that the proposed real-time EV charging/discharging control strategy can perform well among various factors.
  arXiv  Detail & Related papers  (2022-10-04T08:22:46Z)
• Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer [128.19212716007794]
  We propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency.
  arXiv  Detail & Related papers  (2022-06-08T07:48:42Z)
• Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training [71.7750435554693]
  We show that several state-of-the-art RL agents proposed for power system control are vulnerable to adversarial attacks. Specifically, we use an adversary Markov Decision Process to learn an attack policy, and demonstrate the potency of our attack. We propose to use adversarial training to increase the robustness of RL agent against attacks and avoid infeasible operational decisions.
  arXiv  Detail & Related papers  (2021-10-18T00:50:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.