Uncertainty-Aware Critic Augmentation for Hierarchical Multi-Agent EV Charging Control

• URL: http://arxiv.org/abs/2412.18047v3
• Date: Mon, 17 Feb 2025 11:19:13 GMT
• Title: Uncertainty-Aware Critic Augmentation for Hierarchical Multi-Agent EV Charging Control
• Authors: Lo Pang-Yun Ting, Ali Şenol, Huan-Yang Wang, Hsu-Chao Lai, Kun-Ta Chuang, Huan Liu,
• Abstract summary: We propose HUCA, a novel real-time charging control for regulating energy demands for both the building and EVs. HUCA employs hierarchical actor-critic networks to dynamically reduce electricity costs in buildings, accounting for the needs of EV charging in the dynamic pricing scenario. Experiments on real-world electricity datasets under both simulated certain and uncertain departure scenarios demonstrate that HUCA outperforms baselines in terms of total electricity costs.
• Score: 9.96602699887327
• License:
• Abstract: The advanced bidirectional EV charging and discharging technology, aimed at supporting grid stability and emergency operations, has driven a growing interest in workplace applications. It not only reduces electricity expenses but also enhances the resilience in handling practical matters, such as peak power limitation, fluctuating energy prices, and unpredictable EV departures. Considering these factors systematically can benefit energy efficiency in office buildings and for EV users simultaneously. To employ AI to address these issues, we propose HUCA, a novel real-time charging control for regulating energy demands for both the building and EVs. HUCA employs hierarchical actor-critic networks to dynamically reduce electricity costs in buildings, accounting for the needs of EV charging in the dynamic pricing scenario. To tackle the uncertain EV departures, we introduce a new critic augmentation to account for departure uncertainties in evaluating the charging decisions, while maintaining the robustness of the charging control. Experiments on real-world electricity datasets under both simulated certain and uncertain departure scenarios demonstrate that HUCA outperforms baselines in terms of total electricity costs while maintaining competitive performance in fulfilling EV charging requirements. A case study also manifests that HUCA effectively balances energy supply between the building and EVs based on real-time information, showcasing its potential as a key AI-driven solution for vehicle charging control.

Related papers

• A Deep Q-Learning based Smart Scheduling of EVs for Demand Response in Smart Grids [0.0]
  We propose a model-free solution, leveraging Deep Q-Learning to schedule the charging and discharging activities of EVs within a microgrid. We adapted the Bellman Equation to assess the value of a state based on specific rewards for EV scheduling actions and used a neural network to estimate Q-values for available actions and the epsilon-greedy algorithm to balance exploitation and exploration to meet the target energy profile.
  arXiv  Detail & Related papers  (2024-01-05T06:04:46Z)
• A Data-Driven Framework for Improving Public EV Charging Infrastructure: Modeling and Forecasting [13.950084838642228]
  It is suspected that the existing charging infrastructure will soon be no longer capable of sustaining the rapidly growing charging demands. Without suitable QoE metrics, operators, today, face remarkable difficulty in assessing the performance of EV Charging Stations. This paper aims at filling this gap through the formulation of novel and original critical QoE performance metrics.
  arXiv  Detail & Related papers  (2023-12-08T19:37:15Z)
• Electric Vehicles coordination for grid balancing using multi-objective Harris Hawks Optimization [0.0]
  The rise of renewables coincides with the shift towards Electrical Vehicles (EVs) posing technical and operational challenges for the energy balance of the local grid. Coordinating power flow from multiple EVs into the grid requires sophisticated algorithms and load-balancing strategies. This paper proposes an EVs fleet coordination model for the day ahead aiming to ensure a reliable energy supply and maintain a stable local grid.
  arXiv  Detail & Related papers  (2023-11-24T15:50:37Z)
• Charge Manipulation Attacks Against Smart Electric Vehicle Charging Stations and Deep Learning-based Detection Mechanisms [49.37592437398933]
  "Smart" electric vehicle charging stations (EVCSs) will be a key step toward achieving green transportation. We investigate charge manipulation attacks (CMAs) against EV charging, in which an attacker manipulates the information exchanged during smart charging operations. We propose an unsupervised deep learning-based mechanism to detect CMAs by monitoring the parameters involved in EV charging.
  arXiv  Detail & Related papers  (2023-10-18T18:38:59Z)
• An Efficient Distributed Multi-Agent Reinforcement Learning for EV Charging Network Control [2.5477011559292175]
  We introduce a decentralized Multi-agent Reinforcement Learning (MARL) charging framework that prioritizes the preservation of privacy for EV owners. Our results demonstrate that the CTDE framework improves the performance of the charging network by reducing the network costs.
  arXiv  Detail & Related papers  (2023-08-24T16:53:52Z)
• Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks [42.04263644600909]
  Multi-agent deep reinforcement learning (MADRL) has proven its effectiveness in EV charging control. Existing MADRL-based approaches fail to consider the natural power flow of EV charging/discharging in the distribution network. This paper proposes a novel approach that combines multi-EV charging/discharging with a radial distribution network (RDN) operating under optimal power flow.
  arXiv  Detail & Related papers  (2023-08-17T05:34:46Z)
• Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
  This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems. In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
  arXiv  Detail & Related papers  (2022-11-29T01:18:58Z)
• Federated Reinforcement Learning for Real-Time Electric Vehicle Charging and Discharging Control [42.17503767317918]
  This paper develops an optimal EV charging/discharging control strategy for different EV users under dynamic environments. A horizontal federated reinforcement learning (HFRL)-based method is proposed to fit various users' behaviors and dynamic environments. Simulation results illustrate that the proposed real-time EV charging/discharging control strategy can perform well among various factors.
  arXiv  Detail & Related papers  (2022-10-04T08:22:46Z)
• An Energy Consumption Model for Electrical Vehicle Networks via Extended Federated-learning [50.85048976506701]
  This paper proposes a novel solution to range anxiety based on a federated-learning model. It is capable of estimating battery consumption and providing energy-efficient route planning for vehicle networks.
  arXiv  Detail & Related papers  (2021-11-13T15:03:44Z)
• NeurOpt: Neural network based optimization for building energy management and climate control [58.06411999767069]
  We propose a data-driven control algorithm based on neural networks to reduce this cost of model identification. We validate our learning and control algorithms on a two-story building with ten independently controlled zones, located in Italy.
  arXiv  Detail & Related papers  (2020-01-22T00:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.