Reinforcement Learning-based Approach for Vehicle-to-Building Charging with Heterogeneous Agents and Long Term Rewards

• URL: http://arxiv.org/abs/2502.18526v1
• Date: Mon, 24 Feb 2025 19:24:41 GMT
• Title: Reinforcement Learning-based Approach for Vehicle-to-Building Charging with Heterogeneous Agents and Long Term Rewards
• Authors: Fangqi Liu, Rishav Sen, Jose Paolo Talusan, Ava Pettet, Aaron Kandel, Yoshinori Suzue, Ayan Mukhopadhyay, Abhishek Dubey,
• Abstract summary: We introduce a novel RL framework that combines the Deep Deterministic Policy Gradient approach with action masking and efficient MILP-driven policy guidance.<n>Our approach balances the exploration of continuous action spaces to meet user charging demands.<n>Our results show that the proposed approach is one of the first scalable and general approaches to solving the V2B energy management challenge.
• Score: 3.867907469895697
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Strategic aggregation of electric vehicle batteries as energy reservoirs can optimize power grid demand, benefiting smart and connected communities, especially large office buildings that offer workplace charging. This involves optimizing charging and discharging to reduce peak energy costs and net peak demand, monitored over extended periods (e.g., a month), which involves making sequential decisions under uncertainty and delayed and sparse rewards, a continuous action space, and the complexity of ensuring generalization across diverse conditions. Existing algorithmic approaches, e.g., heuristic-based strategies, fall short in addressing real-time decision-making under dynamic conditions, and traditional reinforcement learning (RL) models struggle with large state-action spaces, multi-agent settings, and the need for long-term reward optimization. To address these challenges, we introduce a novel RL framework that combines the Deep Deterministic Policy Gradient approach (DDPG) with action masking and efficient MILP-driven policy guidance. Our approach balances the exploration of continuous action spaces to meet user charging demands. Using real-world data from a major electric vehicle manufacturer, we show that our approach comprehensively outperforms many well-established baselines and several scalable heuristic approaches, achieving significant cost savings while meeting all charging requirements. Our results show that the proposed approach is one of the first scalable and general approaches to solving the V2B energy management challenge.

Related papers

• Advancing Generative Artificial Intelligence and Large Language Models for Demand Side Management with Internet of Electric Vehicles [52.43886862287498]
  This paper explores the integration of large language models (LLMs) into energy management.<n>We propose an innovative solution that enhances LLMs with retrieval-augmented generation for automatic problem formulation, code generation, and customizing optimization.<n>We present a case study to demonstrate the effectiveness of our proposed solution in charging scheduling and optimization for electric vehicles.
  arXiv  Detail & Related papers  (2025-01-26T14:31:03Z)
• Optimizing Load Scheduling in Power Grids Using Reinforcement Learning and Markov Decision Processes [0.0]
  This paper proposes a reinforcement learning (RL) approach to address the challenges of dynamic load scheduling. Our results show that the RL-based method provides a robust and scalable solution for real-time load scheduling.
  arXiv  Detail & Related papers  (2024-10-23T09:16:22Z)
• EnergAIze: Multi Agent Deep Deterministic Policy Gradient for Vehicle to Grid Energy Management [0.0]
  This paper introduces EnergAIze, a Multi-Agent Reinforcement Learning (MARL) energy management framework. It enables user-centric and multi-objective energy management by allowing each prosumer to select from a range of personal management objectives. The efficacy of EnergAIze was evaluated through case studies employing the CityLearn simulation framework.
  arXiv  Detail & Related papers  (2024-04-02T23:16:17Z)
• A Human-on-the-Loop Optimization Autoformalism Approach for Sustainability [27.70596933019959]
  This paper outlines a natural conversational approach to solving personalized energy-related problems using large language models (LLMs) We put forward a strategy that augments an LLM with an optimization solver, enhancing its proficiency in understanding and responding to user specifications and preferences. Our approach pioneers the novel concept of human-guided optimization autoformalism, translating a natural language task specification automatically into an optimization instance.
  arXiv  Detail & Related papers  (2023-08-20T22:42:04Z)
• Sustainable AIGC Workload Scheduling of Geo-Distributed Data Centers: A Multi-Agent Reinforcement Learning Approach [48.18355658448509]
  Recent breakthroughs in generative artificial intelligence have triggered a surge in demand for machine learning training, which poses significant cost burdens and environmental challenges due to its substantial energy consumption. Scheduling training jobs among geographically distributed cloud data centers unveils the opportunity to optimize the usage of computing capacity powered by inexpensive and low-carbon energy. We propose an algorithm based on multi-agent reinforcement learning and actor-critic methods to learn the optimal collaborative scheduling strategy through interacting with a cloud system built with real-life workload patterns, energy prices, and carbon intensities.
  arXiv  Detail & Related papers  (2023-04-17T02:12:30Z)
• Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
  This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems. In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
  arXiv  Detail & Related papers  (2022-11-29T01:18:58Z)
• Optimized cost function for demand response coordination of multiple EV charging stations using reinforcement learning [6.37470346908743]
  We build on previous research on RL, based on a Markov decision process (MDP) to simultaneously coordinate multiple charging stations. We propose an improved cost function that essentially forces the learned control policy to always fulfill any charging demand that does not offer flexibility. We rigorously compare the newly proposed batch RL fitted Q-iteration implementation with the original (costly) one, using real-world data.
  arXiv  Detail & Related papers  (2022-03-03T11:22:27Z)
• Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning [42.31586065609373]
  Electric Vehicle (EV) has become a choice in the modern transportation system due to its environmental and energy sustainability. In many cities, EV drivers often fail to find the proper spots for charging, because of the limited charging infrastructures and the largely unbalanced charging demands. We propose a framework, named Multi-Agent Spatiotemporal-temporal ment Learning (MasterReinforce), for intelligently recommending public charging stations.
  arXiv  Detail & Related papers  (2021-02-15T06:23:59Z)
• Demand-Side Scheduling Based on Multi-Agent Deep Actor-Critic Learning for Smart Grids [56.35173057183362]
  We consider the problem of demand-side energy management, where each household is equipped with a smart meter that is able to schedule home appliances online. The goal is to minimize the overall cost under a real-time pricing scheme. We propose the formulation of a smart grid environment as a Markov game.
  arXiv  Detail & Related papers  (2020-05-05T07:32:40Z)
• Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
  We study a risk-aware energy scheduling problem for a microgrid-powered MEC network. We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
  arXiv  Detail & Related papers  (2020-02-21T02:14:38Z)
• Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems [87.4519172058185]
  An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
  arXiv  Detail & Related papers  (2020-02-20T04:58:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.