Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2102.07359v1
• Date: Mon, 15 Feb 2021 06:23:59 GMT
• Title: Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning
• Authors: Weijia Zhang, Hao Liu, Fan Wang, Tong Xu, Haoran Xin, Dejing Dou, Hui Xiong
• Abstract summary: Electric Vehicle (EV) has become a choice in the modern transportation system due to its environmental and energy sustainability. In many cities, EV drivers often fail to find the proper spots for charging, because of the limited charging infrastructures and the largely unbalanced charging demands. We propose a framework, named Multi-Agent Spatiotemporal-temporal ment Learning (MasterReinforce), for intelligently recommending public charging stations.
• Score: 42.31586065609373
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Electric Vehicle (EV) has become a preferable choice in the modern transportation system due to its environmental and energy sustainability. However, in many large cities, EV drivers often fail to find the proper spots for charging, because of the limited charging infrastructures and the spatiotemporally unbalanced charging demands. Indeed, the recent emergence of deep reinforcement learning provides great potential to improve the charging experience from various aspects over a long-term horizon. In this paper, we propose a framework, named Multi-Agent Spatio-Temporal Reinforcement Learning (Master), for intelligently recommending public accessible charging stations by jointly considering various long-term spatiotemporal factors. Specifically, by regarding each charging station as an individual agent, we formulate this problem as a multi-objective multi-agent reinforcement learning task. We first develop a multi-agent actor-critic framework with the centralized attentive critic to coordinate the recommendation between geo-distributed agents. Moreover, to quantify the influence of future potential charging competition, we introduce a delayed access strategy to exploit the knowledge of future charging competition during training. After that, to effectively optimize multiple learning objectives, we extend the centralized attentive critic to multi-critics and develop a dynamic gradient re-weighting strategy to adaptively guide the optimization direction. Finally, extensive experiments on two real-world datasets demonstrate that Master achieves the best comprehensive performance compared with nine baseline approaches.

Related papers

• Press Start to Charge: Videogaming the Online Centralized Charging Scheduling Problem [6.285230045232784]
  We study the online centralized charging scheduling problem (OCCSP)<n>In this problem, a central authority must decide, in real time, when to charge dynamically arriving electric vehicles (EVs)<n>We first gamify it; that is, we model it as a game where charging blocks are placed within temporal and capacity constraints on a grid.
  arXiv  Detail & Related papers  (2026-01-18T19:15:29Z)
• Optimizing Electric Vehicle Charging Station Placement Using Reinforcement Learning and Agent-Based Simulations [5.6037668742884135]
  Reinforcement learning offers an innovative approach to identifying optimal charging station locations.<n>We propose a novel framework that integrates deep RL with agent-based simulations to model EV movement and estimate charging demand in real time.<n>Our approach employs a hybrid RL agent with dual Q-networks to select optimal locations and configure charging ports, guided by a hybrid reward function that combines deterministic factors with simulation-derived feedback.
  arXiv  Detail & Related papers  (2025-11-03T04:22:39Z)
• A Unified Multi-Task Learning Framework for Generative Auto-Bidding with Validation-Aligned Optimization [51.27959658504722]
  Multi-task learning offers a principled framework to train these tasks jointly through shared representations.<n>Existing multi-task optimization strategies are primarily guided by training dynamics and often generalize poorly in volatile bidding environments.<n>We present Validation-Aligned Multi-task Optimization (VAMO), which adaptively assigns task weights based on the alignment between per-task training gradients and a held-out validation gradient.
  arXiv  Detail & Related papers  (2025-10-09T03:59:51Z)
• UserRL: Training Interactive User-Centric Agent via Reinforcement Learning [104.63494870852894]
  Reinforcement learning (RL) has shown promise in training agentic models that engage in dynamic, multi-turn interactions.<n>We propose UserRL, a unified framework for training and evaluating user-centric abilities through standardized gym environments.
  arXiv  Detail & Related papers  (2025-09-24T03:33:20Z)
• STARec: An Efficient Agent Framework for Recommender Systems via Autonomous Deliberate Reasoning [54.28691219536054]
  We introduce STARec, a slow-thinking augmented agent framework that endows recommender systems with autonomous deliberative reasoning capabilities.<n>We develop anchored reinforcement training - a two-stage paradigm combining structured knowledge distillation from advanced reasoning models with preference-aligned reward shaping.<n>Experiments on MovieLens 1M and Amazon CDs benchmarks demonstrate that STARec achieves substantial performance gains compared with state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-08-26T08:47:58Z)
• Optimizing Electric Bus Charging Scheduling with Uncertainties Using Hierarchical Deep Reinforcement Learning [46.15490780173541]
  Electric Buses (EBs) represent a significant step toward sustainable development.<n>By utilizing Internet of Things (IoT) systems, charging stations can autonomously determine charging schedules based on real-time data.<n>However, optimizing EB charging schedules remains a critical challenge due to uncertainties in travel time, energy consumption, and fluctuating electricity prices.
  arXiv  Detail & Related papers  (2025-05-15T13:44:27Z)
• Joint Resource Management for Energy-efficient UAV-assisted SWIPT-MEC: A Deep Reinforcement Learning Approach [50.52139512096988]
  6G Internet of Things (IoT) networks face challenges in remote areas and disaster scenarios where ground infrastructure is unavailable.<n>This paper proposes a novel aerial unmanned vehicle (UAV)-assisted computing (MEC) system enhanced by directional antennas to provide both computational and energy support for ground edge terminals.
  arXiv  Detail & Related papers  (2025-05-06T06:46:19Z)
• H-FLTN: A Privacy-Preserving Hierarchical Framework for Electric Vehicle Spatio-Temporal Charge Prediction [8.183121832206556]
  Electric Vehicles (EVs) pose critical challenges for energy providers, particularly in predicting charging time (temporal prediction) This paper introduces the Hierarchical Learning Transformer Network framework to address these challenges. Its integration into real-world smart city infrastructure enhances energy demand forecasting, resource allocation, and grid stability.
  arXiv  Detail & Related papers  (2025-02-25T23:20:53Z)
• Reinforcement Learning-based Approach for Vehicle-to-Building Charging with Heterogeneous Agents and Long Term Rewards [3.867907469895697]
  We introduce a novel RL framework that combines the Deep Deterministic Policy Gradient approach with action masking and efficient MILP-driven policy guidance. Our approach balances the exploration of continuous action spaces to meet user charging demands. Our results show that the proposed approach is one of the first scalable and general approaches to solving the V2B energy management challenge.
  arXiv  Detail & Related papers  (2025-02-24T19:24:41Z)
• Centralized vs. Decentralized Multi-Agent Reinforcement Learning for Enhanced Control of Electric Vehicle Charging Networks [1.9188272016043582]
  We introduce a novel approach for distributed and cooperative charging strategy using a Multi-Agent Reinforcement Learning (MARL) framework. Our method is built upon the Deep Deterministic Policy Gradient (DDPG) algorithm for a group of EVs in a residential community. Our results indicate that, despite higher policy variances and training complexity, the CTDE-DDPG framework significantly improves charging efficiency by reducing total variation by approximately %36 and charging cost by around %9.1 on average.
  arXiv  Detail & Related papers  (2024-04-18T21:50:03Z)
• Rethinking Closed-loop Training for Autonomous Driving [82.61418945804544]
  We present the first empirical study which analyzes the effects of different training benchmark designs on the success of learning agents. We propose trajectory value learning (TRAVL), an RL-based driving agent that performs planning with multistep look-ahead. Our experiments show that TRAVL can learn much faster and produce safer maneuvers compared to all the baselines.
  arXiv  Detail & Related papers  (2023-06-27T17:58:39Z)
• Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Driving [103.745551954983]
  In this paper, we investigate the transfer performance of various types of self-supervised methods, including MoCo and SimCLR, on three downstream tasks. We find that their performances are sub-optimal or even lag far behind the single-task baseline. We propose a simple yet effective pretrain-adapt-finetune paradigm for general multi-task training.
  arXiv  Detail & Related papers  (2022-09-19T12:15:31Z)
• A new Hyper-heuristic based on Adaptive Simulated Annealing and Reinforcement Learning for the Capacitated Electric Vehicle Routing Problem [9.655068751758952]
  Electric vehicles (EVs) have been adopted in urban areas to reduce environmental pollution and global warming. There are still deficiencies in routing the trajectories of last-mile logistics that continue to impact social and economic sustainability. This paper proposes a hyper-heuristic approach called Hyper-heuristic Adaptive Simulated Annealing with Reinforcement Learning.
  arXiv  Detail & Related papers  (2022-06-07T11:10:38Z)
• Computationally efficient joint coordination of multiple electric vehicle charging points using reinforcement learning [6.37470346908743]
  A major challenge in todays power grid is to manage the increasing load from electric vehicle (EV) charging. We propose a single-step solution that jointly coordinates multiple charging points at once. We show that our new RL solutions still improve the performance of charging demand coordination by 40-50% compared to a business-as-usual policy.
  arXiv  Detail & Related papers  (2022-03-26T13:42:57Z)
• Optimized cost function for demand response coordination of multiple EV charging stations using reinforcement learning [6.37470346908743]
  We build on previous research on RL, based on a Markov decision process (MDP) to simultaneously coordinate multiple charging stations. We propose an improved cost function that essentially forces the learned control policy to always fulfill any charging demand that does not offer flexibility. We rigorously compare the newly proposed batch RL fitted Q-iteration implementation with the original (costly) one, using real-world data.
  arXiv  Detail & Related papers  (2022-03-03T11:22:27Z)
• Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement Learning Approach [56.84948632954274]
  This paper investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network. We propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV. Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network.
  arXiv  Detail & Related papers  (2021-12-20T15:45:28Z)
• An Energy Consumption Model for Electrical Vehicle Networks via Extended Federated-learning [50.85048976506701]
  This paper proposes a novel solution to range anxiety based on a federated-learning model. It is capable of estimating battery consumption and providing energy-efficient route planning for vehicle networks.
  arXiv  Detail & Related papers  (2021-11-13T15:03:44Z)
• Learning to Operate an Electric Vehicle Charging Station Considering Vehicle-grid Integration [4.855689194518905]
  We propose a novel centralized allocation and decentralized execution (CADE) reinforcement learning (RL) framework to maximize the charging station's profit. In the centralized allocation process, EVs are allocated to either the waiting or charging spots. In the decentralized execution process, each charger makes its own charging/discharging decision while learning the action-value functions from a shared replay memory. Numerical results show that the proposed CADE framework is both computationally efficient and scalable, and significantly outperforms the baseline model predictive control (MPC)
  arXiv  Detail & Related papers  (2021-11-01T23:10:28Z)
• Self-Supervised Reinforcement Learning for Recommender Systems [77.38665506495553]
  We propose self-supervised reinforcement learning for sequential recommendation tasks. Our approach augments standard recommendation models with two output layers: one for self-supervised learning and the other for RL. Based on such an approach, we propose two frameworks namely Self-Supervised Q-learning(SQN) and Self-Supervised Actor-Critic(SAC)
  arXiv  Detail & Related papers  (2020-06-10T11:18:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.