Energy Management of Multi-mode Plug-in Hybrid Electric Vehicle using Multi-agent Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2303.09658v2
• Date: Mon, 28 Aug 2023 00:36:11 GMT
• Title: Energy Management of Multi-mode Plug-in Hybrid Electric Vehicle using Multi-agent Deep Reinforcement Learning
• Authors: Min Hua, Cetengfei Zhang, Fanggang Zhang, Zhi Li, Xiaoli Yu, Hongming Xu, Quan Zhou
• Abstract summary: Multi-mode plug-in hybrid electric vehicle (PHEV) technology is one of the pathways making contributions to decarbonization. This paper studies a multi-agent deep reinforcement learning (MADRL) control method for energy management of the multi-mode PHEV. Using the unified DDPG settings and a relevance ratio of 0.2, the proposed MADRL system can save up to 4% energy compared to the single-agent learning system and up to 23.54% energy compared to the conventional rule-based system.
• Score: 6.519522573636577
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recently emerging multi-mode plug-in hybrid electric vehicle (PHEV) technology is one of the pathways making contributions to decarbonization, and its energy management requires multiple-input and multipleoutput (MIMO) control. At the present, the existing methods usually decouple the MIMO control into singleoutput (MISO) control and can only achieve its local optimal performance. To optimize the multi-mode vehicle globally, this paper studies a MIMO control method for energy management of the multi-mode PHEV based on multi-agent deep reinforcement learning (MADRL). By introducing a relevance ratio, a hand-shaking strategy is proposed to enable two learning agents to work collaboratively under the MADRL framework using the deep deterministic policy gradient (DDPG) algorithm. Unified settings for the DDPG agents are obtained through a sensitivity analysis of the influencing factors to the learning performance. The optimal working mode for the hand-shaking strategy is attained through a parametric study on the relevance ratio. The advantage of the proposed energy management method is demonstrated on a software-in-the-loop testing platform. The result of the study indicates that the learning rate of the DDPG agents is the greatest influencing factor for learning performance. Using the unified DDPG settings and a relevance ratio of 0.2, the proposed MADRL system can save up to 4% energy compared to the single-agent learning system and up to 23.54% energy compared to the conventional rule-based system.

Related papers

• Data-driven modeling and supervisory control system optimization for plug-in hybrid electric vehicles [16.348774515562678]
  Learning-based intelligent energy management systems for plug-in hybrid electric vehicles (PHEVs) are crucial for achieving efficient energy utilization. Their application faces system reliability challenges in the real world, which prevents widespread acceptance by original equipment manufacturers (OEMs) This paper proposes a real-vehicle application-oriented control framework, combining horizon-extended reinforcement learning (RL)-based energy management with the equivalent consumption minimization strategy (ECMS) to enhance practical applicability.
  arXiv  Detail & Related papers  (2024-06-13T13:04:42Z)
• Multi-agent Reinforcement Learning for Energy Saving in Multi-Cell Massive MIMO Systems [6.614630708703594]
  We develop a multi-agent reinforcement learning (MARL) algorithm to minimize the total energy consumption of base stations (BSs) in a multi-cell network. We show that the trained MAPPO agent achieves better performance compared to baseline policies. Specifically, compared to the auto sleep mode 1 algorithm, the MAPPO-neighbor policy reduces power consumption by approximately 8.7% during low-traffic hours and improves energy efficiency by approximately 19% during high-traffic hours, respectively.
  arXiv  Detail & Related papers  (2024-02-05T17:15:00Z)
• Multi-agent Deep Reinforcement Learning for Charge-sustaining Control of Multi-mode Hybrid Vehicles [9.416703139663705]
  Transportation electrification requires an increasing number of electric components on vehicles. This paper focuses on the online optimization of energy management strategy for a multi-mode hybrid electric vehicle. A new collaborative cyber-physical learning with multi-agents is proposed.
  arXiv  Detail & Related papers  (2022-09-06T16:40:55Z)
• Unified Automatic Control of Vehicular Systems with Reinforcement Learning [64.63619662693068]
  This article contributes a streamlined methodology for vehicular microsimulation. It discovers high performance control strategies with minimal manual design. The study reveals numerous emergent behaviors resembling wave mitigation, traffic signaling, and ramp metering.
  arXiv  Detail & Related papers  (2022-07-30T16:23:45Z)
• Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement Learning Approach [56.84948632954274]
  This paper investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network. We propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV. Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network.
  arXiv  Detail & Related papers  (2021-12-20T15:45:28Z)
• Deep Reinforcement Learning Based Multidimensional Resource Management for Energy Harvesting Cognitive NOMA Communications [64.1076645382049]
  Combination of energy harvesting (EH), cognitive radio (CR), and non-orthogonal multiple access (NOMA) is a promising solution to improve energy efficiency. In this paper, we study the spectrum, energy, and time resource management for deterministic-CR-NOMA IoT systems.
  arXiv  Detail & Related papers  (2021-09-17T08:55:48Z)
• Imitation Learning from MPC for Quadrupedal Multi-Gait Control [63.617157490920505]
  We present a learning algorithm for training a single policy that imitates multiple gaits of a walking robot. We use and extend MPC-Net, which is an Imitation Learning approach guided by Model Predictive Control. We validate our approach on hardware and show that a single learned policy can replace its teacher to control multiple gaits.
  arXiv  Detail & Related papers  (2021-03-26T08:48:53Z)
• Transfer Deep Reinforcement Learning-enabled Energy Management Strategy for Hybrid Tracked Vehicle [8.327437591702163]
  This paper proposes an adaptive energy management strategy for hybrid electric vehicles by combining deep reinforcement learning (DRL) and transfer learning (TL) It aims to address the defect of DRL in tedious training time. The founded DRL and TL-enabled control policy is capable of enhancing energy efficiency and improving system performance.
  arXiv  Detail & Related papers  (2020-07-16T23:39:34Z)
• Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences [5.625230013691758]
  Development of hybrid electric vehicles depends on an advanced and efficient energy management strategy (EMS) This article presents a human-like energy management framework for hybrid electric vehicles according to deep reinforcement learning methods and collected historical driving data. Improvements in fuel economy and convergence rate indicate the effectiveness of the constructed control structure.
  arXiv  Detail & Related papers  (2020-07-16T14:15:35Z)
• Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems [87.4519172058185]
  An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
  arXiv  Detail & Related papers  (2020-02-20T04:58:07Z)
• Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework [108.4560749465701]
  The Pontryagin Differentiable Programming methodology establishes a unified framework to solve a broad class of learning and control tasks. We investigate three learning modes of the PDP: inverse reinforcement learning, system identification, and control/planning. We demonstrate the capability of the PDP in each learning mode on different high-dimensional systems, including multi-link robot arm, 6-DoF maneuvering quadrotor, and 6-DoF rocket powered landing.
  arXiv  Detail & Related papers  (2019-12-30T15:35:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.