Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences

• URL: http://arxiv.org/abs/2007.10126v2
• Date: Tue, 26 Sep 2023 01:22:26 GMT
• Title: Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences
• Authors: Hao Chen, Xiaolin Tang, Guo Hu, Teng Liu
• Abstract summary: Development of hybrid electric vehicles depends on an advanced and efficient energy management strategy (EMS) This article presents a human-like energy management framework for hybrid electric vehicles according to deep reinforcement learning methods and collected historical driving data. Improvements in fuel economy and convergence rate indicate the effectiveness of the constructed control structure.
• Score: 5.625230013691758
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Development of hybrid electric vehicles depends on an advanced and efficient energy management strategy (EMS). With online and real-time requirements in mind, this article presents a human-like energy management framework for hybrid electric vehicles according to deep reinforcement learning methods and collected historical driving data. The hybrid powertrain studied has a series-parallel topology, and its control-oriented modeling is founded first. Then, the distinctive deep reinforcement learning (DRL) algorithm, named deep deterministic policy gradient (DDPG), is introduced. To enhance the derived power split controls in the DRL framework, the global optimal control trajectories obtained from dynamic programming (DP) are regarded as expert knowledge to train the DDPG model. This operation guarantees the optimality of the proposed control architecture. Moreover, the collected historical driving data based on experienced drivers are employed to replace the DP-based controls, and thus construct the human-like EMSs. Finally, different categories of experiments are executed to estimate the optimality and adaptability of the proposed human-like EMS. Improvements in fuel economy and convergence rate indicate the effectiveness of the constructed control structure.

Related papers

• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [60.05963742334746]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers. Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy. We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• Data-driven modeling and supervisory control system optimization for plug-in hybrid electric vehicles [16.348774515562678]
  Learning-based intelligent energy management systems for plug-in hybrid electric vehicles (PHEVs) are crucial for achieving efficient energy utilization. Their application faces system reliability challenges in the real world, which prevents widespread acceptance by original equipment manufacturers (OEMs) This paper proposes a real-vehicle application-oriented control framework, combining horizon-extended reinforcement learning (RL)-based energy management with the equivalent consumption minimization strategy (ECMS) to enhance practical applicability.
  arXiv  Detail & Related papers  (2024-06-13T13:04:42Z)
• Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks [55.591662978280894]
  This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs) Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees.
  arXiv  Detail & Related papers  (2023-10-13T21:26:16Z)
• Energy Management of Multi-mode Plug-in Hybrid Electric Vehicle using Multi-agent Deep Reinforcement Learning [6.519522573636577]
  Multi-mode plug-in hybrid electric vehicle (PHEV) technology is one of the pathways making contributions to decarbonization. This paper studies a multi-agent deep reinforcement learning (MADRL) control method for energy management of the multi-mode PHEV. Using the unified DDPG settings and a relevance ratio of 0.2, the proposed MADRL system can save up to 4% energy compared to the single-agent learning system and up to 23.54% energy compared to the conventional rule-based system.
  arXiv  Detail & Related papers  (2023-03-16T21:31:55Z)
• On Transforming Reinforcement Learning by Transformer: The Development Trajectory [97.79247023389445]
  Transformer, originally devised for natural language processing, has also attested significant success in computer vision. We group existing developments in two categories: architecture enhancement and trajectory optimization. We examine the main applications of TRL in robotic manipulation, text-based games, navigation and autonomous driving.
  arXiv  Detail & Related papers  (2022-12-29T03:15:59Z)
• Unified Automatic Control of Vehicular Systems with Reinforcement Learning [64.63619662693068]
  This article contributes a streamlined methodology for vehicular microsimulation. It discovers high performance control strategies with minimal manual design. The study reveals numerous emergent behaviors resembling wave mitigation, traffic signaling, and ramp metering.
  arXiv  Detail & Related papers  (2022-07-30T16:23:45Z)
• Model-based versus Model-free Deep Reinforcement Learning for Autonomous Racing Cars [46.64253693115981]
  This paper investigates how model-based deep reinforcement learning agents generalize to real-world autonomous-vehicle control-tasks. We show that model-based agents capable of learning in imagination, substantially outperform model-free agents with respect to performance, sample efficiency, successful task completion, and generalization.
  arXiv  Detail & Related papers  (2021-03-08T17:15:23Z)
• Data-Driven Transferred Energy Management Strategy for Hybrid Electric Vehicles via Deep Reinforcement Learning [3.313774035672581]
  This paper proposes a real-time EMS via incorporating the DRL method and transfer learning (TL) The related EMSs are derived from and evaluated on the real-world collected driving cycle dataset from Transportation Secure Data Center. Simulation results indicate that the presented transfer DRL-based EMS could effectively reduce time consumption and guarantee control performance.
  arXiv  Detail & Related papers  (2020-09-07T17:53:07Z)
• Adaptive Energy Management for Real Driving Conditions via Transfer Reinforcement Learning [19.383907178714345]
  This article proposes a transfer reinforcement learning (RL) based adaptive energy managing approach for a hybrid electric vehicle (HEV) with parallel topology. The up-level characterizes how to transform the Q-value tables in the RL framework via driving cycle transformation (DCT) The lower-level determines how to set the corresponding control strategies with the transformed Q-value tables and TPMs.
  arXiv  Detail & Related papers  (2020-07-24T15:06:23Z)
• Transfer Deep Reinforcement Learning-enabled Energy Management Strategy for Hybrid Tracked Vehicle [8.327437591702163]
  This paper proposes an adaptive energy management strategy for hybrid electric vehicles by combining deep reinforcement learning (DRL) and transfer learning (TL) It aims to address the defect of DRL in tedious training time. The founded DRL and TL-enabled control policy is capable of enhancing energy efficiency and improving system performance.
  arXiv  Detail & Related papers  (2020-07-16T23:39:34Z)
• Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications [54.610318402371185]
  Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. We minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences.
  arXiv  Detail & Related papers  (2020-05-25T01:42:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.