Adaptive Energy Management for Real Driving Conditions via Transfer Reinforcement Learning

• URL: http://arxiv.org/abs/2007.12560v1
• Date: Fri, 24 Jul 2020 15:06:23 GMT
• Title: Adaptive Energy Management for Real Driving Conditions via Transfer Reinforcement Learning
• Authors: Teng Liu, Wenhao Tan, Xiaolin Tang, Jiaxin Chen, Dongpu Cao
• Abstract summary: This article proposes a transfer reinforcement learning (RL) based adaptive energy managing approach for a hybrid electric vehicle (HEV) with parallel topology. The up-level characterizes how to transform the Q-value tables in the RL framework via driving cycle transformation (DCT) The lower-level determines how to set the corresponding control strategies with the transformed Q-value tables and TPMs.
• Score: 19.383907178714345
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This article proposes a transfer reinforcement learning (RL) based adaptive energy managing approach for a hybrid electric vehicle (HEV) with parallel topology. This approach is bi-level. The up-level characterizes how to transform the Q-value tables in the RL framework via driving cycle transformation (DCT). Especially, transition probability matrices (TPMs) of power request are computed for different cycles, and induced matrix norm (IMN) is employed as a critical criterion to identify the transformation differences and to determine the alteration of the control strategy. The lower-level determines how to set the corresponding control strategies with the transformed Q-value tables and TPMs by using model-free reinforcement learning (RL) algorithm. Numerical tests illustrate that the transferred performance can be tuned by IMN value and the transfer RL controller could receive a higher fuel economy. The comparison demonstrates that the proposed strategy exceeds the conventional RL approach in both calculation speed and control performance.

Related papers

• Data-driven modeling and supervisory control system optimization for plug-in hybrid electric vehicles [16.348774515562678]
  Learning-based intelligent energy management systems for plug-in hybrid electric vehicles (PHEVs) are crucial for achieving efficient energy utilization. Their application faces system reliability challenges in the real world, which prevents widespread acceptance by original equipment manufacturers (OEMs) This paper proposes a real-vehicle application-oriented control framework, combining horizon-extended reinforcement learning (RL)-based energy management with the equivalent consumption minimization strategy (ECMS) to enhance practical applicability.
  arXiv  Detail & Related papers  (2024-06-13T13:04:42Z)
• Rethinking Transformers in Solving POMDPs [47.14499685668683]
  This paper scrutinizes the effectiveness of a popular architecture, namely Transformers, in Partially Observable Markov Decision Processes (POMDPs) Regular languages, which Transformers struggle to model, are reducible to POMDPs. This poses a significant challenge for Transformers in learning POMDP-specific inductive biases, due to their lack of inherent recurrence found in other models like RNNs.
  arXiv  Detail & Related papers  (2024-05-27T17:02:35Z)
• Q-value Regularized Transformer for Offline Reinforcement Learning [70.13643741130899]
  We propose a Q-value regularized Transformer (QT) to enhance the state-of-the-art in offline reinforcement learning (RL) QT learns an action-value function and integrates a term maximizing action-values into the training loss of Conditional Sequence Modeling (CSM) Empirical evaluations on D4RL benchmark datasets demonstrate the superiority of QT over traditional DP and CSM methods.
  arXiv  Detail & Related papers  (2024-05-27T12:12:39Z)
• Rethinking Decision Transformer via Hierarchical Reinforcement Learning [54.3596066989024]
  Decision Transformer (DT) is an innovative algorithm leveraging recent advances of the transformer architecture in reinforcement learning (RL) We introduce a general sequence modeling framework for studying sequential decision making through the lens of Hierarchical RL. We show DT emerges as a special case of this framework with certain choices of high-level and low-level policies, and discuss the potential failure of these choices.
  arXiv  Detail & Related papers  (2023-11-01T03:32:13Z)
• On Transforming Reinforcement Learning by Transformer: The Development Trajectory [97.79247023389445]
  Transformer, originally devised for natural language processing, has also attested significant success in computer vision. We group existing developments in two categories: architecture enhancement and trajectory optimization. We examine the main applications of TRL in robotic manipulation, text-based games, navigation and autonomous driving.
  arXiv  Detail & Related papers  (2022-12-29T03:15:59Z)
• Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer [128.19212716007794]
  We propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency.
  arXiv  Detail & Related papers  (2022-06-08T07:48:42Z)
• Decision Transformer: Reinforcement Learning via Sequence Modeling [102.86873656751489]
  We present a framework that abstracts Reinforcement Learning (RL) as a sequence modeling problem. We present Decision Transformer, an architecture that casts the problem of RL as conditional sequence modeling. Despite its simplicity, Decision Transformer matches or exceeds the performance of state-of-the-art offline RL baselines on Atari, OpenAI Gym, and Key-to-Door tasks.
  arXiv  Detail & Related papers  (2021-06-02T17:53:39Z)
• Data-Driven Transferred Energy Management Strategy for Hybrid Electric Vehicles via Deep Reinforcement Learning [3.313774035672581]
  This paper proposes a real-time EMS via incorporating the DRL method and transfer learning (TL) The related EMSs are derived from and evaluated on the real-world collected driving cycle dataset from Transportation Secure Data Center. Simulation results indicate that the presented transfer DRL-based EMS could effectively reduce time consumption and guarantee control performance.
  arXiv  Detail & Related papers  (2020-09-07T17:53:07Z)
• An Intelligent Control Strategy for buck DC-DC Converter via Deep Reinforcement Learning [1.4502611532302039]
  An innovative intelligent control strategy for buck DC-DC converter with constant power loads (CPLs) is constructed for the first time. A Markov Decision Process (MDP) model and the deep Q network (DQN) algorithm are defined for DC-DC converter. A model-free based deep reinforcement learning (DRL) control strategy is appropriately designed to adjust the agent-environment interaction.
  arXiv  Detail & Related papers  (2020-08-11T06:38:53Z)
• Transfer Deep Reinforcement Learning-enabled Energy Management Strategy for Hybrid Tracked Vehicle [8.327437591702163]
  This paper proposes an adaptive energy management strategy for hybrid electric vehicles by combining deep reinforcement learning (DRL) and transfer learning (TL) It aims to address the defect of DRL in tedious training time. The founded DRL and TL-enabled control policy is capable of enhancing energy efficiency and improving system performance.
  arXiv  Detail & Related papers  (2020-07-16T23:39:34Z)
• Transferred Energy Management Strategies for Hybrid Electric Vehicles Based on Driving Conditions Recognition [16.346064265993782]
  Energy management strategies (EMSs) decide the potential of energy conservation and emission reduction. This work presents a transferred EMS for a parallel HEV via combining the reinforcement learning method and driving conditions recognition.
  arXiv  Detail & Related papers  (2020-07-16T13:57:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.