Residual Policy Learning for Powertrain Control

• URL: http://arxiv.org/abs/2212.07611v1
• Date: Thu, 15 Dec 2022 04:22:21 GMT
• Title: Residual Policy Learning for Powertrain Control
• Authors: Lindsey Kerbel, Beshah Ayalew, Andrej Ivanco, Keith Loiselle
• Abstract summary: This paper outlines an active driver assistance approach that uses a residual policy learning (RPL) agent to provide residual actions to default power train controllers. By implementing on a simulated commercial vehicle in various car-following scenarios, we find that the RPL agent quickly learns significantly improved policies compared to a baseline source policy.
• Score: 2.064612766965483
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Eco-driving strategies have been shown to provide significant reductions in fuel consumption. This paper outlines an active driver assistance approach that uses a residual policy learning (RPL) agent trained to provide residual actions to default power train controllers while balancing fuel consumption against other driver-accommodation objectives. Using previous experiences, our RPL agent learns improved traction torque and gear shifting residual policies to adapt the operation of the powertrain to variations and uncertainties in the environment. For comparison, we consider a traditional reinforcement learning (RL) agent trained from scratch. Both agents employ the off-policy Maximum A Posteriori Policy Optimization algorithm with an actor-critic architecture. By implementing on a simulated commercial vehicle in various car-following scenarios, we find that the RPL agent quickly learns significantly improved policies compared to a baseline source policy but in some measures not as good as those eventually possible with the RL agent trained from scratch.

Related papers

• Shared learning of powertrain control policies for vehicle fleets [3.9325957466009203]
  Deep reinforcement learning (DRL) aims at on-the-field learning of powertrain control policies that optimize fuel economy and other performance metrics. We present a novel framework for shared learning among a fleet of vehicles through the use of a distilled group policy. We find a fleet average improvement of 8.5 percent in fuel economy compared to the baseline.
  arXiv  Detail & Related papers  (2024-04-27T13:01:05Z)
• Robust Driving Policy Learning with Guided Meta Reinforcement Learning [49.860391298275616]
  We introduce an efficient method to train diverse driving policies for social vehicles as a single meta-policy. By randomizing the interaction-based reward functions of social vehicles, we can generate diverse objectives and efficiently train the meta-policy. We propose a training strategy to enhance the robustness of the ego vehicle's driving policy using the environment where social vehicles are controlled by the learned meta-policy.
  arXiv  Detail & Related papers  (2023-07-19T17:42:36Z)
• Safe Reinforcement Learning for an Energy-Efficient Driver Assistance System [1.8899300124593645]
  Reinforcement learning (RL)-based driver assistance systems seek to improve fuel consumption via continual improvement of powertrain control actions. In this paper, an exponential control barrier function (ECBF) is derived and utilized to filter unsafe actions proposed by an RL-based driver assistance system. The proposed safe-RL scheme is trained and evaluated in car following scenarios where it is shown that it effectively avoids collision both during training and evaluation.
  arXiv  Detail & Related papers  (2023-01-03T00:25:00Z)
• Driver Assistance Eco-driving and Transmission Control with Deep Reinforcement Learning [2.064612766965483]
  In this paper, a model-free deep reinforcement learning (RL) control agent is proposed for active Eco-driving assistance. It trades-off fuel consumption against other driver-accommodation objectives, and learns optimal traction torque and transmission shifting policies from experience. It shows superior performance in minimizing fuel consumption compared to a baseline controller that has full knowledge of fuel-efficiency tables.
  arXiv  Detail & Related papers  (2022-12-15T02:52:07Z)
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning [64.11013095004786]
  We propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks.
  arXiv  Detail & Related papers  (2022-12-14T03:11:25Z)
• Learning from Simulation, Racing in Reality [126.56346065780895]
  We present a reinforcement learning-based solution to autonomously race on a miniature race car platform. We show that a policy that is trained purely in simulation can be successfully transferred to the real robotic setup.
  arXiv  Detail & Related papers  (2020-11-26T14:58:49Z)
• Decision-making for Autonomous Vehicles on Highway: Deep Reinforcement Learning with Continuous Action Horizon [14.059728921828938]
  This paper utilizes the deep reinforcement learning (DRL) method to address the continuous-horizon decision-making problem on the highway. The running objective of the ego automated vehicle is to execute an efficient and smooth policy without collision. The PPO-DRL-based decision-making strategy is estimated from multiple perspectives, including the optimality, learning efficiency, and adaptability.
  arXiv  Detail & Related papers  (2020-08-26T22:49:27Z)
• Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings [129.80279257258098]
  Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous. We propose a "safety-critical adaptation" task setting: an agent first trains in non-safety-critical "source" environments. We propose a solution approach, CARL, that builds on the intuition that prior experience in diverse environments equips an agent to estimate risk.
  arXiv  Detail & Related papers  (2020-08-15T01:40:59Z)
• Transfer Deep Reinforcement Learning-enabled Energy Management Strategy for Hybrid Tracked Vehicle [8.327437591702163]
  This paper proposes an adaptive energy management strategy for hybrid electric vehicles by combining deep reinforcement learning (DRL) and transfer learning (TL) It aims to address the defect of DRL in tedious training time. The founded DRL and TL-enabled control policy is capable of enhancing energy efficiency and improving system performance.
  arXiv  Detail & Related papers  (2020-07-16T23:39:34Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)
• Efficient Deep Reinforcement Learning via Adaptive Policy Transfer [50.51637231309424]
  Policy Transfer Framework (PTF) is proposed to accelerate Reinforcement Learning (RL) Our framework learns when and which source policy is the best to reuse for the target policy and when to terminate it. Experimental results show it significantly accelerates the learning process and surpasses state-of-the-art policy transfer methods.
  arXiv  Detail & Related papers  (2020-02-19T07:30:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.