Shared learning of powertrain control policies for vehicle fleets

• URL: http://arxiv.org/abs/2404.17892v1
• Date: Sat, 27 Apr 2024 13:01:05 GMT
• Title: Shared learning of powertrain control policies for vehicle fleets
• Authors: Lindsey Kerbel, Beshah Ayalew, Andrej Ivanco,
• Abstract summary: Deep reinforcement learning (DRL) aims at on-the-field learning of powertrain control policies that optimize fuel economy and other performance metrics. We present a novel framework for shared learning among a fleet of vehicles through the use of a distilled group policy. We find a fleet average improvement of 8.5 percent in fuel economy compared to the baseline.
• Score: 3.9325957466009203
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Emerging data-driven approaches, such as deep reinforcement learning (DRL), aim at on-the-field learning of powertrain control policies that optimize fuel economy and other performance metrics. Indeed, they have shown great potential in this regard for individual vehicles on specific routes or drive cycles. However, for fleets of vehicles that must service a distribution of routes, DRL approaches struggle with learning stability issues that result in high variances and challenge their practical deployment. In this paper, we present a novel framework for shared learning among a fleet of vehicles through the use of a distilled group policy as the knowledge sharing mechanism for the policy learning computations at each vehicle. We detail the mathematical formulation that makes this possible. Several scenarios are considered to analyze the functionality, performance, and computational scalability of the framework with fleet size. Comparisons of the cumulative performance of fleets using our proposed shared learning approach with a baseline of individual learning agents and another state-of-the-art approach with a centralized learner show clear advantages to our approach. For example, we find a fleet average asymptotic improvement of 8.5 percent in fuel economy compared to the baseline while also improving on the metrics of acceleration error and shifting frequency for fleets serving a distribution of suburban routes. Furthermore, we include demonstrative results that show how the framework reduces variance within a fleet and also how it helps individual agents adapt better to new routes.

Related papers

• Mobility-Aware Federated Learning: Multi-Armed Bandit Based Selection in Vehicular Network [6.2413189928023085]
  We design a mobility-aware vehicular federated learning (MAVFL) scheme in which vehicles drive through a road segment to perform FL. We conduct the convergence analysis to indicate the influence of vehicle mobility on training loss. We propose a multi-armed bandit-based vehicle selection algorithm to minimize the utility function considering training loss and delay.
  arXiv  Detail & Related papers  (2024-10-14T12:45:15Z)
• Robot Fleet Learning via Policy Merging [58.5086287737653]
  We propose FLEET-MERGE to efficiently merge policies in the fleet setting. We show that FLEET-MERGE consolidates the behavior of policies trained on 50 tasks in the Meta-World environment. We introduce a novel robotic tool-use benchmark, FLEET-TOOLS, for fleet policy learning in compositional and contact-rich robot manipulation tasks.
  arXiv  Detail & Related papers  (2023-10-02T17:23:51Z)
• Rethinking Closed-loop Training for Autonomous Driving [82.61418945804544]
  We present the first empirical study which analyzes the effects of different training benchmark designs on the success of learning agents. We propose trajectory value learning (TRAVL), an RL-based driving agent that performs planning with multistep look-ahead. Our experiments show that TRAVL can learn much faster and produce safer maneuvers compared to all the baselines.
  arXiv  Detail & Related papers  (2023-06-27T17:58:39Z)
• Learning-based Online Optimization for Autonomous Mobility-on-Demand Fleet Control [8.020856741504794]
  We study online control algorithms for autonomous mobility-on-demand systems. We develop a novel hybrid enriched machine learning pipeline which learns online dispatching and rebalancing policies. We show that our approach outperforms state-of-the-art greedy, and model-predictive control approaches.
  arXiv  Detail & Related papers  (2023-02-08T09:40:30Z)
• Residual Policy Learning for Powertrain Control [2.064612766965483]
  This paper outlines an active driver assistance approach that uses a residual policy learning (RPL) agent to provide residual actions to default power train controllers. By implementing on a simulated commercial vehicle in various car-following scenarios, we find that the RPL agent quickly learns significantly improved policies compared to a baseline source policy.
  arXiv  Detail & Related papers  (2022-12-15T04:22:21Z)
• Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curricula [48.58973705935691]
  We show that an agent trained using only a 10% subset of the data performs just as well as an agent trained on the entire dataset. We then demonstrate that this difficulty score can be used in a zero-shot transfer to generate curricula for an imitation-learning based planning agent.
  arXiv  Detail & Related papers  (2022-12-02T18:57:21Z)
• Unified Automatic Control of Vehicular Systems with Reinforcement Learning [64.63619662693068]
  This article contributes a streamlined methodology for vehicular microsimulation. It discovers high performance control strategies with minimal manual design. The study reveals numerous emergent behaviors resembling wave mitigation, traffic signaling, and ramp metering.
  arXiv  Detail & Related papers  (2022-07-30T16:23:45Z)
• Supervised Permutation Invariant Networks for Solving the CVRP with Bounded Fleet Size [3.5235974685889397]
  Learning to solve optimization problems, such as the vehicle routing problem, offers great computational advantages. We propose a powerful supervised deep learning framework that constructs a complete tour plan from scratch while respecting an apriori fixed number of vehicles. In combination with an efficient post-processing scheme, our supervised approach is not only much faster and easier to train but also competitive results.
  arXiv  Detail & Related papers  (2022-01-05T10:32:18Z)
• Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem [13.389057146418056]
  Vehicles in real-world scenarios are likely to be heterogeneous with different characteristics that affect their capacity (or travel speed) We propose a DRL method based on the attention mechanism with a vehicle selection decoder accounting for the heterogeneous fleet constraint and a node selection decoder accounting for the route construction, which learns to construct a solution by automatically selecting both a vehicle and a node for this vehicle at each step.
  arXiv  Detail & Related papers  (2021-10-06T10:05:19Z)
• Real-world Ride-hailing Vehicle Repositioning using Deep Reinforcement Learning [52.2663102239029]
  We present a new practical framework based on deep reinforcement learning and decision-time planning for real-world vehicle on idle-hailing platforms. Our approach learns ride-based state-value function using a batch training algorithm with deep value. We benchmark our algorithm with baselines in a ride-hailing simulation environment to demonstrate its superiority in improving income efficiency.
  arXiv  Detail & Related papers  (2021-03-08T05:34:05Z)
• Vehicular Cooperative Perception Through Action Branching and Federated Reinforcement Learning [101.64598586454571]
  A novel framework is proposed to allow reinforcement learning-based vehicular association, resource block (RB) allocation, and content selection of cooperative perception messages (CPMs) A federated RL approach is introduced in order to speed up the training process across vehicles. Results show that federated RL improves the training process, where better policies can be achieved within the same amount of time compared to the non-federated approach.
  arXiv  Detail & Related papers  (2020-12-07T02:09:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.