Sample-efficient Imitative Multi-token Decision Transformer for Generalizable Real World Driving
- URL: http://arxiv.org/abs/2407.02508v1
- Date: Tue, 18 Jun 2024 14:27:14 GMT
- Title: Sample-efficient Imitative Multi-token Decision Transformer for Generalizable Real World Driving
- Authors: Hang Zhou, Dan Xu, Yiding Ji,
- Abstract summary: We propose Sample-efficient Imitative Multi-token Decision Transformer (SimDT)
SimDT introduces multi-token prediction, imitative online learning and prioritized experience replay to Decision Transformer.
Results exceed popular imitation and reinforcement learning algorithms on Waymax benchmark.
- Score: 18.34685506480288
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Reinforcement learning via sequence modeling has shown remarkable promise in autonomous systems, harnessing the power of offline datasets to make informed decisions in simulated environments. However, the full potential of such methods in complex dynamic environments remain to be discovered. In autonomous driving domain, learning-based agents face significant challenges when transferring knowledge from simulated to real-world settings and the performance is also significantly impacted by data distribution shift. To address these issue, we propose Sample-efficient Imitative Multi-token Decision Transformer (SimDT). SimDT introduces multi-token prediction, imitative online learning and prioritized experience replay to Decision Transformer. The performance is evaluated through empirical experiments and results exceed popular imitation and reinforcement learning algorithms on Waymax benchmark.
Related papers
- Transfer Learning Study of Motion Transformer-based Trajectory Predictions [1.2972104025246092]
Trajectory planning in autonomous driving is highly dependent on predicting the emergent behavior of other road users.
Learning-based methods are currently showing impressive results in simulation-based challenges.
The study aims to provide insights into possible trade-offs between computational time and performance to support effective transfers into the real world.
arXiv Detail & Related papers (2024-04-12T06:50:32Z) - Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous
Driving Research [76.93956925360638]
Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes.
It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training.
We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
arXiv Detail & Related papers (2023-10-12T20:49:15Z) - Uncertainty-Aware Decision Transformer for Stochastic Driving
Environments [37.31853034449015]
We introduce an UNcertainty-awaRESion Transformer (UNREST) for planning in driving environments.
UNREST estimates uncertainties by the conditional mutual information between transitions and returns.
We show UNREST's superior performance in various driving scenarios and the power of our uncertainty estimation strategy.
arXiv Detail & Related papers (2023-09-28T12:44:51Z) - Reinforcement Learning with Human Feedback for Realistic Traffic
Simulation [53.85002640149283]
Key element of effective simulation is the incorporation of realistic traffic models that align with human knowledge.
This study identifies two main challenges: capturing the nuances of human preferences on realism and the unification of diverse traffic simulation models.
arXiv Detail & Related papers (2023-09-01T19:29:53Z) - Emergent Agentic Transformer from Chain of Hindsight Experience [96.56164427726203]
We show that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches.
This is the first time that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches.
arXiv Detail & Related papers (2023-05-26T00:43:02Z) - From Model-Based to Data-Driven Simulation: Challenges and Trends in
Autonomous Driving [26.397030011439163]
We provide an overview of challenges with regard to different aspects and types of simulation.
We cover aspects around perception-, behavior- and content-realism as well as general hurdles in the domain of simulation.
Among others, we observe a trend of data-driven, generative approaches and high-fidelity data synthesis to increasingly replace model-based simulation.
arXiv Detail & Related papers (2023-05-23T11:39:23Z) - Environment Transformer and Policy Optimization for Model-Based Offline
Reinforcement Learning [25.684201757101267]
We propose an uncertainty-aware sequence modeling architecture called Environment Transformer.
Benefiting from the accurate modeling of the transition dynamics and reward function, Environment Transformer can be combined with arbitrary planning, dynamics programming, or policy optimization algorithms for offline RL.
arXiv Detail & Related papers (2023-03-07T11:26:09Z) - Generative AI-empowered Simulation for Autonomous Driving in Vehicular
Mixed Reality Metaverses [130.15554653948897]
In vehicular mixed reality (MR) Metaverse, distance between physical and virtual entities can be overcome.
Large-scale traffic and driving simulation via realistic data collection and fusion from the physical world is difficult and costly.
We propose an autonomous driving architecture, where generative AI is leveraged to synthesize unlimited conditioned traffic and driving data in simulations.
arXiv Detail & Related papers (2023-02-16T16:54:10Z) - BITS: Bi-level Imitation for Traffic Simulation [38.28736985320897]
We take a data-driven approach and propose a method that can learn to generate traffic behaviors from real-world driving logs.
We empirically validate our method, named Bi-level Imitation for Traffic Simulation (BITS), with scenarios from two large-scale driving datasets.
As part of our core contributions, we develop and open source a software tool that unifies data formats across different driving datasets.
arXiv Detail & Related papers (2022-08-26T02:17:54Z) - Differentiable Agent-based Epidemiology [71.81552021144589]
We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation.
GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
arXiv Detail & Related papers (2022-07-20T07:32:02Z) - From Simulation to Real World Maneuver Execution using Deep
Reinforcement Learning [69.23334811890919]
Deep Reinforcement Learning has proved to be able to solve many control tasks in different fields, but the behavior of these systems is not always as expected when deployed in real-world scenarios.
This is mainly due to the lack of domain adaptation between simulated and real-world data together with the absence of distinction between train and test datasets.
We present a system based on multiple environments in which agents are trained simultaneously, evaluating the behavior of the model in different scenarios.
arXiv Detail & Related papers (2020-05-13T14:22:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.