Modeling Human Driving Behavior through Generative Adversarial Imitation Learning

• URL: http://arxiv.org/abs/2006.06412v1
• Date: Wed, 10 Jun 2020 05:47:39 GMT
• Title: Modeling Human Driving Behavior through Generative Adversarial Imitation Learning
• Authors: Raunak Bhattacharyya, Blake Wulfe, Derek Phillips, Alex Kuefler, Jeremy Morton, Ransalu Senanayake, Mykel Kochenderfer
• Abstract summary: This article describes the use of Generative Adversarial Imitation Learning (GAIL) for learning-based driver modeling. Because driver modeling is inherently a multi-agent problem, this paper describes a parameter-sharing extension of GAIL called PS-GAIL to tackle multi-agent driver modeling. This paper describes Reward Augmented Imitation Learning (RAIL), which modifies the reward signal to provide domain-specific knowledge to the agent.
• Score: 7.387855463533219
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Imitation learning is an approach for generating intelligent behavior when the cost function is unknown or difficult to specify. Building upon work in inverse reinforcement learning (IRL), Generative Adversarial Imitation Learning (GAIL) aims to provide effective imitation even for problems with large or continuous state and action spaces. Driver modeling is one example of a problem where the state and action spaces are continuous. Human driving behavior is characterized by non-linearity and stochasticity, and the underlying cost function is unknown. As a result, learning from human driving demonstrations is a promising approach for generating human-like driving behavior. This article describes the use of GAIL for learning-based driver modeling. Because driver modeling is inherently a multi-agent problem, where the interaction between agents needs to be modeled, this paper describes a parameter-sharing extension of GAIL called PS-GAIL to tackle multi-agent driver modeling. In addition, GAIL is domain agnostic, making it difficult to encode specific knowledge relevant to driving in the learning process. This paper describes Reward Augmented Imitation Learning (RAIL), which modifies the reward signal to provide domain-specific knowledge to the agent. Finally, human demonstrations are dependent upon latent factors that may not be captured by GAIL. This paper describes Burn-InfoGAIL, which allows for disentanglement of latent variability in demonstrations. Imitation learning experiments are performed using NGSIM, a real-world highway driving dataset. Experiments show that these modifications to GAIL can successfully model highway driving behavior, accurately replicating human demonstrations and generating realistic, emergent behavior in the traffic flow arising from the interaction between driving agents.

Related papers

• Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models [60.87795376541144]
  A world model is a neural network capable of predicting an agent's next state given past states and actions. During end-to-end training, our policy learns how to recover from errors by aligning with states observed in human demonstrations. We present qualitative and quantitative results, demonstrating significant improvements upon prior state of the art in closed-loop testing.
  arXiv  Detail & Related papers  (2024-09-25T06:48:25Z)
• GenFollower: Enhancing Car-Following Prediction with Large Language Models [11.847589952558566]
  We propose GenFollower, a novel zero-shot prompting approach that leverages large language models (LLMs) to address these challenges. We reframe car-following behavior as a language modeling problem and integrate heterogeneous inputs into structured prompts for LLMs. Experiments on Open datasets demonstrate GenFollower's superior performance and ability to provide interpretable insights.
  arXiv  Detail & Related papers  (2024-07-08T04:54:42Z)
• BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplay [48.75878234995544]
  Imitation learning learns a policy from demonstrations without requiring hand-designed reward functions. We propose BeTAIL: Behavior Transformer Adversarial Imitation Learning. We test BeTAIL on three challenges with expert-level demonstrations of real human gameplay in Gran Turismo Sport.
  arXiv  Detail & Related papers  (2024-02-22T00:38:43Z)
• Causal Imitative Model for Autonomous Driving [85.78593682732836]
  We propose Causal Imitative Model (CIM) to address inertia and collision problems. CIM explicitly discovers the causal model and utilizes it to train the policy. Our experiments show that our method outperforms previous work in terms of inertia and collision rates.
  arXiv  Detail & Related papers  (2021-12-07T18:59:15Z)
• Learning Interactive Driving Policies via Data-driven Simulation [125.97811179463542]
  Data-driven simulators promise high data-efficiency for driving policy learning. Small underlying datasets often lack interesting and challenging edge cases for learning interactive driving. We propose a simulation method that uses in-painted ado vehicles for learning robust driving policies.
  arXiv  Detail & Related papers  (2021-11-23T20:14:02Z)
• Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments [0.8122270502556374]
  Generative Adversarial Imitation Learning (GAIL) can train policies without explicitly requiring to define a reward function. We show that both of them are capable of imitating the expert trajectory from start to end after training ends.
  arXiv  Detail & Related papers  (2021-10-16T15:04:13Z)
• Inverse Reinforcement Learning Based Stochastic Driver Behavior Learning [3.4979173592795374]
  Drivers have unique and rich driving behaviors when operating vehicles in traffic. This paper presents a novel driver behavior learning approach that captures the uniqueness and richness of human driver behavior in realistic driving scenarios.
  arXiv  Detail & Related papers  (2021-07-01T20:18:03Z)
• Building Safer Autonomous Agents by Leveraging Risky Driving Behavior Knowledge [1.52292571922932]
  This study focuses on creating risk prone scenarios with heavy traffic and unexpected random behavior for creating better model-free learning agents. We generate multiple autonomous driving scenarios by creating new custom Markov Decision Process (MDP) environment iterations in highway-env simulation package. We train model free learning agents with supplement information of risk prone driving scenarios and compare their performance with baseline agents.
  arXiv  Detail & Related papers  (2021-03-16T23:39:33Z)
• A Driving Behavior Recognition Model with Bi-LSTM and Multi-Scale CNN [59.57221522897815]
  We propose a neural network model based on trajectories information for driving behavior recognition. We evaluate the proposed model on the public BLVD dataset, achieving a satisfying performance.
  arXiv  Detail & Related papers  (2021-03-01T06:47:29Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)
• A Probabilistic Framework for Imitating Human Race Driver Behavior [31.524303667746643]
  We propose Probabilistic Modeling of Driver behavior (ProMoD), a modular framework which splits the task of driver behavior modeling into multiple modules. A global target trajectory distribution is learned with Probabilistic Movement Primitives, clothoids are utilized for local path generation, and the corresponding choice of actions is performed by a neural network. Experiments in a simulated car racing setting show considerable advantages in imitation accuracy and robustness compared to other imitation learning algorithms.
  arXiv  Detail & Related papers  (2020-01-22T20:06:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.