On Transferability of Driver Observation Models from Simulated to Real Environments in Autonomous Cars

• URL: http://arxiv.org/abs/2307.16543v1
• Date: Mon, 31 Jul 2023 10:18:49 GMT
• Title: On Transferability of Driver Observation Models from Simulated to Real Environments in Autonomous Cars
• Authors: Walter Morales-Alvarez, Novel Certad, Alina Roitberg, Rainer Stiefelhagen and Cristina Olaverri-Monreal
• Abstract summary: This paper investigates the viability of transferring video-based driver observation models from simulation to real-world scenarios in autonomous vehicles. We record a dataset featuring actual autonomous driving conditions and involving seven participants engaged in highly distracting secondary activities. Our dataset was designed in accordance with an existing large-scale simulator dataset used as the training source.
• Score: 23.514129229090987
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For driver observation frameworks, clean datasets collected in controlled simulated environments often serve as the initial training ground. Yet, when deployed under real driving conditions, such simulator-trained models quickly face the problem of distributional shifts brought about by changing illumination, car model, variations in subject appearances, sensor discrepancies, and other environmental alterations. This paper investigates the viability of transferring video-based driver observation models from simulation to real-world scenarios in autonomous vehicles, given the frequent use of simulation data in this domain due to safety issues. To achieve this, we record a dataset featuring actual autonomous driving conditions and involving seven participants engaged in highly distracting secondary activities. To enable direct SIM to REAL transfer, our dataset was designed in accordance with an existing large-scale simulator dataset used as the training source. We utilize the Inflated 3D ConvNet (I3D) model, a popular choice for driver observation, with Gradient-weighted Class Activation Mapping (Grad-CAM) for detailed analysis of model decision-making. Though the simulator-based model clearly surpasses the random baseline, its recognition quality diminishes, with average accuracy dropping from 85.7% to 46.6%. We also observe strong variations across different behavior classes. This underscores the challenges of model transferability, facilitating our research of more robust driver observation systems capable of dealing with real driving conditions.

Related papers

• An Approach to Systematic Data Acquisition and Data-Driven Simulation for the Safety Testing of Automated Driving Functions [32.37902846268263]
  In R&D areas related to the safety impact of the "open world", there is a significant shortage of real-world data to parameterize and/or validate simulations. We present an approach to systematically acquire data in public traffic by heterogeneous means, transform it into a unified representation, and use it to automatically parameterize traffic behavior models for use in data-driven virtual validation of automated driving functions.
  arXiv  Detail & Related papers  (2024-05-02T23:24:27Z)
• Bridging the Sim-to-Real Gap with Bayesian Inference [53.61496586090384]
  We present SIM-FSVGD for learning robot dynamics from data. We use low-fidelity physical priors to regularize the training of neural network models. We demonstrate the effectiveness of SIM-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system.
  arXiv  Detail & Related papers  (2024-03-25T11:29:32Z)
• Trajeglish: Traffic Modeling as Next-Token Prediction [67.28197954427638]
  A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. We apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Our model tops the Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%.
  arXiv  Detail & Related papers  (2023-12-07T18:53:27Z)
• TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction [149.5716746789134]
  We show data-driven traffic simulation can be formulated as a world model. We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving. Experiments on the open motion dataset show TrafficBots can simulate realistic multi-agent behaviors.
  arXiv  Detail & Related papers  (2023-03-07T18:28:41Z)
• Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation [98.51313127382937]
  We focus on the use of labels in the synthetic domain alone. Our approach introduces both a way to learn neural-invariant representations and a theoretically inspired view on how to sample the data from the simulator. We showcase our approach on the bird's-eye-view vehicle segmentation task with multi-sensor data.
  arXiv  Detail & Related papers  (2021-11-15T18:37:43Z)
• A Hybrid Rule-Based and Data-Driven Approach to Driver Modeling through Particle Filtering [6.9485501711137525]
  We propose a methodology that combines rule-based modeling with data-driven learning. Our results show that driver models based on our hybrid rule-based and data-driven approach can accurately capture real-world driving behavior.
  arXiv  Detail & Related papers  (2021-08-29T11:07:14Z)
• Predicting Take-over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluation [11.007092387379076]
  We develop and train take-over time (TOT) models that operate on mid and high-level features produced by computer vision algorithms operating on different driver-facing camera views. We show that a TOT model supported by augmented data can be used to produce continuous estimates of take-over times without delay.
  arXiv  Detail & Related papers  (2021-07-27T16:39:50Z)
• SimNet: Learning Reactive Self-driving Simulations from Real-world Observations [10.035169936164504]
  We present an end-to-end trainable machine learning system capable of realistically simulating driving experiences. This can be used for the verification of self-driving system performance without relying on expensive and time-consuming road testing.
  arXiv  Detail & Related papers  (2021-05-26T05:14:23Z)
• Objective-aware Traffic Simulation via Inverse Reinforcement Learning [31.26257563160961]
  We formulate traffic simulation as an inverse reinforcement learning problem. We propose a parameter sharing adversarial inverse reinforcement learning model for dynamics-robust simulation learning. Our proposed model is able to imitate a vehicle's trajectories in the real world while simultaneously recovering the reward function.
  arXiv  Detail & Related papers  (2021-05-20T07:26:34Z)
• DriveGAN: Towards a Controllable High-Quality Neural Simulation [147.6822288981004]
  We introduce a novel high-quality neural simulator referred to as DriveGAN. DriveGAN achieves controllability by disentangling different components without supervision. We train DriveGAN on multiple datasets, including 160 hours of real-world driving data.
  arXiv  Detail & Related papers  (2021-04-30T15:30:05Z)
• SimAug: Learning Robust Representations from Simulation for Trajectory Prediction [78.91518036949918]
  We propose a novel approach to learn robust representation through augmenting the simulation training data. We show that SimAug achieves promising results on three real-world benchmarks using zero real training data.
  arXiv  Detail & Related papers  (2020-04-04T21:22:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.