Evaluating Robustness over High Level Driving Instruction for Autonomous Driving

• URL: http://arxiv.org/abs/2105.10014v1
• Date: Thu, 20 May 2021 20:10:14 GMT
• Title: Evaluating Robustness over High Level Driving Instruction for Autonomous Driving
• Abstract summary: In recent years, we have witnessed increasingly high performance in the field of autonomous end-to-end driving. We propose here an evaluation method, namely a benchmark that allows to assess the robustness of an agent.
• Score: 5.180648702293017
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, we have witnessed increasingly high performance in the field of autonomous end-to-end driving. In particular, more and more research is being done on driving in urban environments, where the car has to follow high level commands to navigate. However, few evaluations are made on the ability of these agents to react in an unexpected situation. Specifically, no evaluations are conducted on the robustness of driving agents in the event of a bad high-level command. We propose here an evaluation method, namely a benchmark that allows to assess the robustness of an agent, and to appreciate its understanding of the environment through its ability to keep a safe behavior, regardless of the instruction.

Related papers

• DriveJudge: Rethinking Autonomous Driving Evaluation with Vision-Language Models [53.73898294218455]
  DriveJudge is a driving evaluation agent that combines rule-grounded evaluation with Vision-Language Model (VLM) reasoning.<n>We introduce two benchmark tasks: Driving Quality Classification and Trajectory Preference Selection.<n>DriveJudge outperformsS for driving quality classification by 21.23 AUC, and the recent VLM-based DriveCritic for trajectory preference selection by 6.5%.
  arXiv  Detail & Related papers  (2026-06-15T23:39:36Z)
• From Human Intention to Action Prediction: A Comprehensive Benchmark for Intention-driven End-to-End Autonomous Driving [67.23302649816466]
  Current autonomous driving systems operate at a level of intelligence akin to following simple steering commands.<n>We introduce Intention-Drive, the first comprehensive benchmark designed to evaluate the ability to translate high-level human intent into safe and precise driving actions.
  arXiv  Detail & Related papers  (2025-12-13T11:59:51Z)
• Behavioral Safety Assessment towards Large-scale Deployment of Autonomous Vehicles [6.846750893175613]
  We propose a paradigm shift toward behavioral safety for autonomous vehicles (AVs)<n>We introduce a third-party AV safety assessment framework comprising two complementary evaluation components: Driver Licensing Test and Driving Intelligence Test.<n>We validated our proposed framework using textttAutoware.Universe, an open-source Level 4 AV, tested both in simulated environments and on the physical test track at the University of Michigan's Mcity Testing Facility.
  arXiv  Detail & Related papers  (2025-05-22T04:28:59Z)
• A Comprehensive LLM-powered Framework for Driving Intelligence Evaluation [12.289371672810466]
  We propose an evaluation framework for driving behavior intelligence in complex traffic environments. Our research provides valuable insights for evaluating and designing more intelligent, human-like autonomous driving agents.
  arXiv  Detail & Related papers  (2025-03-07T06:03:02Z)
• Being good (at driving): Characterizing behavioral expectations on automated and human driven vehicles [0.4476047439400351]
  We put forth the concept of Drivership as a framing for the realization of good driving behaviors. Drivership grounds the evaluation of driving behaviors in the alignment between the mutualistic expectations that exist amongst road users.
  arXiv  Detail & Related papers  (2025-02-12T04:59:20Z)
• Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
  This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs. We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
  arXiv  Detail & Related papers  (2024-05-14T09:42:21Z)
• RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes [57.319845580050924]
  We propose a reinforcement learning framework that combines risk-sensitive control with an adaptive action space curriculum. We show that our algorithm is capable of learning high-speed policies for a real-world off-road driving task.
  arXiv  Detail & Related papers  (2024-05-07T23:32:36Z)
• Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Driving [4.1636282808157254]
  The end-to-end learning pipeline is gradually creating a paradigm shift in the ongoing development of highly autonomous vehicles. A lack of interpretability in real-time decisions with contemporary learning methods impedes user trust and attenuates the widespread deployment and commercialization of such vehicles. This survey seeks to answer the question: When and how can explanations improve safety of end-to-end autonomous driving?
  arXiv  Detail & Related papers  (2024-03-18T18:49:20Z)
• Analyze Drivers' Intervention Behavior During Autonomous Driving -- A VR-incorporated Approach [2.7532019227694344]
  This work sheds light on understanding human drivers' intervention behavior involved in the operation of autonomous vehicles. Experiment environments were implemented where the virtual reality (VR) and traffic micro-simulation are integrated. Performance indicators such as the probability of intervention, accident rates are defined and used to quantify and compare the risk levels.
  arXiv  Detail & Related papers  (2023-12-04T06:36:57Z)
• Classification of Safety Driver Attention During Autonomous Vehicle Operation [11.33083039877258]
  This paper introduces a dual-source approach integrating data from an infrared camera facing the vehicle operator and vehicle perception systems. The proposed system effectively determines a metric for the attention levels of the vehicle operator, enabling interventions such as warnings or reducing autonomous functionality as appropriate.
  arXiv  Detail & Related papers  (2023-10-17T22:04:42Z)
• Anticipating Driving Behavior through Deep Learning-Based Policy Prediction [66.344923925939]
  We developed a system that processes integrated visual features derived from video frames captured by a regular camera, along with depth details obtained from a point cloud scanner. This system is designed to anticipate driving actions, encompassing both vehicle speed and steering angle. Our evaluations indicate that the forecasts achieve a noteworthy level of accuracy in a minimum of half the test scenarios.
  arXiv  Detail & Related papers  (2023-07-20T17:38:55Z)
• Can Agents Run Relay Race with Strangers? Generalization of RL to Out-of-Distribution Trajectories [88.08381083207449]
  We show the prevalence of emphgeneralization failure on controllable states from stranger agents. We propose a novel method called Self-Trajectory Augmentation (STA), which will reset the environment to the agent's old states according to the Q function during training.
  arXiv  Detail & Related papers  (2023-04-26T10:12:12Z)
• From Spoken Thoughts to Automated Driving Commentary: Predicting and Explaining Intelligent Vehicles' Actions [10.557942353553859]
  In commentary driving, drivers verbalise their observations, assessments and intentions. By speaking out their thoughts, both learning and expert drivers are able to create a better understanding and awareness of their surroundings. In the intelligent vehicle context, automated driving commentary can provide intelligible explanations about driving actions.
  arXiv  Detail & Related papers  (2022-04-19T19:39:13Z)
• Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving [104.32241082170044]
  We study a new task, safety-aware motion prediction with unseen vehicles for autonomous driving. Unlike the existing trajectory prediction task for seen vehicles, we aim at predicting an occupancy map. Our approach is the first one that can predict the existence of unseen vehicles in most cases.
  arXiv  Detail & Related papers  (2021-09-03T13:33:33Z)
• End-to-End Urban Driving by Imitating a Reinforcement Learning Coach [148.2683592850329]
  Humans are good drivers, but not good coaches for end-to-end algorithms. We train a reinforcement learning expert that maps bird's-eye view images to continuous low-level actions. Supervised by our reinforcement learning coach, a baseline end-to-end agent with monocular camera-input achieves expert-level performance.
  arXiv  Detail & Related papers  (2021-08-18T17:36:51Z)
• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts? [104.04999499189402]
  Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment. We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP) Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes. We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
  arXiv  Detail & Related papers  (2020-06-26T11:07:32Z)
• Probabilistic End-to-End Vehicle Navigation in Complex Dynamic Environments with Multimodal Sensor Fusion [16.018962965273495]
  All-day and all-weather navigation is a critical capability for autonomous driving. We propose a probabilistic driving model with ultiperception capability utilizing the information from the camera, lidar and radar. The results suggest that our proposed model outperforms baselines and achieves excellent generalization performance in unseen environments.
  arXiv  Detail & Related papers  (2020-05-05T03:48:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.