On Robustness of Lane Detection Models to Physical-World Adversarial Attacks in Autonomous Driving

• URL: http://arxiv.org/abs/2107.02488v1
• Date: Tue, 6 Jul 2021 09:04:47 GMT
• Title: On Robustness of Lane Detection Models to Physical-World Adversarial Attacks in Autonomous Driving
• Authors: Takami Sato and Qi Alfred Chen
• Abstract summary: After the 2017 TuSimple Lane Detection Challenge, its evaluation based on accuracy and F1 score has become the de facto standard to measure the performance of lane detection methods. We conduct the first large-scale empirical study to evaluate the robustness of state-of-the-art lane detection methods under physical-world adversarial attacks in autonomous driving.
• Score: 12.412448947321828
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: After the 2017 TuSimple Lane Detection Challenge, its evaluation based on accuracy and F1 score has become the de facto standard to measure the performance of lane detection methods. In this work, we conduct the first large-scale empirical study to evaluate the robustness of state-of-the-art lane detection methods under physical-world adversarial attacks in autonomous driving. We evaluate 4 major types of lane detection approaches with the conventional evaluation and end-to-end evaluation in autonomous driving scenarios and then discuss the security proprieties of each lane detection model. We demonstrate that the conventional evaluation fails to reflect the robustness in end-to-end autonomous driving scenarios. Our results show that the most robust model on the conventional metrics is the least robust in the end-to-end evaluation. Although the competition dataset and its metrics have played a substantial role in developing performant lane detection methods along with the rapid development of deep neural networks, the conventional evaluation is becoming obsolete and the gap between the metrics and practicality is critical. We hope that our study will help the community make further progress in building a more comprehensive framework to evaluate lane detection models.

Related papers

• Exploring the Causality of End-to-End Autonomous Driving [57.631400236930375]
  We propose a comprehensive approach to explore and analyze the causality of end-to-end autonomous driving. Our work is the first to unveil the mystery of end-to-end autonomous driving and turn the black box into a white one.
  arXiv  Detail & Related papers  (2024-07-09T04:56:11Z)
• Evaluating Roadside Perception for Autonomous Vehicles: Insights from Field Testing [7.755003755937953]
  This paper introduces a comprehensive evaluation methodology specifically designed to assess the performance of roadside perception systems. Our methodology encompasses measurement techniques, metric selection, and experimental trial design, all grounded in real-world field testing. The findings of this study are poised to inform the development of industry-standard benchmarks and evaluation methods.
  arXiv  Detail & Related papers  (2024-01-22T22:47:02Z)
• DRUformer: Enhancing the driving scene Important object detection with driving relationship self-understanding [50.81809690183755]
  Traffic accidents frequently lead to fatal injuries, contributing to over 50 million deaths until 2023. Previous research primarily assessed the importance of individual participants, treating them as independent entities. We introduce Driving scene Relationship self-Understanding transformer (DRUformer) to enhance the important object detection task.
  arXiv  Detail & Related papers  (2023-11-11T07:26:47Z)
• From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework [91.94389491920309]
  Textual adversarial attacks can discover models' weaknesses by adding semantic-preserved but misleading perturbations to the inputs. The existing practice of robustness evaluation may exhibit issues of incomprehensive evaluation, impractical evaluation protocol, and invalid adversarial samples. We set up a unified automatic robustness evaluation framework, shifting towards model-centric evaluation to exploit the advantages of adversarial attacks.
  arXiv  Detail & Related papers  (2023-05-29T14:55:20Z)
• Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embedding [51.8579160500354]
  We propose an unsupervised way to predict self-driving attention by uncertainty modeling and driving knowledge integration. Results show equivalent or even more impressive performance compared to fully-supervised state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-03-17T00:28:33Z)
• FBLNet: FeedBack Loop Network for Driver Attention Prediction [75.83518507463226]
  Nonobjective driving experience is difficult to model. In this paper, we propose a FeedBack Loop Network (FBLNet) which attempts to model the driving experience accumulation procedure. Under the guidance of the incremental knowledge, our model fuses the CNN feature and Transformer feature that are extracted from the input image to predict driver attention.
  arXiv  Detail & Related papers  (2022-12-05T08:25:09Z)
• Is my Driver Observation Model Overconfident? Input-guided Calibration Networks for Reliable and Interpretable Confidence Estimates [23.449073032842076]
  Driver observation models are rarely deployed under perfect conditions. We show that raw neural network-based approaches tend to significantly overestimate their prediction quality. We introduce Calibrated Action Recognition with Input Guidance (CARING)-a novel approach leveraging an additional neural network to learn scaling the confidences depending on the video representation.
  arXiv  Detail & Related papers  (2022-04-10T12:43:58Z)
• Towards Driving-Oriented Metric for Lane Detection Models [19.81163190104571]
  We design 2 new driving-oriented metrics for lane detection: End-to-End Lateral Deviation metric (E2E-LD) and Per-frame Simulated Lateral Deviation metric (PSLD) To evaluate the validity of the metrics, we conduct a large-scale empirical study with 4 major types of lane detection approaches on the TuSimple dataset and our newly constructed dataset Comma2k19-LD.
  arXiv  Detail & Related papers  (2022-03-31T07:24:44Z)
• Evaluating Adversarial Attacks on Driving Safety in Vision-Based Autonomous Vehicles [21.894836150974093]
  In recent years, many deep learning models have been adopted in autonomous driving. Recent studies have demonstrated that adversarial attacks can cause a significant decline in detection precision of deep learning-based 3D object detection models.
  arXiv  Detail & Related papers  (2021-08-06T04:52:09Z)
• A Multi-Layered Approach for Measuring the Simulation-to-Reality Gap of Radar Perception for Autonomous Driving [0.0]
  In order to rely on virtual tests the employed sensor models have to be validated. There exists no sound method to measure this simulation-to-reality gap of radar perception. We have shown the effectiveness of the proposed approach in terms of providing an in-depth sensor model assessment.
  arXiv  Detail & Related papers  (2021-06-15T18:51:39Z)
• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts? [104.04999499189402]
  Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment. We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP) Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes. We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
  arXiv  Detail & Related papers  (2020-06-26T11:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.