Explainable deep learning improves human mental models of self-driving cars

• URL: http://arxiv.org/abs/2411.18714v1
• Date: Wed, 27 Nov 2024 19:38:43 GMT
• Title: Explainable deep learning improves human mental models of self-driving cars
• Authors: Eoin M. Kenny, Akshay Dharmavaram, Sang Uk Lee, Tung Phan-Minh, Shreyas Rajesh, Yunqing Hu, Laura Major, Momchil S. Tomov, Julie A. Shah,
• Abstract summary: Concept-wrapper network (i.e., CW-Net) is a method for explaining the behavior of black-box motion planners. We deploy CW-Net on a real self-driving car and show that the resulting explanations refine the human driver's mental model of the car. We anticipate our method could be applied to other safety-critical systems with a human in the loop, such as autonomous drones and robotic surgeons.
• Score: 12.207001033390226
• License:
• Abstract: Self-driving cars increasingly rely on deep neural networks to achieve human-like driving. However, the opacity of such black-box motion planners makes it challenging for the human behind the wheel to accurately anticipate when they will fail, with potentially catastrophic consequences. Here, we introduce concept-wrapper network (i.e., CW-Net), a method for explaining the behavior of black-box motion planners by grounding their reasoning in human-interpretable concepts. We deploy CW-Net on a real self-driving car and show that the resulting explanations refine the human driver's mental model of the car, allowing them to better predict its behavior and adjust their own behavior accordingly. Unlike previous work using toy domains or simulations, our study presents the first real-world demonstration of how to build authentic autonomous vehicles (AVs) that give interpretable, causally faithful explanations for their decisions, without sacrificing performance. We anticipate our method could be applied to other safety-critical systems with a human in the loop, such as autonomous drones and robotic surgeons. Overall, our study suggests a pathway to explainability for autonomous agents as a whole, which can help make them more transparent, their deployment safer, and their usage more ethical.

Related papers

• IGDrivSim: A Benchmark for the Imitation Gap in Autonomous Driving [35.64960921334498]
  textbfIGDrivSim is a benchmark built on top of the Waymax simulator. Our experiments show that this perception gap can hinder the learning of safe and effective driving behaviors. We show that combining imitation with reinforcement learning, using a simple penalty reward for prohibited behaviors, effectively mitigates these failures.
  arXiv  Detail & Related papers  (2024-11-07T12:28:52Z)
• Exploring the Causality of End-to-End Autonomous Driving [57.631400236930375]
  We propose a comprehensive approach to explore and analyze the causality of end-to-end autonomous driving. Our work is the first to unveil the mystery of end-to-end autonomous driving and turn the black box into a white one.
  arXiv  Detail & Related papers  (2024-07-09T04:56:11Z)
• Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
  This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs. We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
  arXiv  Detail & Related papers  (2024-05-14T09:42:21Z)
• Incorporating Explanations into Human-Machine Interfaces for Trust and Situation Awareness in Autonomous Vehicles [4.1636282808157254]
  We study the role of explainable AI and human-machine interface jointly in building trust in vehicle autonomy. We present a situation awareness framework for calibrating users' trust in self-driving behavior.
  arXiv  Detail & Related papers  (2024-04-10T23:02:13Z)
• A Language Agent for Autonomous Driving [31.359413767191608]
  We propose a paradigm shift to integrate human-like intelligence into autonomous driving systems. Our approach, termed Agent-Driver, transforms the traditional autonomous driving pipeline by introducing a versatile tool library. Powered by Large Language Models (LLMs), our Agent-Driver is endowed with intuitive common sense and robust reasoning capabilities.
  arXiv  Detail & Related papers  (2023-11-17T18:59:56Z)
• Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Driving [22.21822829138535]
  We propose a new approach using concept bottlenecks as visual features for control command predictions and explanations of user and vehicle behavior. We learn a human-understandable concept layer that we use to explain sequential driving scenes while learning vehicle control commands. This approach can then be used to determine whether a change in a preferred gap or steering commands from a human (or autonomous vehicle) is led by an external stimulus or change in preferences.
  arXiv  Detail & Related papers  (2023-10-25T13:39:04Z)
• Robot Learning with Sensorimotor Pre-training [98.7755895548928]
  We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
  arXiv  Detail & Related papers  (2023-06-16T17:58:10Z)
• Tackling Real-World Autonomous Driving using Deep Reinforcement Learning [63.3756530844707]
  In this work, we propose a model-free Deep Reinforcement Learning Planner training a neural network that predicts acceleration and steering angle. In order to deploy the system on board the real self-driving car, we also develop a module represented by a tiny neural network.
  arXiv  Detail & Related papers  (2022-07-05T16:33:20Z)
• Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving [104.32241082170044]
  We study a new task, safety-aware motion prediction with unseen vehicles for autonomous driving. Unlike the existing trajectory prediction task for seen vehicles, we aim at predicting an occupancy map. Our approach is the first one that can predict the existence of unseen vehicles in most cases.
  arXiv  Detail & Related papers  (2021-09-03T13:33:33Z)
• Explainability of vision-based autonomous driving systems: Review and challenges [33.720369945541805]
  The need for explainability is strong in driving, a safety-critical application. This survey gathers contributions from several research fields, namely computer vision, deep learning, autonomous driving, explainable AI (X-AI)
  arXiv  Detail & Related papers  (2021-01-13T19:09:38Z)
• Learning Accurate and Human-Like Driving using Semantic Maps and Attention [152.48143666881418]
  This paper investigates how end-to-end driving models can be improved to drive more accurately and human-like. We exploit semantic and visual maps from HERE Technologies and augment the existing Drive360 dataset with such. Our models are trained and evaluated on the Drive360 + HERE dataset, which features 60 hours and 3000 km of real-world driving data.
  arXiv  Detail & Related papers  (2020-07-10T22:25:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.