A Counterfactual Safety Margin Perspective on the Scoring of Autonomous
Vehicles' Riskiness
- URL: http://arxiv.org/abs/2308.01050v4
- Date: Tue, 28 Nov 2023 21:23:04 GMT
- Title: A Counterfactual Safety Margin Perspective on the Scoring of Autonomous
Vehicles' Riskiness
- Authors: Alessandro Zanardi, Andrea Censi, Margherita Atzei, Luigi Di Lillo,
Emilio Frazzoli
- Abstract summary: This paper presents a data-driven framework for assessing the risk of different AVs' behaviors.
We propose the notion of counterfactual safety margin, which represents the minimum deviation from nominal behavior that could cause a collision.
- Score: 52.27309191283943
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Autonomous Vehicles (AVs) promise a range of societal advantages, including
broader access to mobility, reduced road accidents, and enhanced transportation
efficiency. However, evaluating the risks linked to AVs is complex due to
limited historical data and the swift progression of technology. This paper
presents a data-driven framework for assessing the risk of different AVs'
behaviors in various operational design domains (ODDs), based on counterfactual
simulations of "misbehaving" road users. We propose the notion of
counterfactual safety margin, which represents the minimum deviation from
nominal behavior that could cause a collision. This methodology not only
pinpoints the most critical scenarios but also quantifies the (relative) risk's
frequency and severity concerning AVs. Importantly, we show that our approach
is applicable even when the AV's behavioral policy remains undisclosed, through
worst- and best-case analyses, benefiting external entities like regulators and
risk evaluators. Our experimental outcomes demonstrate the correlation between
the safety margin, the quality of the driving policy, and the ODD, shedding
light on the relative risks of different AV providers. Overall, this work
contributes to the safety assessment of AVs and addresses legislative and
insurance concerns surrounding this burgeoning technology.
Related papers
- SafetyAnalyst: Interpretable, transparent, and steerable safety moderation for AI behavior [56.10557932893919]
We present SafetyAnalyst, a novel AI safety moderation framework.
Given an AI behavior, SafetyAnalyst uses chain-of-thought reasoning to analyze its potential consequences.
It aggregates all harmful and beneficial effects into a harmfulness score using fully interpretable weight parameters.
arXiv Detail & Related papers (2024-10-22T03:38:37Z) - Automated Vehicles at Unsignalized Intersections: Safety and Efficiency Implications of Mixed-Human-Automated Traffic [6.9492069439607995]
The integration of automated vehicles (AVs) into transportation systems presents an unprecedented opportunity to enhance road safety and efficiency.
This study aims to bridge the gap by examining behavioral differences and adaptations of AVs and human-driven vehicles (HVs) at unsignalized intersections.
The findings reveal a paradox in mixed traffic flow: while AVs maintain larger safety margins, their conservative behavior can lead to unexpected situations for human drivers.
arXiv Detail & Related papers (2024-10-16T13:19:32Z) - EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction.
Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results.
However, the deployment of these agents in physical environments presents significant safety challenges.
This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
arXiv Detail & Related papers (2024-08-08T13:19:37Z) - Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Model [73.8765529028288]
We introduce a novel safety alignment challenge called Safe Inputs but Unsafe Output (SIUO) to evaluate cross-modality safety alignment.
To empirically investigate this problem, we developed the SIUO, a cross-modality benchmark encompassing 9 critical safety domains, such as self-harm, illegal activities, and privacy violations.
Our findings reveal substantial safety vulnerabilities in both closed- and open-source LVLMs, underscoring the inadequacy of current models to reliably interpret and respond to complex, real-world scenarios.
arXiv Detail & Related papers (2024-06-21T16:14:15Z) - FREA: Feasibility-Guided Generation of Safety-Critical Scenarios with Reasonable Adversariality [13.240598841087841]
We introduce FREA, a novel safety-critical scenarios generation method that incorporates the Largest Feasible Region (LFR) of AV as guidance.
Experiments illustrate that FREA can effectively generate safety-critical scenarios, yielding considerable near-miss events.
arXiv Detail & Related papers (2024-06-05T06:26:15Z) - A novel framework for adaptive stress testing of autonomous vehicles in
highways [3.2112502548606825]
We propose a novel framework to explore corner cases that can result in safety concerns in a highway traffic scenario.
We develop a new reward function for DRL to guide the AST in identifying crash scenarios based on the collision probability estimate.
The proposed framework is further integrated with a new driving model enabling us to create more realistic traffic scenarios.
arXiv Detail & Related papers (2024-02-19T04:02:40Z) - Safety Margins for Reinforcement Learning [53.10194953873209]
We show how to leverage proxy criticality metrics to generate safety margins.
We evaluate our approach on learned policies from APE-X and A3C within an Atari environment.
arXiv Detail & Related papers (2023-07-25T16:49:54Z) - Evaluating Model-free Reinforcement Learning toward Safety-critical
Tasks [70.76757529955577]
This paper revisits prior work in this scope from the perspective of state-wise safe RL.
We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection.
To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
arXiv Detail & Related papers (2022-12-12T06:30:17Z) - Watch Out for the Safety-Threatening Actors: Proactively Mitigating
Safety Hazards [5.898210877584262]
We propose a safety threat indicator (STI) using counterfactual reasoning to estimate the importance of each actor on the road with respect to its influence on the AV's safety.
Our approach reduces the accident rate for the state-of-the-art AV agent(s) in rare hazardous scenarios by more than 70%.
arXiv Detail & Related papers (2022-06-02T05:56:25Z) - Analyzing vehicle pedestrian interactions combining data cube structure
and predictive collision risk estimation model [5.73658856166614]
This study introduces a new concept of a pedestrian safety system that combines the field and the centralized processes.
The system can warn of upcoming risks immediately in the field and improve the safety of risk frequent areas by assessing the safety levels of roads without actual collisions.
arXiv Detail & Related papers (2021-07-26T23:00:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.