Dataset Safety in Autonomous Driving: Requirements, Risks, and Assurance

• URL: http://arxiv.org/abs/2511.08439v1
• Date: Wed, 12 Nov 2025 01:58:48 GMT
• Title: Dataset Safety in Autonomous Driving: Requirements, Risks, and Assurance
• Authors: Alireza Abbaspour, Tejaskumar Balgonda Patil, B Ravi Kiran, Russel Mohr, Senthil Yogamani,
• Abstract summary: This paper presents a structured framework for developing safe datasets aligned with ISO/PAS 8800 guidelines.<n>Using AI-based perception systems as the primary use case, it introduces the AI Data Flywheel and the dataset lifecycle.<n>The framework incorporates rigorous safety analyses to identify hazards and mitigate risks caused by dataset insufficiencies.
• Score: 1.5495593104596397
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dataset integrity is fundamental to the safety and reliability of AI systems, especially in autonomous driving. This paper presents a structured framework for developing safe datasets aligned with ISO/PAS 8800 guidelines. Using AI-based perception systems as the primary use case, it introduces the AI Data Flywheel and the dataset lifecycle, covering data collection, annotation, curation, and maintenance. The framework incorporates rigorous safety analyses to identify hazards and mitigate risks caused by dataset insufficiencies. It also defines processes for establishing dataset safety requirements and proposes verification and validation strategies to ensure compliance with safety standards. In addition to outlining best practices, the paper reviews recent research and emerging trends in dataset safety and autonomous vehicle development, providing insights into current challenges and future directions. By integrating these perspectives, the paper aims to advance robust, safety-assured AI systems for autonomous driving applications.

Related papers

• Rethinking Data Protection in the (Generative) Artificial Intelligence Era [138.07763415496288]
  We propose a four-level taxonomy that captures the diverse protection needs arising in modern (generative) AI models and systems.<n>Our framework offers a structured understanding of the trade-offs between data utility and control, spanning the entire AI pipeline.
  arXiv  Detail & Related papers  (2025-07-03T02:45:51Z)
• A Verification Methodology for Safety Assurance of Robotic Autonomous Systems [0.1962656580942496]
  This paper presents a verification workflow for the safety assurance of an autonomous agricultural robot.<n>It covers the entire development life-cycle, from concept study and design to runtime verification.<n>Results show that the methodology can be effectively used to verify safety-critical properties and facilitate the early identification of design issues.
  arXiv  Detail & Related papers  (2025-06-24T13:39:51Z)
• SafeAgent: Safeguarding LLM Agents via an Automated Risk Simulator [77.86600052899156]
  Large Language Model (LLM)-based agents are increasingly deployed in real-world applications.<n>We propose AutoSafe, the first framework that systematically enhances agent safety through fully automated synthetic data generation.<n>We show that AutoSafe boosts safety scores by 45% on average and achieves a 28.91% improvement on real-world tasks.
  arXiv  Detail & Related papers  (2025-05-23T10:56:06Z)
• UniSTPA: A Safety Analysis Framework for End-to-End Autonomous Driving [10.063740202765343]
  We propose the Unified System Theoretic Process Analysis (UniSTPA) framework.<n>UniSTPA performs hazard analysis not only at the component level but also within the model's internal layers.<n>The proposed framework thus offers both theoretical and practical guidance for the safe development and deployment of end-to-end autonomous driving systems.
  arXiv  Detail & Related papers  (2025-05-21T01:23:31Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction.<n>Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results.<n>However, the deployment of these agents in physical environments presents significant safety challenges.<n>This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems [88.80306881112313]
  We will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them.
  arXiv  Detail & Related papers  (2024-05-10T17:38:32Z)
• Inherent Diverse Redundant Safety Mechanisms for AI-based Software Elements in Automotive Applications [1.6495054381576084]
  This paper explores the role and challenges of Artificial Intelligence (AI) algorithms in autonomous driving systems. A primary concern relates to the ability (and necessity) of AI models to generalize beyond their initial training data. This paper investigates the risk associated with overconfident AI models in safety-critical applications like autonomous driving.
  arXiv  Detail & Related papers  (2024-02-13T04:15:26Z)
• Safety of autonomous vehicles: A survey on Model-based vs. AI-based approaches [1.370633147306388]
  It is proposed to review research on relevant methods and concepts defining an overall control architecture for AVs. It is intended through this reviewing process to highlight researches that use either model-based methods or AI-based approaches. This paper ends with discussions on the methods used to guarantee the safety of AVs namely: safety verification techniques and the standardization/generalization of safety frameworks.
  arXiv  Detail & Related papers  (2023-05-29T08:05:32Z)
• Meta-Learning Priors for Safe Bayesian Optimization [72.8349503901712]
  We build on a meta-learning algorithm, F-PACOH, capable of providing reliable uncertainty quantification in settings of data scarcity. As core contribution, we develop a novel framework for choosing safety-compliant priors in a data-riven manner. On benchmark functions and a high-precision motion system, we demonstrate that our meta-learned priors accelerate the convergence of safe BO approaches.
  arXiv  Detail & Related papers  (2022-10-03T08:38:38Z)
• An Empirical Analysis of the Use of Real-Time Reachability for the Safety Assurance of Autonomous Vehicles [7.1169864450668845]
  We propose using a real-time reachability algorithm for the implementation of the simplex architecture to assure the safety of a 1/10 scale open source autonomous vehicle platform. In our approach, the need to analyze an underlying controller is abstracted away, instead focusing on the effects of the controller's decisions on the system's future states.
  arXiv  Detail & Related papers  (2022-05-03T11:12:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.