RE for AI in Practice: Managing Data Annotation Requirements for AI Autonomous Driving Systems

• URL: http://arxiv.org/abs/2511.15859v1
• Date: Wed, 19 Nov 2025 20:27:30 GMT
• Title: RE for AI in Practice: Managing Data Annotation Requirements for AI Autonomous Driving Systems
• Authors: Hina Saeeda, Mazen Mohamad, Eric Knauss, Jennifer Horkoff, Ali Nouri,
• Abstract summary: High-quality data annotation requirements are crucial for the development of safe and reliable AI-enabled systems.<n>Our study investigates how annotation requirements are defined and used in practice.<n>Key challenges include ambiguity, edge case complexity, evolving requirements, inconsistencies, and resource constraints.
• Score: 3.9255502531644204
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-quality data annotation requirements are crucial for the development of safe and reliable AI-enabled perception systems (AIePS) in autonomous driving. Although these requirements play a vital role in reducing bias and enhancing performance, their formulation and management remain underexplored, leading to inconsistencies, safety risks, and regulatory concerns. Our study investigates how annotation requirements are defined and used in practice, the challenges in ensuring their quality, practitioner-recommended improvements, and their impact on AIePS development and performance. We conducted $19$ semi-structured interviews with participants from six international companies and four research organisations. Our thematic analysis reveals five main key challenges: ambiguity, edge case complexity, evolving requirements, inconsistencies, and resource constraints and three main categories of best practices, including ensuring compliance with ethical standards, improving data annotation requirements guidelines, and embedded quality assurance for data annotation requirements. We also uncover critical interrelationships between annotation requirements, annotation practices, annotated data quality, and AIePS performance and development, showing how requirement flaws propagate through the AIePS development pipeline. To the best of our knowledge, this study is the first to offer empirically grounded guidance on improving annotation requirements, offering actionable insights to enhance annotation quality, regulatory compliance, and system reliability. It also contributes to the emerging fields of Software Engineering (SE for AI) and Requirements Engineering (RE for AI) by bridging the gap between RE and AI in a timely and much-needed manner.

Related papers

• Safe and Certifiable AI Systems: Concepts, Challenges, and Lessons Learned [45.44933002008943]
  This white paper presents the T"UV AUSTRIA Trusted AI framework.<n>It is an end-to-end audit catalog and methodology for assessing and certifying machine learning systems.<n>Building on three pillars - Secure Software Development, Functional Requirements, and Ethics & Data Privacy - it translates the high-level obligations of the EU AI Act into specific, testable criteria.
  arXiv  Detail & Related papers  (2025-09-08T17:52:08Z)
• Never Compromise to Vulnerabilities: A Comprehensive Survey on AI Governance [211.5823259429128]
  We propose a comprehensive framework integrating technical and societal dimensions, structured around three interconnected pillars: Intrinsic Security, Derivative Security, and Social Ethics.<n>We identify three core challenges: (1) the generalization gap, where defenses fail against evolving threats; (2) inadequate evaluation protocols that overlook real-world risks; and (3) fragmented regulations leading to inconsistent oversight.<n>Our framework offers actionable guidance for researchers, engineers, and policymakers to develop AI systems that are not only robust and secure but also ethically aligned and publicly trustworthy.
  arXiv  Detail & Related papers  (2025-08-12T09:42:56Z)
• Explainability as a Compliance Requirement: What Regulated Industries Need from AI Tools for Design Artifact Generation [0.7874708385247352]
  We investigate the explainability gap in AI-driven design artifact generation through semistructured interviews with ten practitioners from safety-critical industries.<n>Our findings reveal that non-explainable AI outputs necessitate extensive manual validation, reduce stakeholder trust, struggle to handle domain-specific terminology, disrupt team collaboration, and introduce regulatory compliance risks.<n>This study outlines a practical roadmap for improving the transparency, reliability, and applicability of AI tools in requirements engineering.
  arXiv  Detail & Related papers  (2025-07-12T09:34:39Z)
• Rethinking Data Protection in the (Generative) Artificial Intelligence Era [138.07763415496288]
  We propose a four-level taxonomy that captures the diverse protection needs arising in modern (generative) AI models and systems.<n>Our framework offers a structured understanding of the trade-offs between data utility and control, spanning the entire AI pipeline.
  arXiv  Detail & Related papers  (2025-07-03T02:45:51Z)
• The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
  We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
  arXiv  Detail & Related papers  (2025-06-09T18:37:14Z)
• Work in Progress: AI-Powered Engineering-Bridging Theory and Practice [0.0]
  This paper explores how generative AI can help automate and improve key steps in systems engineering.<n>It examines AI's ability to analyze system requirements based on INCOSE's "good requirement" criteria.<n>The research aims to assess AI's potential to streamline engineering processes and improve learning outcomes.
  arXiv  Detail & Related papers  (2025-02-06T17:42:00Z)
• Bridging the Communication Gap: Evaluating AI Labeling Practices for Trustworthy AI Development [41.64451715899638]
  High-level AI labels, inspired by frameworks like EU energy labels, have been proposed to make the properties of AI models more transparent.<n>This study evaluates AI labeling through qualitative interviews along four key research questions.
  arXiv  Detail & Related papers  (2025-01-21T06:00:14Z)
• Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
  The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs) It examines the challenges introduced by AI components and the impact on testing procedures. The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
  arXiv  Detail & Related papers  (2024-02-21T08:29:42Z)
• Towards Implementing Responsible AI [22.514717870367623]
  We propose four aspects of AI system design and development, adapting processes used in software engineering. The salient findings cover four aspects of AI system design and development, adapting processes used in software engineering.
  arXiv  Detail & Related papers  (2022-05-09T14:59:23Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.