Ergo, SMIRK is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake System

• URL: http://arxiv.org/abs/2204.07874v1
• Date: Sat, 16 Apr 2022 21:28:50 GMT
• Title: Ergo, SMIRK is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake System
• Authors: Markus Borg, Jens Henriksson, Kasper Socha, Olof Lennartsson, Elias Sonnsj\"o L\"onegren, Thanh Bui, Piotr Tomaszewski, Sankar Raman Sathyamoorthy, Sebastian Brink, Mahshid Helali Moghadam
• Abstract summary: Integration of Machine Learning (ML) components in critical applications introduces novel challenges for software certification and verification. New safety standards and technical guidelines are under development to support the safety of ML-based systems. We report results from an industry-academia collaboration on safety assurance of SMIRK, an ML-based pedestrian automatic emergency braking demonstrator.
• Score: 5.571920596648914
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Integration of Machine Learning (ML) components in critical applications introduces novel challenges for software certification and verification. New safety standards and technical guidelines are under development to support the safety of ML-based systems, e.g., ISO 21448 SOTIF for the automotive domain and the Assurance of Machine Learning for use in Autonomous Systems (AMLAS) framework. SOTIF and AMLAS provide high-level guidance but the details must be chiseled out for each specific case. We report results from an industry-academia collaboration on safety assurance of SMIRK, an ML-based pedestrian automatic emergency braking demonstrator running in an industry-grade simulator. We present the outcome of applying AMLAS on SMIRK for a minimalistic operational design domain, i.e., a complete safety case for its integrated ML-based component. Finally, we report lessons learned and provide both SMIRK and the safety case under an open-source licence for the research community to reuse.

Related papers

• SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models [75.67623347512368]
  We propose toolns, a comprehensive framework designed for conducting safety evaluations of MLLMs. Our framework consists of a comprehensive harmful query dataset and an automated evaluation protocol. Based on our framework, we conducted large-scale experiments on 15 widely-used open-source MLLMs and 6 commercial MLLMs.
  arXiv  Detail & Related papers  (2024-10-24T17:14:40Z)
• Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation [98.02846901473697]
  We propose ECSO (Eyes Closed, Safety On), a training-free protecting approach that exploits the inherent safety awareness of MLLMs. ECSO generates safer responses via adaptively transforming unsafe images into texts to activate the intrinsic safety mechanism of pre-aligned LLMs.
  arXiv  Detail & Related papers  (2024-03-14T17:03:04Z)
• ChatSOS: LLM-based knowledge Q&A system for safety engineering [0.0]
  This study introduces an LLM-based Q&A system for safety engineering, enhancing the comprehension and response accuracy of the model. We employ prompt engineering to incorporate external knowledge databases, thus enriching the LLM with up-to-date and reliable information. Our findings indicate that the integration of external knowledge significantly augments the capabilities of LLM for in-depth problem analysis and autonomous task assignment.
  arXiv  Detail & Related papers  (2023-12-14T03:25:23Z)
• Empowering Autonomous Driving with Large Language Models: A Safety Perspective [82.90376711290808]
  This paper explores the integration of Large Language Models (LLMs) into Autonomous Driving systems. LLMs are intelligent decision-makers in behavioral planning, augmented with a safety verifier shield for contextual safety learning. We present two key studies in a simulated environment: an adaptive LLM-conditioned Model Predictive Control (MPC) and an LLM-enabled interactive behavior planning scheme with a state machine.
  arXiv  Detail & Related papers  (2023-11-28T03:13:09Z)
• Benchmarking Automated Machine Learning Methods for Price Forecasting Applications [58.720142291102135]
  We show the possibility of substituting manually created ML pipelines with automated machine learning (AutoML) solutions. Based on the CRISP-DM process, we split the manual ML pipeline into a machine learning and non-machine learning part. We show in a case study for the industrial use case of price forecasting, that domain knowledge combined with AutoML can weaken the dependence on ML experts.
  arXiv  Detail & Related papers  (2023-04-28T10:27:38Z)
• Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks [70.76757529955577]
  This paper revisits prior work in this scope from the perspective of state-wise safe RL. We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
  arXiv  Detail & Related papers  (2022-12-12T06:30:17Z)
• Reliability Assessment and Safety Arguments for Machine Learning Components in Assuring Learning-Enabled Autonomous Systems [19.65793237440738]
  We present an overall assurance framework for Learning-Enabled Systems (LES) We then introduce a novel model-agnostic Reliability Assessment Model (RAM) for ML classifiers. We discuss the model assumptions and the inherent challenges of assessing ML reliability uncovered by our RAM.
  arXiv  Detail & Related papers  (2021-11-30T14:39:22Z)
• SoK: Machine Learning Governance [16.36671448193025]
  We develop the concept of ML governance to balance such benefits and risks. We use identities to hold principals accountable for failures of ML systems. We highlight the need for techniques that allow a model owner to manage the life cycle of their system.
  arXiv  Detail & Related papers  (2021-09-20T17:56:22Z)
• Guidance on the Assurance of Machine Learning in Autonomous Systems (AMLAS) [16.579772998870233]
  We introduce a methodology for the Assurance of Machine Learning for use in Autonomous Systems (AMLAS) AMLAS comprises a set of safety case patterns and a process for integrating safety assurance into the development of ML components.
  arXiv  Detail & Related papers  (2021-02-02T15:41:57Z)
• Safety design concepts for statistical machine learning components toward accordance with functional safety standards [0.38073142980732994]
  In recent years, curial incidents and accidents have been reported due to misjudgment of statistical machine learning. In this paper, we organize five kinds of technical safety concepts (TSCs) for components toward accordance with functional safety standards.
  arXiv  Detail & Related papers  (2020-08-04T01:01:00Z)
• Technology Readiness Levels for AI & ML [79.22051549519989]
  Development of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. Engineering systems follow well-defined processes and testing standards to streamline development for high-quality, reliable results. We propose a proven systems engineering approach for machine learning development and deployment.
  arXiv  Detail & Related papers  (2020-06-21T17:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.