Safety design concepts for statistical machine learning components toward accordance with functional safety standards

• URL: http://arxiv.org/abs/2008.01263v1
• Date: Tue, 4 Aug 2020 01:01:00 GMT
• Title: Safety design concepts for statistical machine learning components toward accordance with functional safety standards
• Authors: Akihisa Morikawa and Yutaka Matsubara
• Abstract summary: In recent years, curial incidents and accidents have been reported due to misjudgment of statistical machine learning. In this paper, we organize five kinds of technical safety concepts (TSCs) for components toward accordance with functional safety standards.
• Score: 0.38073142980732994
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, curial incidents and accidents have been reported due to un-intended control caused by misjudgment of statistical machine learning (SML), which include deep learning. The international functional safety standards for Electric/Electronic/Programmable (E/E/P) systems have been widely spread to improve safety. However, most of them do not recom-mended to use SML in safety critical systems so far. In practical the new concepts and methods are urgently required to enable SML to be safely used in safety critical systems. In this paper, we organize five kinds of technical safety concepts (TSCs) for SML components toward accordance with functional safety standards. We discuss not only quantitative evaluation criteria, but also development process based on XAI (eXplainable Artificial Intelligence) and Automotive SPICE to improve explainability and reliability in development phase. Fi-nally, we briefly compare the TSCs in cost and difficulty, and expect to en-courage further discussion in many communities and domain.

Related papers

• Defining and Evaluating Physical Safety for Large Language Models [62.4971588282174]
  Large Language Models (LLMs) are increasingly used to control robotic systems such as drones. Their risks of causing physical threats and harm in real-world applications remain unexplored. We classify the physical safety risks of drones into four categories: (1) human-targeted threats, (2) object-targeted threats, (3) infrastructure attacks, and (4) regulatory violations.
  arXiv  Detail & Related papers  (2024-11-04T17:41:25Z)
• Multimodal Situational Safety [73.63981779844916]
  We present the first evaluation and analysis of a novel safety challenge termed Multimodal Situational Safety. For an MLLM to respond safely, whether through language or action, it often needs to assess the safety implications of a language query within its corresponding visual context. We develop the Multimodal Situational Safety benchmark (MSSBench) to assess the situational safety performance of current MLLMs.
  arXiv  Detail & Related papers  (2024-10-08T16:16:07Z)
• SAFETY-J: Evaluating Safety with Critique [24.723999605458832]
  We introduce SAFETY-J, a bilingual generative safety evaluator for English and Chinese with critique-based judgment. We establish an automated meta-evaluation benchmark that objectively assesses the quality of critiques with minimal human intervention. Our evaluations demonstrate that SAFETY-J provides more nuanced and accurate safety evaluations, thereby enhancing both critique quality and predictive reliability in complex content scenarios.
  arXiv  Detail & Related papers  (2024-07-24T08:04:00Z)
• Towards Comprehensive and Efficient Post Safety Alignment of Large Language Models via Safety Patching [77.36097118561057]
  textscSafePatching is a novel framework for comprehensive and efficient PSA. textscSafePatching achieves a more comprehensive and efficient PSA than baseline methods.
  arXiv  Detail & Related papers  (2024-05-22T16:51:07Z)
• Safety Analysis of Autonomous Railway Systems: An Introduction to the SACRED Methodology [2.47737926497181]
  We introduce SACRED, a safety methodology for producing an initial safety case for autonomous systems. The development of SACRED is motivated by the proposed GoA-4 light-rail system in Berlin.
  arXiv  Detail & Related papers  (2024-03-18T11:12:19Z)
• The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness [56.174255970895466]
  Large Language Models (LLMs) play an increasingly pivotal role in natural language processing applications. This paper presents Safety and Over-Defensiveness Evaluation (SODE) benchmark.
  arXiv  Detail & Related papers  (2023-12-30T17:37:06Z)
• Leveraging Traceability to Integrate Safety Analysis Artifacts into the Software Development Process [51.42800587382228]
  Safety assurance cases (SACs) can be challenging to maintain during system evolution. We propose a solution that leverages software traceability to connect relevant system artifacts to safety analysis models. We elicit design rationales for system changes to help safety stakeholders analyze the impact of system changes on safety.
  arXiv  Detail & Related papers  (2023-07-14T16:03:27Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• Reliability Assessment and Safety Arguments for Machine Learning Components in Assuring Learning-Enabled Autonomous Systems [19.65793237440738]
  We present an overall assurance framework for Learning-Enabled Systems (LES) We then introduce a novel model-agnostic Reliability Assessment Model (RAM) for ML classifiers. We discuss the model assumptions and the inherent challenges of assessing ML reliability uncovered by our RAM.
  arXiv  Detail & Related papers  (2021-11-30T14:39:22Z)
• Usable Security for ML Systems in Mental Health: A Framework [2.436681150766912]
  This paper introduces a framework to guide and evaluate security-related designs, implementations, and deployments of Machine learning systems in mental health. We propose new principles and requirements to make security mechanisms usable for end-users of those ML systems in mental health. We present several concrete scenarios where different usable security cases and profiles in ML-systems in mental health applications are examined and evaluated.
  arXiv  Detail & Related papers  (2020-08-18T04:44:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.