STAMP/STPA informed characterization of Factors Leading to Loss of Control in AI Systems

• URL: http://arxiv.org/abs/2512.17600v1
• Date: Fri, 19 Dec 2025 14:07:32 GMT
• Title: STAMP/STPA informed characterization of Factors Leading to Loss of Control in AI Systems
• Authors: Steve Barrett, Anna Bruvere, Sean P. Fillingham, Catherine Rhodes, Stefano Vergani,
• Abstract summary: We set out to provide a more structured framework for discussing and characterizing loss of control.<n>We use this framework to assist those responsible for the safe operation of AI-containing socio-technical systems to identify causal factors leading to loss of control.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A major concern amongst AI safety practitioners is the possibility of loss of control, whereby humans lose the ability to exert control over increasingly advanced AI systems. The range of concerns is wide, spanning current day risks to future existential risks, and a range of loss of control pathways from rapid AI self-exfiltration scenarios to more gradual disempowerment scenarios. In this work we set out to firstly, provide a more structured framework for discussing and characterizing loss of control and secondly, to use this framework to assist those responsible for the safe operation of AI-containing socio-technical systems to identify causal factors leading to loss of control. We explore how these two needs can be better met by making use of a methodology developed within the safety-critical systems community known as STAMP and its associated hazard analysis technique of STPA. We select the STAMP methodology primarily because it is based around a world-view that socio-technical systems can be functionally modeled as control structures, and that safety issues arise when there is a loss of control in these structures.

Related papers

• A New Perspective On AI Safety Through Control Theory Methodologies [16.51699616065134]
  AI promises to achieve a new level of autonomy but is hampered by a lack of safety assurance.<n>This article outlines a new perspective on AI safety based on an interdisciplinary interpretation of the underlying data-generation process.<n>The new perspective, also referred to as data control, aims to stimulate AI engineering to take advantage of existing safety analysis and assurance.
  arXiv  Detail & Related papers  (2025-06-30T10:26:59Z)
• Towards provable probabilistic safety for scalable embodied AI systems [79.31011047593492]
  Embodied AI systems are increasingly prevalent across various applications.<n> Ensuring their safety in complex operating environments remains a major challenge.<n>This Perspective offers a pathway toward safer, large-scale adoption of embodied AI systems in safety-critical applications.
  arXiv  Detail & Related papers  (2025-06-05T15:46:25Z)
• Systematic Hazard Analysis for Frontier AI using STPA [0.0]
  frontier AI companies currently do not describe in detail any structured approach to identifying and analysing hazards.<n>A (Systems-Theoretic Process Analysis) is a systematic methodology for identifying how complex systems can become unsafe, leading to hazards.<n>We evaluateA's ability to broaden the scope, improve traceability and strengthen the robustness of safety assurance for frontier AI systems.
  arXiv  Detail & Related papers  (2025-06-02T15:28:34Z)
• Adapting Probabilistic Risk Assessment for AI [0.0]
  General-purpose artificial intelligence (AI) systems present an urgent risk management challenge.<n>Current methods often rely on selective testing and undocumented assumptions about risk priorities.<n>This paper introduces the probabilistic risk assessment (PRA) for AI framework.
  arXiv  Detail & Related papers  (2025-04-25T17:59:14Z)
• An Approach to Technical AGI Safety and Security [72.83728459135101]
  We develop an approach to address the risk of harms consequential enough to significantly harm humanity.<n>We focus on technical approaches to misuse and misalignment.<n>We briefly outline how these ingredients could be combined to produce safety cases for AGI systems.
  arXiv  Detail & Related papers  (2025-04-02T15:59:31Z)
• From Silos to Systems: Process-Oriented Hazard Analysis for AI Systems [2.226040060318401]
  We translate System Theoretic Process Analysis (STPA) for analyzing AI operation and development processes. We focus on systems that rely on machine learning algorithms and conductedA on three case studies. We find that key concepts and steps of conducting anA readily apply, albeit with a few adaptations tailored for AI systems.
  arXiv  Detail & Related papers  (2024-10-29T20:43:18Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction.<n>Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results.<n>However, the deployment of these agents in physical environments presents significant safety challenges.<n>This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems [88.80306881112313]
  We will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them.
  arXiv  Detail & Related papers  (2024-05-10T17:38:32Z)
• Leveraging Traceability to Integrate Safety Analysis Artifacts into the Software Development Process [51.42800587382228]
  Safety assurance cases (SACs) can be challenging to maintain during system evolution. We propose a solution that leverages software traceability to connect relevant system artifacts to safety analysis models. We elicit design rationales for system changes to help safety stakeholders analyze the impact of system changes on safety.
  arXiv  Detail & Related papers  (2023-07-14T16:03:27Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• System Safety and Artificial Intelligence [0.0]
  New applications of AI across societal domains come with new hazards. The field of system safety has dealt with accidents and harm in safety-critical systems. This chapter honors system safety pioneer Nancy Leveson.
  arXiv  Detail & Related papers  (2022-02-18T16:37:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.