Beyond Accidents and Misuse: Decoding the Structural Risk Dynamics of Artificial Intelligence

• URL: http://arxiv.org/abs/2406.14873v3
• Date: Fri, 16 May 2025 02:23:18 GMT
• Title: Beyond Accidents and Misuse: Decoding the Structural Risk Dynamics of Artificial Intelligence
• Authors: Kyle A Kilian,
• Abstract summary: This paper advances the concept of structural risk by introducing a framework grounded in complex systems research.<n>We classify structural risks into three categories: antecedent structural causes, antecedent AI system causes, and deleterious feedback loops.<n>To anticipate and govern these dynamics, the paper proposes a methodological agenda incorporating scenario mapping, simulation, and exploratory foresight.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: As artificial intelligence (AI) becomes increasingly embedded in the core functions of social, political, and economic life, it catalyzes structural transformations with far-reaching societal implications. This paper advances the concept of structural risk by introducing a framework grounded in complex systems research to examine how rapid AI integration can generate emergent, system-level dynamics beyond conventional, proximate threats such as system failures or malicious misuse. It argues that such risks are both influenced by and constitutive of broader sociotechnical structures. We classify structural risks into three interrelated categories: antecedent structural causes, antecedent AI system causes, and deleterious feedback loops. By tracing these interactions, we show how unchecked AI development can destabilize trust, shift power asymmetries, and erode decision-making agency across scales. To anticipate and govern these dynamics, the paper proposes a methodological agenda incorporating scenario mapping, simulation, and exploratory foresight. We conclude with policy recommendations aimed at cultivating institutional resilience and adaptive governance strategies for navigating an increasingly volatile AI risk landscape.

Related papers

• When Autonomy Goes Rogue: Preparing for Risks of Multi-Agent Collusion in Social Systems [78.04679174291329]
  We introduce a proof-of-concept to simulate the risks of malicious multi-agent systems (MAS)<n>We apply this framework to two high-risk fields: misinformation spread and e-commerce fraud.<n>Our findings show that decentralized systems are more effective at carrying out malicious actions than centralized ones.
  arXiv  Detail & Related papers  (2025-07-19T15:17:30Z)
• A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents [45.53643260046778]
  Recent advances in large language models (LLMs) have catalyzed the rise of autonomous AI agents.<n>These large-model agents mark a paradigm shift from static inference systems to interactive, memory-augmented entities.
  arXiv  Detail & Related papers  (2025-06-30T13:34:34Z)
• Toward Adaptive Categories: Dimensional Governance for Agentic AI [0.0]
  dimensional governance is a framework that tracks how decision authority, process autonomy, and accountability (the 3As) distribute dynamically across human-AI relationships.<n>A critical advantage of this approach is its ability to explicitly monitor system movement toward and across key governance thresholds.<n>We outline key dimensions, critical trust thresholds, and practical examples illustrating where rigid categorical frameworks fail.
  arXiv  Detail & Related papers  (2025-05-16T14:43:12Z)
• Adapting Probabilistic Risk Assessment for AI [0.0]
  General-purpose artificial intelligence (AI) systems present an urgent risk management challenge.<n>Current methods often rely on selective testing and undocumented assumptions about risk priorities.<n>This paper introduces the probabilistic risk assessment (PRA) for AI framework.
  arXiv  Detail & Related papers  (2025-04-25T17:59:14Z)
• SYMBIOSIS: Systems Thinking and Machine Intelligence for Better Outcomes in Society [0.0]
  SYMBIOSIS is an AI-powered framework and platform designed to make Systems Thinking accessible for addressing societal challenges. To address this, we developed a generative co-pilot that translates complex systems representations into natural language. SYMBIOSIS aims to serve as a foundational step to unlock future research into responsible and society-centered AI.
  arXiv  Detail & Related papers  (2025-03-07T17:07:26Z)
• AI Risk Atlas: Taxonomy and Tooling for Navigating AI Risks and Resources [24.502423087280008]
  We introduce the AI Risk Atlas, a structured taxonomy that consolidates AI risks from diverse sources and aligns them with governance frameworks. We also present the Risk Atlas Nexus, a collection of open-source tools designed to bridge the divide between risk definitions, benchmarks, datasets, and mitigation strategies.
  arXiv  Detail & Related papers  (2025-02-26T12:23:14Z)
• Multi-Agent Risks from Advanced AI [90.74347101431474]
  Multi-agent systems of advanced AI pose novel and under-explored risks. We identify three key failure modes based on agents' incentives, as well as seven key risk factors. We highlight several important instances of each risk, as well as promising directions to help mitigate them.
  arXiv  Detail & Related papers  (2025-02-19T23:03:21Z)
• Safety is Essential for Responsible Open-Ended Systems [47.172735322186]
  Open-Endedness is the ability of AI systems to continuously and autonomously generate novel and diverse artifacts or solutions. This position paper argues that the inherently dynamic and self-propagating nature of Open-Ended AI introduces significant, underexplored risks.
  arXiv  Detail & Related papers  (2025-02-06T21:32:07Z)
• Lessons from complexity theory for AI governance [1.6122472145662998]
  Complexity theory can help illuminate features of AI that pose central challenges for policymakers. We examine how efforts to govern AI are marked by deep uncertainty. We propose a set of complexity-compatible principles concerning the timing and structure of AI governance.
  arXiv  Detail & Related papers  (2025-01-07T07:56:40Z)
• Considerations Influencing Offense-Defense Dynamics From Artificial Intelligence [0.0]
  AI can enhance defensive capabilities but also presents avenues for malicious exploitation and large-scale societal harm. This paper proposes a taxonomy to map and examine the key factors that influence whether AI systems predominantly pose threats or offer protective benefits to society.
  arXiv  Detail & Related papers  (2024-12-05T10:05:53Z)
• A Taxonomy of Systemic Risks from General-Purpose AI [2.5956465292067867]
  We consider systemic risks as large-scale threats that can affect entire societies or economies.<n>Key sources of systemic risk emerge from knowledge gaps, challenges in recognizing harm, and the unpredictable trajectory of AI development.<n>This paper contributes to AI safety research by providing a structured groundwork for understanding and addressing the potential large-scale negative societal impacts of general-purpose AI.
  arXiv  Detail & Related papers  (2024-11-24T22:16:18Z)
• Causal Responsibility Attribution for Human-AI Collaboration [62.474732677086855]
  This paper presents a causal framework using Structural Causal Models (SCMs) to systematically attribute responsibility in human-AI systems. Two case studies illustrate the framework's adaptability in diverse human-AI collaboration scenarios.
  arXiv  Detail & Related papers  (2024-11-05T17:17:45Z)
• Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI [52.138044013005]
  generative AI, particularly large language models (LLMs), become increasingly integrated into production applications. New attack surfaces and vulnerabilities emerge and put a focus on adversarial threats in natural language and multi-modal systems. Red-teaming has gained importance in proactively identifying weaknesses in these systems, while blue-teaming works to protect against such adversarial attacks. This work aims to bridge the gap between academic insights and practical security measures for the protection of generative AI systems.
  arXiv  Detail & Related papers  (2024-09-23T10:18:10Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction.<n>Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results.<n>However, the deployment of these agents in physical environments presents significant safety challenges.<n>This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• The Transformation Risk-Benefit Model of Artificial Intelligence: Balancing Risks and Benefits Through Practical Solutions and Use Cases [0.0]
  The authors propose a new framework called "The Transformation Risk-Benefit Model of Artificial Intelligence" Using the model characteristics, the article emphasizes practical and innovative solutions where benefits outweigh risks.
  arXiv  Detail & Related papers  (2024-04-11T19:19:57Z)
• Quantifying AI Vulnerabilities: A Synthesis of Complexity, Dynamical Systems, and Game Theory [0.0]
  We propose a novel approach that introduces three metrics: System Complexity Index (SCI), Lyapunov Exponent for AI Stability (LEAIS), and Nash Equilibrium Robustness (NER) SCI quantifies the inherent complexity of an AI system, LEAIS captures its stability and sensitivity to perturbations, and NER evaluates its strategic robustness against adversarial manipulation.
  arXiv  Detail & Related papers  (2024-04-07T07:05:59Z)
• Emergent Explainability: Adding a causal chain to neural network inference [0.0]
  This position paper presents a theoretical framework for enhancing explainable artificial intelligence (xAI) through emergent communication (EmCom) We explore the novel integration of EmCom into AI systems, offering a paradigm shift from conventional associative relationships between inputs and outputs to a more nuanced, causal interpretation. The paper discusses the theoretical underpinnings of this approach, its potential broad applications, and its alignment with the growing need for responsible and transparent AI systems.
  arXiv  Detail & Related papers  (2024-01-29T02:28:39Z)
• A Framework for Exploring the Consequences of AI-Mediated Enterprise Knowledge Access and Identifying Risks to Workers [3.4568218861862556]
  This paper presents the Consequence-Mechanism-Risk framework to identify risks to workers from AI-mediated enterprise knowledge access systems. We have drawn on wide-ranging literature detailing risks to workers, and categorised risks as being to worker value, power, and wellbeing. Future work could apply this framework to other technological systems to promote the protection of workers and other groups.
  arXiv  Detail & Related papers  (2023-12-08T17:05:40Z)
• Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic Systems [67.01132165581667]
  We propose to enable high-level reasoning in AI systems by integrating cognitive architectures with external neuro-symbolic components. We illustrate a hybrid framework centered on ACT-R and we discuss the role of generative models in recent and future applications.
  arXiv  Detail & Related papers  (2023-11-13T21:20:17Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• Digital Deception: Generative Artificial Intelligence in Social Engineering and Phishing [7.1795069620810805]
  This paper investigates the transformative role of Generative AI in Social Engineering (SE) attacks. We use a theory of social engineering to identify three pillars where Generative AI amplifies the impact of SE attacks. Our study aims to foster a deeper understanding of the risks, human implications, and countermeasures associated with this emerging paradigm.
  arXiv  Detail & Related papers  (2023-10-15T07:55:59Z)
• Predictable Artificial Intelligence [77.1127726638209]
  This paper introduces the ideas and challenges of Predictable AI. It explores the ways in which we can anticipate key validity indicators of present and future AI ecosystems. We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems.
  arXiv  Detail & Related papers  (2023-10-09T21:36:21Z)
• Towards an Interface Description Template for AI-enabled Systems [77.34726150561087]
  Reuse is a common system architecture approach that seeks to instantiate a system architecture with existing components. There is currently no framework that guides the selection of necessary information to assess their portability to operate in a system different than the one for which the component was originally purposed. We present ongoing work on establishing an interface description template that captures the main information of an AI-enabled component.
  arXiv  Detail & Related papers  (2020-07-13T20:30:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.