Who's Driving? Game Theoretic Path Risk of AGI Development

• URL: http://arxiv.org/abs/2501.15280v1
• Date: Sat, 25 Jan 2025 17:13:12 GMT
• Title: Who's Driving? Game Theoretic Path Risk of AGI Development
• Authors: Robin Young,
• Abstract summary: Who controls the development of Artificial General Intelligence (AGI) might matter less than how we handle the fight for control itself.<n>We formalize this "steering wheel problem" as humanity's greatest near-term existential risk may stem not from misaligned AGI, but from the dynamics of competing to develop it.<n>We present a game theoretic framework modeling AGI development dynamics and prove conditions for sustainable cooperative equilibria.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Who controls the development of Artificial General Intelligence (AGI) might matter less than how we handle the fight for control itself. We formalize this "steering wheel problem" as humanity's greatest near-term existential risk may stem not from misaligned AGI, but from the dynamics of competing to develop it. Just as a car crash can occur from passengers fighting over the wheel before reaching any destination, catastrophic outcomes could arise from development competition long before AGI exists. While technical alignment research focuses on ensuring safe arrival, we show how coordination failures during development could drive us off the cliff first. We present a game theoretic framework modeling AGI development dynamics and prove conditions for sustainable cooperative equilibria. Drawing from nuclear control while accounting for AGI's unique characteristics, we propose concrete mechanisms including pre-registration, shared technical infrastructure, and automated deterrence to stabilize cooperation. Our key insight is that AGI creates network effects in safety: shared investments become more valuable as participation grows, enabling mechanism designs where cooperation dominates defection. This work bridges formal methodology and policy frameworks, providing foundations for practical governance of AGI competition risks.

Related papers

• Charting Uncertain Waters: A Socio-Technical Framework for Navigating GenAI's Impact on Open Source Communities [53.812795099349295]
  We conduct a scenario-driven, conceptual exploration using a socio-technical framework inspired by McLuhan's Tetrad to surface both risks and opportunities for community resilience amid GenAI-driven disruption of OSS development across four domains: software practices, documentation, community engagement, and governance.<n>By adopting this lens, OSS leaders and researchers can proactively shape the future of their ecosystems, rather than simply reacting to technological upheaval.
  arXiv  Detail & Related papers  (2025-08-06T22:54:15Z)
• Against racing to AGI: Cooperation, deterrence, and catastrophic risks [0.0]
  AGI Racing is the view that it is in the self-interest of major actors in AI development, especially powerful nations, to accelerate their frontier AI development.<n>We argue against AGI Racing.
  arXiv  Detail & Related papers  (2025-07-29T14:17:08Z)
• When Autonomy Goes Rogue: Preparing for Risks of Multi-Agent Collusion in Social Systems [78.04679174291329]
  We introduce a proof-of-concept to simulate the risks of malicious multi-agent systems (MAS)<n>We apply this framework to two high-risk fields: misinformation spread and e-commerce fraud.<n>Our findings show that decentralized systems are more effective at carrying out malicious actions than centralized ones.
  arXiv  Detail & Related papers  (2025-07-19T15:17:30Z)
• Misalignment or misuse? The AGI alignment tradeoff [0.0]
  We defend the view that misaligned AGI - future, generally intelligent (robotic) AI agents - poses catastrophic risks.<n>We show that there is room for alignment approaches which do not increase misuse risk.
  arXiv  Detail & Related papers  (2025-06-04T09:22:37Z)
• Generative AI for Autonomous Driving: Frontiers and Opportunities [145.6465312554513]
  This survey delivers a comprehensive synthesis of the emerging role of GenAI across the autonomous driving stack.<n>We begin by distilling the principles and trade-offs of modern generative modeling, encompassing VAEs, GANs, Diffusion Models, and Large Language Models.<n>We categorize practical applications, such as synthetic data generalization, end-to-end driving strategies, high-fidelity digital twin systems, smart transportation networks, and cross-domain transfer to embodied AI.
  arXiv  Detail & Related papers  (2025-05-13T17:59:20Z)
• Neurodivergent Influenceability as a Contingent Solution to the AI Alignment Problem [1.3905735045377272]
  The AI alignment problem, which focusses on ensuring that artificial intelligence (AI) systems act according to human values, presents profound challenges.<n>With the progression from narrow AI to Artificial General Intelligence (AGI) and Superintelligence, fears about control and existential risk have escalated.<n>Here, we investigate whether embracing inevitable AI misalignment can be a contingent strategy to foster a dynamic ecosystem of competing agents.
  arXiv  Detail & Related papers  (2025-05-05T11:33:18Z)
• Intelligence Sequencing and the Path-Dependence of Intelligence Evolution: AGI-First vs. DCI-First as Irreversible Attractors [0.0]
  This paper introduces the concept of intelligence sequencing. It argues that intelligence follows a path-dependent, irreversible trajectory. Findings suggest that intelligence sequencing is a civilizational tipping point.
  arXiv  Detail & Related papers  (2025-03-22T08:09:04Z)
• Humanoid Whole-Body Locomotion on Narrow Terrain via Dynamic Balance and Reinforcement Learning [54.26816599309778]
  We propose a novel whole-body locomotion algorithm based on dynamic balance and Reinforcement Learning (RL) Specifically, we introduce a dynamic balance mechanism by leveraging an extended measure of Zero-Moment Point (ZMP)-driven rewards and task-driven rewards in a whole-body actor-critic framework. Experiments conducted on a full-sized Unitree H1-2 robot verify the ability of our method to maintain balance on extremely narrow terrains.
  arXiv  Detail & Related papers  (2025-02-24T14:53:45Z)
• Beyond Accidents and Misuse: Decoding the Structural Risk Dynamics of Artificial Intelligence [0.0]
  This paper explores the concept of structural risks associated with the rapid integration of advanced AI systems across social, economic, and political systems. By analyzing the interactions between technological advancements and social dynamics, this study isolates three primary categories of structural risk. We present a comprehensive framework to understand the causal chains that drive these risks, highlighting the interdependence between structural forces and the more proximate risks of misuse and system failures.
  arXiv  Detail & Related papers  (2024-06-21T05:44:50Z)
• How Far Are We From AGI: Are LLMs All We Need? [15.705756259264932]
  AGI is distinguished by its ability to execute diverse real-world tasks with efficiency and effectiveness comparable to human intelligence. This paper outlines the requisite capability frameworks for AGI, integrating the internal, interface, and system dimensions. To give tangible insights into the ubiquitous impact of the integration of AI, we outline existing challenges and potential pathways toward AGI in multiple domains.
  arXiv  Detail & Related papers  (2024-05-16T17:59:02Z)
• Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
  This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs. We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
  arXiv  Detail & Related papers  (2024-05-14T09:42:21Z)
• Near to Mid-term Risks and Opportunities of Open-Source Generative AI [94.06233419171016]
  Applications of Generative AI are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about potential risks and resulted in calls for tighter regulation. This regulation is likely to put at risk the budding field of open-source Generative AI.
  arXiv  Detail & Related papers  (2024-04-25T21:14:24Z)
• What's my role? Modelling responsibility for AI-based safety-critical systems [1.0549609328807565]
  It is difficult for developers and manufacturers to be held responsible for harmful behaviour of an AI-SCS. A human operator can become a "liability sink" absorbing blame for the consequences of AI-SCS outputs they weren't responsible for creating. This paper considers different senses of responsibility (role, moral, legal and causal), and how they apply in the context of AI-SCS safety.
  arXiv  Detail & Related papers  (2023-12-30T13:45:36Z)
• RACER: Rational Artificial Intelligence Car-following-model Enhanced by Reality [51.244807332133696]
  This paper introduces RACER, a cutting-edge deep learning car-following model to predict Adaptive Cruise Control (ACC) driving behavior. Unlike conventional models, RACER effectively integrates Rational Driving Constraints (RDCs), crucial tenets of actual driving. RACER excels across key metrics, such as acceleration, velocity, and spacing, registering zero violations.
  arXiv  Detail & Related papers  (2023-12-12T06:21:30Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• The Promise and Peril of Artificial Intelligence -- Violet Teaming Offers a Balanced Path Forward [56.16884466478886]
  This paper reviews emerging issues with opaque and uncontrollable AI systems. It proposes an integrative framework called violet teaming to develop reliable and responsible AI. It emerged from AI safety research to manage risks proactively by design.
  arXiv  Detail & Related papers  (2023-08-28T02:10:38Z)
• A Game-Theoretic Framework for AI Governance [8.658519485150423]
  We show that the strategic interaction between the regulatory agencies and AI firms has an intrinsic structure reminiscent of a Stackelberg game. We propose a game-theoretic modeling framework for AI governance. To the best of our knowledge, this work is the first to use game theory for analyzing and structuring AI governance.
  arXiv  Detail & Related papers  (2023-05-24T08:18:42Z)
• Towards Safe, Explainable, and Regulated Autonomous Driving [11.043966021881426]
  We propose a framework that integrates autonomous control, explainable AI (XAI), and regulatory compliance. We describe relevant XAI approaches that can help achieve the goals of the framework.
  arXiv  Detail & Related papers  (2021-11-20T05:06:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.