Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development

• URL: http://arxiv.org/abs/2501.16946v2
• Date: Wed, 29 Jan 2025 14:58:49 GMT
• Title: Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development
• Authors: Jan Kulveit, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger, David Duvenaud,
• Abstract summary: We analyze how even incremental improvements in AI capabilities can undermine human influence over large-scale systems that society depends on. We argue that this dynamic could lead to an effectively irreversible loss of human influence over crucial societal systems, precipitating an existential catastrophe through the permanent disempowerment of humanity.
• Score: 15.701299669203618
• License:
• Abstract: This paper examines the systemic risks posed by incremental advancements in artificial intelligence, developing the concept of `gradual disempowerment', in contrast to the abrupt takeover scenarios commonly discussed in AI safety. We analyze how even incremental improvements in AI capabilities can undermine human influence over large-scale systems that society depends on, including the economy, culture, and nation-states. As AI increasingly replaces human labor and cognition in these domains, it can weaken both explicit human control mechanisms (like voting and consumer choice) and the implicit alignments with human interests that often arise from societal systems' reliance on human participation to function. Furthermore, to the extent that these systems incentivise outcomes that do not line up with human preferences, AIs may optimize for those outcomes more aggressively. These effects may be mutually reinforcing across different domains: economic power shapes cultural narratives and political decisions, while cultural shifts alter economic and political behavior. We argue that this dynamic could lead to an effectively irreversible loss of human influence over crucial societal systems, precipitating an existential catastrophe through the permanent disempowerment of humanity. This suggests the need for both technical research and governance approaches that specifically address the risk of incremental erosion of human influence across interconnected societal systems.

Related papers

• Human Decision-making is Susceptible to AI-driven Manipulation [71.20729309185124]
  AI systems may exploit users' cognitive biases and emotional vulnerabilities to steer them toward harmful outcomes. This study examined human susceptibility to such manipulation in financial and emotional decision-making contexts.
  arXiv  Detail & Related papers  (2025-02-11T15:56:22Z)
• Causal Responsibility Attribution for Human-AI Collaboration [62.474732677086855]
  This paper presents a causal framework using Structural Causal Models (SCMs) to systematically attribute responsibility in human-AI systems. Two case studies illustrate the framework's adaptability in diverse human-AI collaboration scenarios.
  arXiv  Detail & Related papers  (2024-11-05T17:17:45Z)
• Imagining and building wise machines: The centrality of AI metacognition [78.76893632793497]
  We argue that shortcomings stem from one overarching failure: AI systems lack wisdom. While AI research has focused on task-level strategies, metacognition is underdeveloped in AI systems. We propose that integrating metacognitive capabilities into AI systems is crucial for enhancing their robustness, explainability, cooperation, and safety.
  arXiv  Detail & Related papers  (2024-11-04T18:10:10Z)
• Societal Adaptation to Advanced AI [1.2607853680700076]
  Existing strategies for managing risks from advanced AI systems often focus on affecting what AI systems are developed and how they diffuse. We urge a complementary approach: increasing societal adaptation to advanced AI. We introduce a conceptual framework which helps identify adaptive interventions that avoid, defend against and remedy potentially harmful uses of AI systems.
  arXiv  Detail & Related papers  (2024-05-16T17:52:12Z)
• Exploiting the Margin: How Capitalism Fuels AI at the Expense of Minoritized Groups [0.0]
  This paper explores the relationship between capitalism, racial injustice, and artificial intelligence (AI) It argues that AI acts as a contemporary vehicle for age-old forms of exploitation. The paper promotes an approach that integrates social justice and equity into the core of technological design and policy.
  arXiv  Detail & Related papers  (2024-03-10T22:40:07Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• Are You Worthy of My Trust?: A Socioethical Perspective on the Impacts of Trustworthy AI Systems on the Environment and Human Society [0.47138177023764666]
  We offer a brief, high-level overview of societal impacts of AI systems. We highlight the requirement of multi-disciplinary governance and convergence throughout its lifecycle.
  arXiv  Detail & Related papers  (2023-09-18T03:07:47Z)
• Reflective Hybrid Intelligence for Meaningful Human Control in Decision-Support Systems [4.1454448964078585]
  We introduce the notion of self-reflective AI systems for meaningful human control over AI systems. We propose a framework that integrates knowledge from psychology and philosophy with formal reasoning methods and machine learning approaches. We argue that self-reflective AI systems can lead to self-reflective hybrid systems (human + AI)
  arXiv  Detail & Related papers  (2023-07-12T13:32:24Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• Examining the Differential Risk from High-level Artificial Intelligence and the Question of Control [0.0]
  The extent and scope of future AI capabilities remain a key uncertainty. There are concerns over the extent of integration and oversight of AI opaque decision processes. This study presents a hierarchical complex systems framework to model AI risk and provide a template for alternative futures analysis.
  arXiv  Detail & Related papers  (2022-11-06T15:46:02Z)
• Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
  In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society. A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests. This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
  arXiv  Detail & Related papers  (2021-06-25T22:31:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.