Position Paper: Bounded Alignment: What (Not) To Expect From AGI Agents

• URL: http://arxiv.org/abs/2505.11866v1
• Date: Sat, 17 May 2025 06:17:57 GMT
• Title: Position Paper: Bounded Alignment: What (Not) To Expect From AGI Agents
• Authors: Ali A. Minai,
• Abstract summary: The goal of this position paper is to argue that the currently dominant vision of AGI in the AI and machine learning (AI/ML) community needs to evolve.<n>This change in perspective will lead to a more realistic view of the technology, and allow for better policy decisions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The issues of AI risk and AI safety are becoming critical as the prospect of artificial general intelligence (AGI) looms larger. The emergence of extremely large and capable generative models has led to alarming predictions and created a stir from boardrooms to legislatures. As a result, AI alignment has emerged as one of the most important areas in AI research. The goal of this position paper is to argue that the currently dominant vision of AGI in the AI and machine learning (AI/ML) community needs to evolve, and that expectations and metrics for its safety must be informed much more by our understanding of the only existing instance of general intelligence, i.e., the intelligence found in animals, and especially in humans. This change in perspective will lead to a more realistic view of the technology, and allow for better policy decisions.

Related papers

• Alignment, Agency and Autonomy in Frontier AI: A Systems Engineering Perspective [0.0]
  Concepts of alignment, agency, and autonomy have become central to AI safety, governance, and control.<n>This paper traces the historical, philosophical, and technical evolution of these concepts, emphasizing how their definitions influence AI development, deployment, and oversight.
  arXiv  Detail & Related papers  (2025-02-20T21:37:20Z)
• Why do Experts Disagree on Existential Risk and P(doom)? A Survey of AI Experts [0.0]
  Research on catastrophic risks and AI alignment is often met with skepticism by experts.<n>Online debate over the existential risk of AI has begun to turn tribal.<n>I surveyed 111 AI experts on their familiarity with AI safety concepts.
  arXiv  Detail & Related papers  (2025-01-25T01:51:29Z)
• Aligning Generalisation Between Humans and Machines [74.120848518198]
  AI technology can support humans in scientific discovery and forming decisions, but may also disrupt democracies and target individuals.<n>The responsible use of AI and its participation in human-AI teams increasingly shows the need for AI alignment.<n>A crucial yet often overlooked aspect of these interactions is the different ways in which humans and machines generalise.
  arXiv  Detail & Related papers  (2024-11-23T18:36:07Z)
• Imagining and building wise machines: The centrality of AI metacognition [78.76893632793497]
  We examine what is known about human wisdom and sketch a vision of its AI counterpart.<n>We argue that AI systems particularly struggle with metacognition.<n>We discuss how wise AI might be benchmarked, trained, and implemented.
  arXiv  Detail & Related papers  (2024-11-04T18:10:10Z)
• Position Paper: Agent AI Towards a Holistic Intelligence [53.35971598180146]
  We emphasize developing Agent AI -- an embodied system that integrates large foundation models into agent actions. In this paper, we propose a novel large action model to achieve embodied intelligent behavior, the Agent Foundation Model.
  arXiv  Detail & Related papers  (2024-02-28T16:09:56Z)
• Killer Apps: Low-Speed, Large-Scale AI Weapons [2.2899177316144943]
  Artificial Intelligence (AI) and Machine Learning (ML) advancements present new challenges and opportunities in warfare and security. This paper explores the concept of AI weapons, their deployment, detection, and potential countermeasures.
  arXiv  Detail & Related papers  (2024-01-14T12:09:40Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• Examining the Differential Risk from High-level Artificial Intelligence and the Question of Control [0.0]
  The extent and scope of future AI capabilities remain a key uncertainty. There are concerns over the extent of integration and oversight of AI opaque decision processes. This study presents a hierarchical complex systems framework to model AI risk and provide a template for alternative futures analysis.
  arXiv  Detail & Related papers  (2022-11-06T15:46:02Z)
• Trustworthy AI: A Computational Perspective [54.80482955088197]
  We focus on six of the most crucial dimensions in achieving trustworthy AI: (i) Safety & Robustness, (ii) Non-discrimination & Fairness, (iii) Explainability, (iv) Privacy, (v) Accountability & Auditability, and (vi) Environmental Well-Being. For each dimension, we review the recent related technologies according to a taxonomy and summarize their applications in real-world systems.
  arXiv  Detail & Related papers  (2021-07-12T14:21:46Z)
• Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
  In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society. A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests. This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
  arXiv  Detail & Related papers  (2021-06-25T22:31:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.