Artificial Intelligent Disobedience: Rethinking the Agency of Our Artificial Teammates

• URL: http://arxiv.org/abs/2506.22276v1
• Date: Fri, 27 Jun 2025 14:45:27 GMT
• Title: Artificial Intelligent Disobedience: Rethinking the Agency of Our Artificial Teammates
• Authors: Reuth Mirsky,
• Abstract summary: This paper argues for expanding the agency of AI teammates to include intelligent disobedience.<n>It introduces a scale of AI agency levels and uses representative examples to highlight the importance of treating AI autonomy as an independent research focus.
• Score: 3.7692411550925677
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial intelligence has made remarkable strides in recent years, achieving superhuman performance across a wide range of tasks. Yet despite these advances, most cooperative AI systems remain rigidly obedient, designed to follow human instructions without question and conform to user expectations, even when doing so may be counterproductive or unsafe. This paper argues for expanding the agency of AI teammates to include \textit{intelligent disobedience}, empowering them to make meaningful and autonomous contributions within human-AI teams. It introduces a scale of AI agency levels and uses representative examples to highlight the importance and growing necessity of treating AI autonomy as an independent research focus in cooperative settings. The paper then explores how intelligent disobedience manifests across different autonomy levels and concludes by proposing initial boundaries and considerations for studying disobedience as a core capability of artificial agents.

Related papers

• The Ultimate Test of Superintelligent AI Agents: Can an AI Balance Care and Control in Asymmetric Relationships? [11.29688025465972]
  The Shepherd Test is a new conceptual test for assessing the moral and relational dimensions of superintelligent artificial agents.<n>We argue that AI crosses an important, and potentially dangerous, threshold of intelligence when it exhibits the ability to manipulate, nurture, and instrumentally use less intelligent agents.<n>This includes the ability to weigh moral trade-offs between self-interest and the well-being of subordinate agents.
  arXiv  Detail & Related papers  (2025-06-02T15:53:56Z)
• A Task-Driven Human-AI Collaboration: When to Automate, When to Collaborate, When to Challenge [16.734679201317896]
  We show how proper human-AI integration maintains meaningful agency while improving performance.<n>This framework lays the foundation for practically effective and morally sound human-AI collaboration.
  arXiv  Detail & Related papers  (2025-05-23T23:19:15Z)
• The Philosophic Turn for AI Agents: Replacing centralized digital rhetoric with decentralized truth-seeking [0.0]
  In the face of AI technology, individuals will increasingly rely on AI agents to navigate life's growing complexities.<n>This paper addresses a fundamental dilemma posed by AI decision-support systems: the risk of either becoming overwhelmed by complex decisions, or having autonomy compromised.
  arXiv  Detail & Related papers  (2025-04-24T19:34:43Z)
• A Basis for Human Responsibility in Artificial Intelligence Computation [0.0]
  Recent advancements in artificial intelligence have reopened the question about the boundaries of AI autonomy.<n>This paper explores these boundaries through the analysis of the Alignment Research Center experiment on GPT-4.<n>By examining the thought experiment and its counterarguments will be enlightened how in the need for human activation and purpose definition lies the AI's inherent dependency on human-initiated actions.
  arXiv  Detail & Related papers  (2025-01-21T20:59:48Z)
• Aligning Generalisation Between Humans and Machines [74.120848518198]
  AI technology can support humans in scientific discovery and forming decisions, but may also disrupt democracies and target individuals.<n>The responsible use of AI and its participation in human-AI teams increasingly shows the need for AI alignment.<n>A crucial yet often overlooked aspect of these interactions is the different ways in which humans and machines generalise.
  arXiv  Detail & Related papers  (2024-11-23T18:36:07Z)
• Imagining and building wise machines: The centrality of AI metacognition [78.76893632793497]
  We examine what is known about human wisdom and sketch a vision of its AI counterpart.<n>We argue that AI systems particularly struggle with metacognition.<n>We discuss how wise AI might be benchmarked, trained, and implemented.
  arXiv  Detail & Related papers  (2024-11-04T18:10:10Z)
• Position Paper: Agent AI Towards a Holistic Intelligence [53.35971598180146]
  We emphasize developing Agent AI -- an embodied system that integrates large foundation models into agent actions. In this paper, we propose a novel large action model to achieve embodied intelligent behavior, the Agent Foundation Model.
  arXiv  Detail & Related papers  (2024-02-28T16:09:56Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• Exploration with Principles for Diverse AI Supervision [88.61687950039662]
  Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI. While this generative AI approach has produced impressive results, it heavily leans on human supervision. This strong reliance on human oversight poses a significant hurdle to the advancement of AI innovation. We propose a novel paradigm termed Exploratory AI (EAI) aimed at autonomously generating high-quality training data.
  arXiv  Detail & Related papers  (2023-10-13T07:03:39Z)
• The Rise and Potential of Large Language Model Based Agents: A Survey [91.71061158000953]
  Large language models (LLMs) are regarded as potential sparks for Artificial General Intelligence (AGI) We start by tracing the concept of agents from its philosophical origins to its development in AI, and explain why LLMs are suitable foundations for agents. We explore the extensive applications of LLM-based agents in three aspects: single-agent scenarios, multi-agent scenarios, and human-agent cooperation.
  arXiv  Detail & Related papers  (2023-09-14T17:12:03Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• AI and the Sense of Self [0.0]
  We focus on the cognitive sense of "self" and its role in autonomous decision-making leading to responsible behaviour. Authors hope to make a case for greater research interest in building richer computational models of AI agents with a sense of self.
  arXiv  Detail & Related papers  (2022-01-07T10:54:06Z)
• Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
  In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society. A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests. This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
  arXiv  Detail & Related papers  (2021-06-25T22:31:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.