You Shall Not Pass! Where and Why Developers Draw The Line on AI Autonomy

• URL: http://arxiv.org/abs/2607.00533v1
• Date: Wed, 01 Jul 2026 07:24:14 GMT
• Title: You Shall Not Pass! Where and Why Developers Draw The Line on AI Autonomy
• Abstract summary: Most developers accepted AI producing work under their oversight, although accepted autonomy varied substantively across tasks and individuals.<n> Acceptance was lowest for identity-defining, human-facing, and design-oriented work, and higher among developers with more AI experience and risk tolerance.<n>Our findings suggest that preferences for AI autonomy reflect how developers cognitively experience their work.
• Score: 7.477031451733673
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As AI takes on more software work, the line between human and AI effort is shifting. Where developers draw that line around AI autonomy bears on how we design tools and roles that preserve meaningful work. Drawing on cognitive appraisal theory, work design, and automation research, we conducted a mixed-methods study of 448 professional developers at Microsoft to investigate their accepted levels of AI autonomy across software engineering work. Most developers accepted AI producing work under their oversight, although accepted autonomy varied substantively across tasks and individuals. Acceptance was lowest for identity-defining, human-facing, and design-oriented work, and higher among developers with more AI experience and risk tolerance. Task accountability was associated with lower odds of allowing AI to act on developers' behalf, whereas task identity was associated with lower odds of granting AI decision-making autonomy. Task demands had the opposite effect, increasing willingness to delegate decision-making to AI. Our findings suggest that preferences for AI autonomy reflect how developers cognitively experience their work, highlighting important considerations for designing meaningful work.

Related papers

• The Boundaries of Automation: A Theory of Persistent Human Participation [107.84677954791773]
  We argue that human participation may persist even with highly capable AI systems.<n>In some activities, the target is not fully specified in advance but instead emerges through the interaction itself.<n>This perspective has important implications for the limits of automation.
  arXiv  Detail & Related papers  (2026-07-23T17:33:43Z)
• How Do Developers Interact with AI? An Exploratory Study on Modeling Developer Programming Behavior [8.956327421839205]
  We show how hidden intentions and emotional dimensions intertwine with concrete actions during AI-assisted programming.<n>Using AI assistants often makes developers more focused on actively creating code, evaluating, and verifying generated results.<n>Some developers reported impostor-like feelings, expressing guilt or self-doubt about relying on AI.
  arXiv  Detail & Related papers  (2026-03-28T14:37:19Z)
• AI Where It Matters: Where, Why, and How Developers Want AI Support in Daily Work [3.1984206905916]
  Generative AI is reshaping software work, yet we lack clear guidance on where developers most need and want support.<n>We report a large-scale, mixed-methods study of N=860 developers that examines where, why, and how they seek or limit AI help.<n>Our results offer concrete, contextual guidance for delivering AI where it matters to developers and their work.
  arXiv  Detail & Related papers  (2025-10-01T10:51:03Z)
• Artificial Intelligent Disobedience: Rethinking the Agency of Our Artificial Teammates [3.7692411550925677]
  This paper argues for expanding the agency of AI teammates to include intelligent disobedience.<n>It introduces a scale of AI agency levels and uses representative examples to highlight the importance of treating AI autonomy as an independent research focus.
  arXiv  Detail & Related papers  (2025-06-27T14:45:27Z)
• From Teacher to Colleague: How Coding Experience Shapes Developer Perceptions of AI Tools [0.0]
  AI-assisted development tools promise productivity gains and improved code quality, yet their adoption among developers remains inconsistent.<n>We analyze survey data from 3380 developers to examine how coding experience relates to AI awareness, adoption, and the roles developers assign to AI in their workflow.
  arXiv  Detail & Related papers  (2025-04-08T08:58:06Z)
• Imagining and building wise machines: The centrality of AI metacognition [78.76893632793497]
  We examine what is known about human wisdom and sketch a vision of its AI counterpart.<n>We argue that AI systems particularly struggle with metacognition.<n>We discuss how wise AI might be benchmarked, trained, and implemented.
  arXiv  Detail & Related papers  (2024-11-04T18:10:10Z)
• Engineering Trustworthy AI: A Developer Guide for Empirical Risk Minimization [53.80919781981027]
  Key requirements for trustworthy AI can be translated into design choices for the components of empirical risk minimization. We hope to provide actionable guidance for building AI systems that meet emerging standards for trustworthiness of AI.
  arXiv  Detail & Related papers  (2024-10-25T07:53:32Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• AI and the Sense of Self [0.0]
  We focus on the cognitive sense of "self" and its role in autonomous decision-making leading to responsible behaviour. Authors hope to make a case for greater research interest in building richer computational models of AI agents with a sense of self.
  arXiv  Detail & Related papers  (2022-01-07T10:54:06Z)
• Trustworthy AI: A Computational Perspective [54.80482955088197]
  We focus on six of the most crucial dimensions in achieving trustworthy AI: (i) Safety & Robustness, (ii) Non-discrimination & Fairness, (iii) Explainability, (iv) Privacy, (v) Accountability & Auditability, and (vi) Environmental Well-Being. For each dimension, we review the recent related technologies according to a taxonomy and summarize their applications in real-world systems.
  arXiv  Detail & Related papers  (2021-07-12T14:21:46Z)
• Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
  In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society. A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests. This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
  arXiv  Detail & Related papers  (2021-06-25T22:31:55Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.