Design Principles for Human-Agent Interaction

• URL: http://arxiv.org/abs/2606.20630v1
• Date: Thu, 28 May 2026 21:49:15 GMT
• Title: Design Principles for Human-Agent Interaction
• Abstract summary: We argue that AI agents should not be solely evaluated or deployed based on autonomous task capability alone.<n>We present 14 design principles that articulate the ideal human-agent relationship across four interaction stages.
• Score: 20.971374130080907
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI agents are rapidly evolving into autonomous systems capable of sustained interaction, tool use, and long-term collaboration. Yet their real-world adoption remains limited, suggesting that the key barrier lies not only in technical capability but also in a lack of design knowledge for successful human-agent interaction. This position paper argues that AI agents should not be solely evaluated or deployed based on autonomous task capability alone; because agents interact with, adapt to, influence, and sometimes fail humans, human-agent interaction must be treated as a core design and evaluation target for agentic AI. We present 14 design principles that articulate the ideal human-agent relationship across four interaction stages: initially, during interaction, over time, and when things go wrong. We use these principles to evaluate nine agent systems to illustrate that these design principles can provide actionable guidance for AI design teams to systematically design and evaluate agents that are usable, trustworthy, and effective in real-world interactive settings.

Related papers

• Human-AI Agent Interaction in a Business Context [0.0]
  This study identifies and evaluates principles and criteria for a positive User Experience (UX) with AI agents.<n>Using a mixed-methods approach, we explore interaction patterns between humans and AI agents.
  arXiv  Detail & Related papers  (2026-06-17T05:55:07Z)
• Cooperation in Human and Machine Agents: Promise Theory Considerations [0.0]
  A Promise Theory perspective on cooperation, in systems of human-machine agents, offers a unified perspective on organization and functional design.<n>I revisit established principles of agent cooperation, as applied to humans, machines, and their mutual interactions.
  arXiv  Detail & Related papers  (2026-04-12T07:48:31Z)
• Modeling Distinct Human Interaction in Web Agents [59.600507469754575]
  We introduce the task of modeling human intervention to support collaborative web task execution.<n>We identify four distinct patterns of user interaction with agents -- hands-off supervision, hands-on oversight, collaborative task-solving, and full user takeover.<n>We deploy these intervention-aware models in live web navigation agents and evaluate them in a user study, finding a 26.5% increase in user-rated agent usefulness.
  arXiv  Detail & Related papers  (2026-02-19T18:11:28Z)
• A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems [53.37728204835912]
  Most existing AI systems rely on manually crafted configurations that remain static after deployment.<n>Recent research has explored agent evolution techniques that aim to automatically enhance agent systems based on interaction data and environmental feedback.<n>This survey aims to provide researchers and practitioners with a systematic understanding of self-evolving AI agents.
  arXiv  Detail & Related papers  (2025-08-10T16:07:32Z)
• Agentic Web: Weaving the Next Web with AI Agents [109.13815627467514]
  The emergence of AI agents powered by large language models (LLMs) marks a pivotal shift toward the Agentic Web.<n>In this paradigm, agents interact directly with one another to plan, coordinate, and execute complex tasks on behalf of users.<n>We present a structured framework for understanding and building the Agentic Web.
  arXiv  Detail & Related papers  (2025-07-28T17:58:12Z)
• ADEPTS: A Capability Framework for Human-Centered Agent Design [16.17708866196561]
  We introduce ADEPTS, a capability framework defining a set of core user-facing capabilities for AI agents.<n>We aim to condense complex AI-UX requirements into a compact framework that is actionable guidance for AI researchers, designers, engineers, and policy reviewers alike.
  arXiv  Detail & Related papers  (2025-07-18T22:27:40Z)
• AI Agent Behavioral Science [29.262537008412412]
  AI Agent Behavioral Science focuses on the systematic observation of behavior, design of interventions to test hypotheses, and theory-guided interpretation of how AI agents act, adapt, and interact over time.<n>We systematize a growing body of research across individual agent, multi-agent, and human-agent interaction settings, and demonstrate how this perspective informs responsible AI by treating fairness, safety, interpretability, accountability, and privacy as behavioral properties.
  arXiv  Detail & Related papers  (2025-06-04T08:12:32Z)
• Internet of Agents: Fundamentals, Applications, and Challenges [68.9543153075464]
  We introduce the Internet of Agents (IoA) as a foundational framework that enables seamless interconnection, dynamic discovery, and collaborative orchestration among heterogeneous agents at scale.<n>We analyze the key operational enablers of IoA, including capability notification and discovery, adaptive communication protocols, dynamic task matching, consensus and conflict-resolution mechanisms, and incentive models.
  arXiv  Detail & Related papers  (2025-05-12T02:04:37Z)
• Autonomous Open-Ended Learning of Tasks with Non-Stationary Interdependencies [64.0476282000118]
  Intrinsic motivations have proven to generate a task-agnostic signal to properly allocate the training time amongst goals. While the majority of works in the field of intrinsically motivated open-ended learning focus on scenarios where goals are independent from each other, only few of them studied the autonomous acquisition of interdependent tasks. In particular, we first deepen the analysis of a previous system, showing the importance of incorporating information about the relationships between tasks at a higher level of the architecture. Then we introduce H-GRAIL, a new system that extends the previous one by adding a new learning layer to store the autonomously acquired sequences
  arXiv  Detail & Related papers  (2022-05-16T10:43:01Z)
• A Cognitive Framework for Delegation Between Error-Prone AI and Human Agents [0.0]
  We investigate the use of cognitively inspired models of behavior to predict the behavior of both human and AI agents. The predicted behavior is used to delegate control between humans and AI agents through the use of an intermediary entity.
  arXiv  Detail & Related papers  (2022-04-06T15:15:21Z)
• Building Affordance Relations for Robotic Agents - A Review [7.50722199393581]
  Affordances describe the possibilities for an agent to perform actions with an object. We review and find common ground amongst different strategies that use the concept of affordances within robotic tasks. We identify and discuss a range of interesting research directions involving affordances that have the potential to improve the capabilities of an AI agent.
  arXiv  Detail & Related papers  (2021-05-14T08:35:18Z)
• Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration [116.28433607265573]
  We introduce Watch-And-Help (WAH), a challenge for testing social intelligence in AI agents. In WAH, an AI agent needs to help a human-like agent perform a complex household task efficiently. We build VirtualHome-Social, a multi-agent household environment, and provide a benchmark including both planning and learning based baselines.
  arXiv  Detail & Related papers  (2020-10-19T21:48:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.