Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent

• URL: http://arxiv.org/abs/2607.18257v1
• Date: Thu, 14 May 2026 21:42:56 GMT
• Title: Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
• Abstract summary: We show that users face a problem when they decide what to delegate to a system whose action space they cannot fully anticipate.<n>We call the resulting dissatisfaction delegation regret, a pattern in which users regret not that the agent erred, but that it acted beyond what they would have authorized.<n>We discuss implications for agent designs that expose action boundaries, support per-task autonomy policies, and separate advisory output from agentic execution.
• Score: 10.633573668629486
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When AI agents shift from answering questions to taking actions, users face a new problem: deciding what to delegate, to a system whose action space they cannot fully anticipate. We call the resulting dissatisfaction delegation regret, a pattern in which users regret not that the agent erred, but that it acted beyond what they would have authorized. In a controlled study, 20 university students completed five common daily tasks using OpenClaw, a general-purpose AI agent, across tasks chosen to vary in privacy, stakes, and reversibility. For each task we measured trust, perceived control, transparency, supervision burden, and approval preference on 5-point Likert scales, and collected free-text reflections analyzed through thematic coding. Three findings emerged. First, participants calibrated trust per task rather than per agent: they granted wide autonomy for advisory and low-stakes tasks but demanded confirmation for irreversible, externally visible actions. Second, irreversibility combined with external visibility, rather than stakes alone, appeared to drive trust withdrawal: the moderate-stakes email task triggered the sharpest drop in trust (M = 3.10) and the highest demand for approval (M = 4.65), whereas a high-stakes but verifiable task did not produce the same response. Third, delegation regret appeared consistently when the agent executed actions without preview, even when the output was rated as successful. We discuss implications for agent designs that expose action boundaries, support per-task autonomy policies, and separate advisory output from agentic execution.

Related papers

• AgentAbstain: Do LLM Agents Know When Not to Act? [19.223843408018997]
  Existing evaluations mostly focus on task success rather than whether agents know when to abstain.<n>We present the first systematic evaluation framework for agentic abstention: the calibrated ability of tool-using LLM agents to recognize when not to act.<n>AgentAbstain is a paired-task benchmark built on an agent-native taxonomy of 8 abstention scenarios across pre-execution reasoning and runtime discovery.
  arXiv  Detail & Related papers  (2026-07-11T00:57:36Z)
• Governed Individuation: Cryptographically Decoupling an Agent's Learning from Its Authority [7.392721604463545]
  We show that confinement can be guaranteed as an invariant of the agent's execution architecture.<n>We prove that no amount of learning, skill acquisition, or self-induced governance abstraction can widen the agent's permitted authority.
  arXiv  Detail & Related papers  (2026-07-06T02:42:06Z)
• Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity? [54.701199583505144]
  Cross-application access creates a privacy risk that has been largely overlooked.<n>We introduce AgentCIBench, an evaluation harness that turns this risk into executable, deterministically scored scenarios.<n>We evaluate 15 frontier agents and find a surprisingly high failure rate.
  arXiv  Detail & Related papers  (2026-06-22T11:36:58Z)
• ProcAgent: An Agentic Framework for Procedural Task Guidance on Edge with Human-in-the-Loop [1.4279471205248535]
  ProcAgent is a vision-based procedural assistant for real-time adaptive guidances on a single NVIDIA Jetson AGX Orin.<n>We evaluate ProcAgent along four dimensions: perception accuracy, reasoning, task-level performance, and user experience.
  arXiv  Detail & Related papers  (2026-06-09T17:56:28Z)
• Auditing and Controlling AI Agent Actions in Spreadsheets [9.249091427192786]
  AI agents can perform sophisticated, multi-step knowledge work autonomously from start to finish.<n>By the time users receive the output, all underlying decisions have already been made without their involvement.<n>We introduce Pista, a spreadsheet AI agent that decomposes execution into auditable, controllable actions.
  arXiv  Detail & Related papers  (2026-04-22T00:32:35Z)
• AgentV-RL: Scaling Reward Modeling with Agentic Verifier [63.55502685076245]
  Verifiers have been demonstrated to enhance LLM reasoning via test-time scaling (TTS)<n>We propose Agentic Verifier, a framework that transforms reward modeling into a multi-turn, tool-augmented deliberative process.<n>We show that Agentic Verifier yields consistent performance gains under both parallel and sequential TTS.
  arXiv  Detail & Related papers  (2026-04-17T12:27:36Z)
• SmartSnap: Proactive Evidence Seeking for Self-Verifying Agents [45.71333459905404]
  SmartSnap is a paradigm shift from passive, post-hoc verification to proactive, in-situ self-verification by the agent itself.<n>We introduce the Self-Verifying Agent, a new type of agent designed with dual missions: to complete a task and to prove its accomplishment with curated evidences.<n>Experiments on mobile tasks across model families and scales demonstrate that our SmartSnap paradigm allows training LLM-driven agents in a scalable manner.
  arXiv  Detail & Related papers  (2025-12-26T14:51:39Z)
• Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberation [42.38513187601995]
  Large Language Models (LLMs) trained with reinforcement learning and verifiable rewards have achieved strong results on complex reasoning tasks.<n>Recent work extends this paradigm to a multi-agent setting, where a meta-thinking agent proposes plans and monitors progress while a reasoning agent executes subtasks through sequential conversational turns.<n>Despite promising performance, we identify a critical limitation: lazy agent behavior, in which one agent dominates while the other contributes little, undermining collaboration and collapsing the setup to an ineffective single agent.<n>We propose a verifiable reward mechanism that encourages deliberation by allowing the reasoning agent to discard noisy outputs, consolidate instructions, and restart its reasoning process
  arXiv  Detail & Related papers  (2025-11-04T06:37:31Z)
• Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight [51.53020962098759]
  This study examines how agents, human participants, and human-AI teams respond to 16 types of dark patterns across diverse scenarios.<n>Phase 1 highlights that agents often fail to recognize dark patterns, and even when aware, prioritize task completion over protective action.<n>Phase 2 revealed divergent failure modes: humans succumb due to cognitive shortcuts and habitual compliance, while agents falter from procedural blind spots.
  arXiv  Detail & Related papers  (2025-09-12T22:26:31Z)
• SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints [59.645885492637845]
  SOPBench is an evaluation pipeline that transforms each service-specific SOP code program into a directed graph of executable functions.<n>Our approach transforms each service-specific SOP code program into a directed graph of executable functions and requires agents to call these functions based on natural language SOP descriptions.<n>We evaluate 18 leading models, and results show the task is challenging even for top-tier models.
  arXiv  Detail & Related papers  (2025-03-11T17:53:02Z)
• Criticality and Safety Margins for Reinforcement Learning [53.10194953873209]
  We seek to define a criticality framework with both a quantifiable ground truth and a clear significance to users.<n>We introduce true criticality as the expected drop in reward when an agent deviates from its policy for n consecutive random actions.<n>We also introduce the concept of proxy criticality, a low-overhead metric that has a statistically monotonic relationship to true criticality.
  arXiv  Detail & Related papers  (2024-09-26T21:00:45Z)
• Formalizing the Problem of Side Effect Regularization [81.97441214404247]
  We propose a formal criterion for side effect regularization via the assistance game framework. In these games, the agent solves a partially observable Markov decision process. We show that this POMDP is solved by trading off the proxy reward with the agent's ability to achieve a range of future tasks.
  arXiv  Detail & Related papers  (2022-06-23T16:36:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.