Agentic Abstention: Do Agents Know When to Stop Instead of Act?

• URL: http://arxiv.org/abs/2606.28733v1
• Date: Sat, 27 Jun 2026 04:49:37 GMT
• Title: Agentic Abstention: Do Agents Know When to Stop Instead of Act?
• Abstract summary: LLM agents are expected to act over multiple turns, using search, browsing interfaces, and terminal tools to complete user goals.<n>We define Agentic Abstention, the problem of deciding when an agent should stop acting under uncertainty.<n>We study this problem across web shopping, terminal environments, and question answering, evaluating 13 LLM-as-agent systems and 2 agent scaffolds on more than 28,000 tasks.
• Score: 13.780119058961331
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM agents are expected to act over multiple turns, using search, browsing interfaces, and terminal tools to complete user goals. Yet not every goal is well specified or achievable in the available environment. In such cases, a reliable agent should recognize that further interaction is unlikely to help and abstain from additional tool calls. We define Agentic Abstention, the problem of deciding when an agent should stop acting under uncertainty. Unlike standard LLM abstention, which is usually evaluated as a single-turn answer-or-abstain decision, agentic abstention is a sequential decision problem: an agent can answer, abstain, or gather more information at each turn, and the need to abstain may only become clear after interacting with the environment. We study this problem across web shopping, terminal environments, and question answering, evaluating 13 LLM-as-agent systems and 2 agent scaffolds on more than 28,000 tasks. Our results show that the main challenge is not only whether agents can abstain, but also when they abstain. Some agents never abstain when they should, while others do so only after many unnecessary interactions. This gap is especially large on tasks where the instruction appears feasible until the environment reveals otherwise (e.g., no valid result matches the instruction). We further find that model scale, reasoning, and agent scaffolding affect abstention in different ways, where larger or more capable models sometimes perform worse at timely abstention. Finally, we introduce CONVOLVE, a context engineering method for improving agentic abstention that distills full interaction trajectories into reusable stopping rules. On WebShop, CONVOLVE substantially improves timely abstention without updating model parameters, raising Llama-3.3-70B's timely recall rate from 26.7 to 57.4. Our dataset and code are available at https://lhannnn.github.io/agentic-abstention

Related papers

• 2AM: Grounding Agent-Side Memory as Guidance for Steerable Action Models in Long-Horizon Manipulation [6.040186174770221]
  Long-horizon robot manipulation requires memory, but not necessarily inside the action policy.<n>2AM makes a multimodal Agent the sole holder of task memory and a single RGB-based, episodically stateless Action Model the sole executor of task-relevant motion.<n>On LIBERO-Mem, without depth, online geometry, or planner-based object motion, 2AM reaches 76.3% average completion, a 61.5-point improvement over the strongest reported baseline of 14.8%, together with 63.0% relaxed and 11.8% strict success.
  arXiv  Detail & Related papers  (2026-09-10T09:35:56Z)
• From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents [56.187590188698955]
  Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful.<n>Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles.<n>We introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories.<n>We propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment.
  arXiv  Detail & Related papers  (2026-08-03T12:52:04Z)
• AgentAbstain: Do LLM Agents Know When Not to Act? [19.223843408018997]
  Existing evaluations mostly focus on task success rather than whether agents know when to abstain.<n>We present the first systematic evaluation framework for agentic abstention: the calibrated ability of tool-using LLM agents to recognize when not to act.<n>AgentAbstain is a paired-task benchmark built on an agent-native taxonomy of 8 abstention scenarios across pre-execution reasoning and runtime discovery.
  arXiv  Detail & Related papers  (2026-07-11T00:57:36Z)
• Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity? [54.701199583505144]
  Cross-application access creates a privacy risk that has been largely overlooked.<n>We introduce AgentCIBench, an evaluation harness that turns this risk into executable, deterministically scored scenarios.<n>We evaluate 15 frontier agents and find a surprisingly high failure rate.
  arXiv  Detail & Related papers  (2026-06-22T11:36:58Z)
• Misinformation Propagation in Benign Multi-Agent Systems [11.967259729189388]
  Multi-agent systems are increasingly deployed in high-stakes settings such as medical diagnosis, legal analysis, and forensic decision-making.<n>Their reliability can be at risk when single agents reason from incorrect or misleading context.<n>This work studies this risk by injecting intent-based misinformation into benign single-agent and multi-agent systems.
  arXiv  Detail & Related papers  (2026-06-15T13:40:01Z)
• Ask Now, Use Later: Benchmarking the Proactivity Gap in Long-Lived LLM Agents [18.132295539880896]
  A long-lived LLM agent earns its value by acting on a user's preferences across sessions, not just the current request.<n>Yet today's agents keep what a user volunteers but rarely ask for what stays unspoken.<n>As users delegate more of their affairs to agents, the impact of this gap grows.<n>ATRBench makes it measurable by fixing each user's preferences as hidden ground truth.
  arXiv  Detail & Related papers  (2026-05-27T08:00:58Z)
• Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems [60.79651380135334]
  Long-lived AI agents are increasingly deployed as persistent operational systems.<n>How long does an agent remain reliable after deployment?<n>We introduce AgingBench, a longitudinal reliability benchmark for agent lifespan engineering.
  arXiv  Detail & Related papers  (2026-05-25T19:55:12Z)
• Persuasion Propagation in LLM Agents [23.64887423923855]
  We study how belief-level intervention can influence downstream task behavior.<n>Across web research and coding tasks, we find that on-the-fly persuasion induces weak and inconsistent behavioral effects.<n>When the belief state is explicitly specified at task time, belief-prefilled agents conduct on average 26.9% fewer searches and visit 16.9% fewer unique sources than neutral-prefilled agents.
  arXiv  Detail & Related papers  (2026-01-31T18:33:14Z)
• Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents [58.00130492861884]
  TraitBasis is a lightweight, model-agnostic method for systematically stress testing AI agents.<n>TraitBasis learns directions in activation space corresponding to steerable user traits.<n>We observe on average a 2%-30% performance degradation on $tau$-Trait across frontier models.
  arXiv  Detail & Related papers  (2025-10-06T05:03:57Z)
• Agent-Based Detection and Resolution of Incompleteness and Ambiguity in Interactions with Large Language Models [0.9856777842758593]
  This paper examines the use of agent-based architecture to bolster LLM-based Question-Answering systems with additional reasoning capabilities.<n>We equip different LLMs with agents that act as specialists in detecting and resolving deficiencies of incompleteness and ambiguity.<n>Suggesting the agent-based approach could be a useful mechanism to harness the power of LLMs to develop more robust QA systems.
  arXiv  Detail & Related papers  (2025-07-04T17:28:33Z)
• Dissecting Adversarial Robustness of Multimodal LM Agents [70.2077308846307]
  We manually create 200 targeted adversarial tasks and evaluation scripts in a realistic threat model on top of VisualWebArena.<n>We find that we can successfully break latest agents that use black-box frontier LMs, including those that perform reflection and tree search.<n>We also use ARE to rigorously evaluate how the robustness changes as new components are added.
  arXiv  Detail & Related papers  (2024-06-18T17:32:48Z)
• On the Use and Misuse of Absorbing States in Multi-agent Reinforcement Learning [55.95253619768565]
  Current MARL algorithms assume that the number of agents within a group remains fixed throughout an experiment. In many practical problems, an agent may terminate before their teammates. We present a novel architecture for an existing state-of-the-art MARL algorithm which uses attention instead of a fully connected layer with absorbing states.
  arXiv  Detail & Related papers  (2021-11-10T23:45:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.