Knowing When to Ask: Self-Gated Clarification for Hierarchical Language Agents

• URL: http://arxiv.org/abs/2606.11349v2
• Date: Fri, 12 Jun 2026 20:21:42 GMT
• Title: Knowing When to Ask: Self-Gated Clarification for Hierarchical Language Agents
• Abstract summary: In hierarchical reasoning, failures often originate at intermediate decision points where the agent commits to a wrong branch.<n>We propose ACTION-RATING, a formulation that places clarification inside the agent's action space on a shared ordinal scale with navigation.<n>Two structurally distinct information-seeking modes emerge from the agent's own ratings: mandatory (no viable branch) and opportunistic (residual uncertainty despite a leading candidate)
• Score: 4.309840398782996
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In hierarchical reasoning, failures often originate at intermediate decision points where the agent commits to a wrong branch without recognizing that it lacks critical information. Rather than treating clarification as an external uncertainty trigger, we propose ACTION-RATING, a formulation that places it inside the agent's action space on a shared ordinal scale with navigation, so that asking competes directly with acting at every decision point and help-seeking becomes observable at intermediate states. Two structurally distinct information-seeking modes emerge from the agent's own ratings: mandatory (no viable branch) and opportunistic (residual uncertainty despite a leading candidate). On Harmonized Tariff Schedule classification (30,000-node taxonomy, three benchmarks, 9~LLMs across 4 families), we observe a regime shift from mandatory to opportunistic clarification, with Information-Seeking Effectiveness (ISE), a local diagnostic defined as the fraction of help interactions followed by a correct next navigation step (not a final-task metric), rising from 50% to 74%. Three diagnostic contrasts fail to reproduce this structure. A separability test shows that the information-seeking pattern (mode split, ISE ranking) persists when answer quality is degraded (-18.8% accuracy), supporting an empirical separation between where an agent seeks help and the quality of the help it receives. Under the controlled answer channel, accuracy gains reach +16.2% at 10-digit; we read this as an upper bound on what better localization could unlock, not a deployment estimate.

Related papers

• Do Large Language Models Know What They Don't Know II? A Fully Behavioral, Non-Cognitive Measure of Epistemic Honesty [8.148357447770893]
  Large Language Models (LLMs) are frequently confident, eloquent, and well versed.<n>Do they know what they don't know?<n>We develop a novel metric to evaluate whether an LLM appropriately acknowledges the boundaries of its knowledge.
  arXiv  Detail & Related papers  (2026-09-07T18:40:08Z)
• Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learning [63.24624171679711]
  Large language model (LLM)-based agents often make suboptimal tool-use decisions.<n>We propose TRUST, which incorporates uncertainty quantification into reward design as a repulsive force for maintaining uncertainty separation.
  arXiv  Detail & Related papers  (2026-06-05T07:08:34Z)
• SAAG: Structured Agent Assessment and Grounding [13.62148061055744]
  We propose a cascaded diagnostic framework that decomposes agent-calling evaluation into three sequential stages.<n>We evaluate this framework on a benchmark derived from Glaive's function-calling dataset across registry sizes of 5, 10, and 15 agents.
  arXiv  Detail & Related papers  (2026-04-30T19:33:58Z)
• Don't Start What You Can't Finish: A Counterfactual Audit of Support-State Triage in LLM Agents [0.0]
  We introduce the Support-State Triage Audit (SSTA-32), a matched-item diagnostic framework in which minimal counterfactual edits flip the same base request across four support states.<n>We evaluate a frontier model under four prompting conditions using Dual-Persona Auto-Auditing (DPAA) with deterministic scoring.
  arXiv  Detail & Related papers  (2026-04-17T23:54:34Z)
• HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help? [32.54022440678003]
  coding agents solve complex tasks when given complete context but collapse when specifications are incomplete or ambiguous.<n>Current benchmarks are blind to this failure mode.<n>We present HiL-Bench to measure this selective escalation skill.
  arXiv  Detail & Related papers  (2026-04-10T15:21:44Z)
• The Confidence Gate Theorem: When Should Ranked Decision Systems Abstain? [0.0]
  Ranked decision systems must decide when to intervene in ranked outputs and when to abstain.<n>We study when confidence-based abstention monotonically improves decision quality, and when it fails.
  arXiv  Detail & Related papers  (2026-03-10T17:44:10Z)
• Guideline-Grounded Evidence Accumulation for High-Stakes Agent Verification [60.18369393468405]
  Existing verifiers usually underperform owing to a lack of domain knowledge and limited calibration.<n>GLEAN compiles expert-curated protocols into trajectory-informed, well-calibrated correctness signals.<n>We empirically validate GLEAN with agentic clinical diagnosis across three diseases from the MIMIC-IV dataset.
  arXiv  Detail & Related papers  (2026-03-03T09:36:43Z)
• Decomposed Prompting Does Not Fix Knowledge Gaps, But Helps Models Say "I Don't Know" [47.930782177987446]
  Large language models often struggle to recognize their knowledge limits in closed-book question answering, leading to confident hallucinations.<n>We evaluate three task-equivalent prompting regimes: Direct, Assistive, and Incremental, across different model scales and multi-hop QA benchmarks.<n>Because factual knowledge is stable while hallucinations are agreement, cross-regime provides a precise signal of internal uncertainty.
  arXiv  Detail & Related papers  (2026-02-04T18:39:58Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Online Decision Mediation [72.80902932543474]
  Consider learning a decision support assistant to serve as an intermediary between (oracle) expert behavior and (imperfect) human behavior. In clinical diagnosis, fully-autonomous machine behavior is often beyond ethical affordances.
  arXiv  Detail & Related papers  (2023-10-28T05:59:43Z)
• A New Bandit Setting Balancing Information from State Evolution and Corrupted Context [52.67844649650687]
  We propose a new sequential decision-making setting combining key aspects of two established online learning problems with bandit feedback. The optimal action to play at any given moment is contingent on an underlying changing state which is not directly observable by the agent. We present an algorithm that uses a referee to dynamically combine the policies of a contextual bandit and a multi-armed bandit.
  arXiv  Detail & Related papers  (2020-11-16T14:35:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.