Why2Speak: Faithful Reasoning for Abstaining Action Policies

• URL: http://arxiv.org/abs/2608.20670v1
• Date: Fri, 21 Aug 2026 02:00:21 GMT
• Title: Why2Speak: Faithful Reasoning for Abstaining Action Policies
• Abstract summary: We compare direct decision policies, reasoning policies, supervised fine-tuning, and reinforcement learning.<n>We find a capability-auditability tradeoff: the strongest direct policy achieves higher quality but exposes no reasoning to inspect.<n>Supervised fine-tuning either suppresses reasoning or preserves it without improving decision quality, while reinforcement learning also fails to improve the reasoning policy.
• Score: 0.19336815376402716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many agentic systems must repeatedly choose between acting and abstaining, making faithful reasoning important for oversight: an explanation is useful only if it reflects the computation that produced the action. We study this problem through intervention timing in multi-party conversation, where an assistant must decide whether to speak or remain silent. This setting exposes class imbalance, asymmetric action costs, and the possibility that exposing reasoning changes the policy being audited. Using Qwen3-8B, decoded with or without chain-of-thought reasoning, we compare direct decision policies, reasoning policies, supervised fine-tuning, and reinforcement learning. We find a capability-auditability tradeoff: the strongest direct policy achieves higher quality but exposes no reasoning to inspect, while the reasoning policy provides a trace at the cost of lower performance, particularly recall of true intervention opportunities. Supervised fine-tuning either suppresses reasoning or preserves it without improving decision quality, while reinforcement learning also fails to improve the reasoning policy. We identify one mechanism underlying this failure: group relative objectives provide no learning signal on confidently wrong prompts when sampled rollouts all select the same action. Controlled activation probes and behavioral ablations show that standard faithfulness methods can overstate evidence that exposed reasoning reflects the underlying decision process. Probability-based metrics saturate under confident decisions, probes are vulnerable to class imbalance and textual leakage, and reasoning ablations can confound reasoning content with changes in inference mode. Together, these results show that exposing reasoning can change an agent's action policy rather than simply make it observable. We provide controls for evaluating reasoning-based oversight of agents that can act or abstain.

Related papers

• Reconciling Process Supervision with Outcome-Based Credit in Agentic Policy Optimization [13.354353071601379]
  We introduce TASPO, which converts privileged supervision into outcome-grounded action credit.<n>Across three agentic benchmarks, TASPO improves over GRPO by 10.6% and generalizes better to unseen tasks.
  arXiv  Detail & Related papers  (2026-08-31T16:51:50Z)
• Do Vision-Language-Action Models Mean What They Say? On the Role of Faithfulness in Embodied Reasoning [17.56588495990139]
  Embodied Chain-of-Thought has emerged as a promising mechanism to enhance robot decision-making.<n>We argue that SoTA alignment strategies offer a necessary but insufficient notion of faithfulness.<n>We operationalize a behavioral surrogate for embodied faithfulness through a learned critic, Pinocchio.
  arXiv  Detail & Related papers  (2026-07-06T05:10:27Z)
• ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails [57.25411733152009]
  Reasoning-based guardrails improve safety moderation by generating explicit rationales before issuing final decisions.<n>A model may recognize a harmful intent in its reasoning but still predict a safe label, or issue an unsafe decision without policy-grounded justification.<n>We propose ConsisGuard, a consistency-aware framework for reasoning-based guardrails.
  arXiv  Detail & Related papers  (2026-05-29T09:42:08Z)
• When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making [68.12864562049957]
  Embodied robotic systems increasingly rely on large language model (LLM)-based agents to support high-level reasoning.<n>This raises a fundamental question for embodied agents: when should the agent reason, and when should it act?<n>We propose RARRL (Resource-Aware Reasoning via Reinforcement Learning), a hierarchical framework for resource-aware orchestration of embodied agents.
  arXiv  Detail & Related papers  (2026-03-17T15:38:50Z)
• Towards Generalizable Reasoning: Group Causal Counterfactual Policy Optimization for LLM Reasoning [50.352417879912515]
  Large language models (LLMs) excel at complex tasks with advances in reasoning capabilities.<n>We propose Group Causal Counterfactual Policy Optimization to explicitly train LLMs to learn generalizable reasoning patterns.<n>We then construct token-level advantages from this reward and optimize the policy, encouraging LLMs to favor reasoning patterns that are process-valid and counterfactually robust.
  arXiv  Detail & Related papers  (2026-02-06T08:03:11Z)
• From Sycophancy to Sensemaking: Premise Governance for Human-AI Decision Making [0.0]
  Low-friction assistants can become sycophantic, baking in implicit assumptions and pushing verification costs onto experts.<n>We argue reliable human-AI partnership requires a shift from answer generation to collaborative premise governance.
  arXiv  Detail & Related papers  (2026-02-02T17:42:54Z)
• Preventing the Collapse of Peer Review Requires Verification-First AI [49.995126139461085]
  We propose truth-coupling, i.e. how tightly venue scores track latent scientific truth.<n>We formalize two forces that drive a phase transition toward proxy-sovereign evaluation.
  arXiv  Detail & Related papers  (2026-01-23T17:17:32Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)
• Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations [60.27156500679296]
  We study the role of Chain-of-Thought (CoT) explanations in moral scenarios by systematically perturbing reasoning chains and manipulating delivery tones.<n>Our findings reveal two key effects: (1) users often trust with outcome agreement, sustaining reliance even when reasoning is flawed.<n>These results highlight how CoT explanations can simultaneously clarify and mislead, underscoring the need for NLP systems to provide explanations that encourage scrutiny and critical thinking rather than blind trust.
  arXiv  Detail & Related papers  (2025-11-15T02:38:49Z)
• Learning to Seek Evidence: A Verifiable Reasoning Agent with Causal Faithfulness Analysis [10.749786847079163]
  Explanations for AI models in high-stakes domains like medicine often lack verifiability, which can hinder trust.<n>We propose an interactive agent that produces explanations through an auditable sequence of actions.<n>This policy is optimized using reinforcement learning, resulting in a model that is both efficient and generalizable.
  arXiv  Detail & Related papers  (2025-11-03T10:21:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.