From Sycophancy to Sensemaking: Premise Governance for Human-AI Decision Making

• URL: http://arxiv.org/abs/2602.02378v1
• Date: Mon, 02 Feb 2026 17:42:54 GMT
• Title: From Sycophancy to Sensemaking: Premise Governance for Human-AI Decision Making
• Abstract summary: Low-friction assistants can become sycophantic, baking in implicit assumptions and pushing verification costs onto experts.<n>We argue reliable human-AI partnership requires a shift from answer generation to collaborative premise governance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As LLMs expand from assistance to decision support, a dangerous pattern emerges: fluent agreement without calibrated judgment. Low-friction assistants can become sycophantic, baking in implicit assumptions and pushing verification costs onto experts, while outcomes arrive too late to serve as reward signals. In deep-uncertainty decisions (where objectives are contested and reversals are costly), scaling fluent agreement amplifies poor commitments faster than it builds expertise. We argue reliable human-AI partnership requires a shift from answer generation to collaborative premise governance over a knowledge substrate, negotiating only what is decision-critical. A discrepancy-driven control loop operates over this substrate: detecting conflicts, localizing misalignment via typed discrepancies (teleological, epistemic, procedural), and triggering bounded negotiation through decision slices. Commitment gating blocks action on uncommitted load-bearing premises unless overridden under logged risk; value-gated challenge allocates probing under interaction cost. Trust then attaches to auditable premises and evidence standards, not conversational fluency. We illustrate with tutoring and propose falsifiable evaluation criteria.

Related papers

• Why2Speak: Faithful Reasoning for Abstaining Action Policies [0.19336815376402716]
  We compare direct decision policies, reasoning policies, supervised fine-tuning, and reinforcement learning.<n>We find a capability-auditability tradeoff: the strongest direct policy achieves higher quality but exposes no reasoning to inspect.<n>Supervised fine-tuning either suppresses reasoning or preserves it without improving decision quality, while reinforcement learning also fails to improve the reasoning policy.
  arXiv  Detail & Related papers  (2026-08-21T02:00:21Z)
• Sharding Prevents LLM Oversight Failures and Adversarial Exploitation [45.380649793616705]
  Sharding partitions requirements into smaller groups, assigns each group to a separate call, and aggregates the verdicts.<n>We find that a sharded weaker judge can outperform a more capable holistic judge and match that judge even when the latter receives the panel's full budget.
  arXiv  Detail & Related papers  (2026-08-05T18:21:21Z)
• Consensus is Strategically Insufficient: Reasoning-Trace Disagreement as a Knowledge-Representation Signal [0.0]
  Multi-agent systems are commonly designed to reduce disagreement through voting, consensus protocols, debate, or fault-tolerant aggregation.<n>We argue that this objective is insufficient for value-laden tasks, where disagreement may reflect genuine normative uncertainty rather than agent error.<n>We propose a knowledge-representation layer in which reasoning traces and agent decisions are abstracted into symbolic disagreement states.
  arXiv  Detail & Related papers  (2026-06-02T21:21:02Z)
• Explainable AI Isn't Enough! Rethinking Algorithmic Contestability [3.135152720206845]
  We propose an operational definition of contestability as a natural complement to recourse.<n>We identify three types of evidence warranting reversal according to the decision maker's own ethical standards.<n>We argue that these render decisions normatively indefensible and thus successfully contestable.
  arXiv  Detail & Related papers  (2026-05-15T15:14:13Z)
• TERMS-Bench: Diagnosing LLM Negotiation Agents Beyond Deal Rate [34.67393151003599]
  Negotiation is a canonical testbed for agentic language models.<n>Unlike math or code, it has no intrinsic verifier.<n>We introduce Terms-Bench, a Bayesian-game framework that makes the environment itself the verifier.
  arXiv  Detail & Related papers  (2026-05-13T06:22:50Z)
• When Contextual Inference Fails: Cancelability in Interactive Instruction Following [51.2195840589474]
  We introduce Build What I Mean, an interactive benchmark for contextual meaning construction.<n>In BWIM, models must resolve ambiguity by either performing a contextual inference or requesting clarification at a small communication cost.<n>We observe suboptimal strategies, such as partner-blind over-clarification and question-averse guessing under uncertainty.
  arXiv  Detail & Related papers  (2026-03-20T14:46:59Z)
• AI Knows What's Wrong But Cannot Fix It: Helicoid Dynamics in Frontier LLMs Under High-Stakes Decisions [51.56484100374058]
  Helicoid dynamics is the name given to a specific failure regime in that second domain.<n>System engages competently, drifts into error, accurately names what went wrong, then reproduces the same pattern at a higher level of sophistication.<n>This prospective case series documents that regime across seven leading systems.
  arXiv  Detail & Related papers  (2026-03-12T05:25:49Z)
• The Chancellor Trap: Administrative Mediation and the Hollowing of Sovereignty in the Algorithmic Age [0.0]
  In high- throughput organizations, AI-mediated decision support can reduce the probability that failures become publicly legible and politically contestable.<n>The article formalizes this dynamic as a principal-agent problem characterized by a verification gap.<n>The results are consistent with a paradox of competence: governance systems may become more effective at absorbing and resolving failures internally while simultaneously raising the threshold at which those failures become politically visible.
  arXiv  Detail & Related papers  (2026-02-09T07:28:44Z)
• Drift-Bench: Diagnosing Cooperative Breakdowns in LLM Agents under Input Faults via Multi-Turn Interaction [20.610305266852638]
  textbfDrift-Bench is the first diagnostic benchmark that evaluates agentic pragmatics under input faults.<n>Method bridges clarification research and agent safety evaluation, enabling systematic diagnosis of failures that can lead to unsafe executions.
  arXiv  Detail & Related papers  (2026-02-02T18:46:16Z)
• Preventing the Collapse of Peer Review Requires Verification-First AI [49.995126139461085]
  We propose truth-coupling, i.e. how tightly venue scores track latent scientific truth.<n>We formalize two forces that drive a phase transition toward proxy-sovereign evaluation.
  arXiv  Detail & Related papers  (2026-01-23T17:17:32Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)
• Making LLMs Reliable When It Matters Most: A Five-Layer Architecture for High-Stakes Decisions [51.56484100374058]
  Current large language models (LLMs) excel in verifiable domains where outputs can be checked before action but prove less reliable for high-stakes strategic decisions with uncertain outcomes.<n>This gap, driven by mutually cognitive biases in both humans and artificial intelligence (AI) systems, threatens the defensibility of valuations and sustainability of investments in the sector.<n>This report describes a framework emerging from systematic qualitative assessment across 7 frontier-grade LLMs and 3 market-facing venture vignettes under time pressure.
  arXiv  Detail & Related papers  (2025-11-10T22:24:21Z)
• The Epistemic Suite: A Post-Foundational Diagnostic Methodology for Assessing AI Knowledge Claims [0.7233897166339268]
  This paper introduces the Epistemic Suite, a diagnostic methodology for surfacing the conditions under which AI outputs are produced and received.<n>Rather than determining truth or falsity, the Suite operates through twenty diagnostic lenses to reveal patterns such as confidence laundering, narrative compression, displaced authority, and temporal drift.
  arXiv  Detail & Related papers  (2025-09-20T00:29:38Z)
• Information Bargaining: Bilateral Commitment in Bayesian Persuasion [60.3761154043329]
  We introduce a unified framework and a well-structured solution concept for long-term persuasion.<n>This perspective makes explicit the common knowledge of the game structure and grants the receiver comparable commitment capabilities.<n>The framework is validated through a two-stage validation-and-inference paradigm.
  arXiv  Detail & Related papers  (2025-06-06T08:42:34Z)
• Debate, Deliberate, Decide (D3): A Cost-Aware Adversarial Framework for Reliable and Interpretable LLM Evaluation [0.0]
  We present Debate, Deliberate, Decide (D3), a cost-aware, adversarial multi-agent framework that orchestrates structured debate among role-specialized agents.<n>We develop a probabilistic model of score gaps that characterizes reliability and convergence under iterative debate.<n>We show state-of-the-art agreement with human judgments, reduced positional and verbosity biases via anonymization, and a favorable cost-accuracy frontier.
  arXiv  Detail & Related papers  (2024-10-07T00:22:07Z)
• Multi-Agent Imitation Learning: Value is Easy, Regret is Hard [52.31989962031179]
  We study a multi-agent imitation learning (MAIL) problem where we take the perspective of a learner attempting to coordinate a group of agents. Most prior work in MAIL essentially reduces the problem to matching the behavior of the expert within the support of the demonstrations. While doing so is sufficient to drive the value gap between the learner and the expert to zero under the assumption that agents are non-strategic, it does not guarantee to deviations by strategic agents.
  arXiv  Detail & Related papers  (2024-06-06T16:18:20Z)
• Online Decision Mediation [72.80902932543474]
  Consider learning a decision support assistant to serve as an intermediary between (oracle) expert behavior and (imperfect) human behavior. In clinical diagnosis, fully-autonomous machine behavior is often beyond ethical affordances.
  arXiv  Detail & Related papers  (2023-10-28T05:59:43Z)
• `Why didn't you allocate this task to them?' Negotiation-Aware Explicable Task Allocation and Contrastive Explanation Generation [19.265913720022574]
  We propose a central Artificial Intelligence Task Allocation (AITA) that simulates a negotiation. If a team-member is unhappy with the proposed allocation, we allow them to question the proposed allocation using a counterfactual. We show that (1) the allocation proposed using our method appears fair to the majority, and (2) when a counterfactual is raised, explanations generated are easy to comprehend and convincing.
  arXiv  Detail & Related papers  (2020-02-05T04:58:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.