Escaping the Agreement Trap: Defensibility Signals for Evaluating Rule-Governed AI

• URL: http://arxiv.org/abs/2604.20972v1
• Date: Wed, 22 Apr 2026 18:05:29 GMT
• Title: Escaping the Agreement Trap: Defensibility Signals for Evaluating Rule-Governed AI
• Abstract summary: We formalize evaluation as policy-grounded correctness and introduce the Defensibility Index (DI) and Ambiguity Index (AI)<n>We validate the framework on 193,000+ Reddit moderation decisions across multiple communities and evaluation cohorts.
• Score: 0.6138671548064355
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Content moderation systems are typically evaluated by measuring agreement with human labels. In rule-governed environments this assumption fails: multiple decisions may be logically consistent with the governing policy, and agreement metrics penalize valid decisions while mischaracterizing ambiguity as error - a failure mode we term the Agreement Trap. We formalize evaluation as policy-grounded correctness and introduce the Defensibility Index (DI) and Ambiguity Index (AI). To estimate reasoning stability without additional audit passes, we introduce the Probabilistic Defensibility Signal (PDS), derived from audit-model token logprobs. We harness LLM reasoning traces as a governance signal rather than a classification output by deploying the audit model not to decide whether content violates policy, but to verify whether a proposed decision is logically derivable from the governing rule hierarchy. We validate the framework on 193,000+ Reddit moderation decisions across multiple communities and evaluation cohorts, finding a 33-46.6 percentage-point gap between agreement-based and policy-grounded metrics, with 79.8-80.6% of the model's false negatives corresponding to policy-grounded decisions rather than true errors. We further show that measured ambiguity is driven by rule specificity: auditing 37,286 identical decisions under three tiers of the same community rules reduces AI by 10.8 pp while DI remains stable. Repeated-sampling analysis attributes PDS variance primarily to governance ambiguity rather than decoding noise. A Governance Gate built on these signals achieves 78.6% automation coverage with 64.9% risk reduction. Together, these results show that evaluation in rule-governed environments should shift from agreement with historical labels to reasoning-grounded validity under explicit rules.

Related papers

• Grounded Normative Rule Generation with Structured Search [47.28595196084396]
  We formalize the problem as Grounded Normative Rule Synthesis (GNRS)<n>We introduce GNRS-Search, a framework that utilizes Markov Chain Monte Carlo (MCMC) sampling to optimize a discrete, five-slot And-Or Graph (AOG)<n>We evaluate our approach on GNRS-Bench, a benchmark spanning 116 controlled goals across eight scene families, and RealCharterBench, which evaluates transfer to 53 real-derived policy tasks with hidden source clauses.
  arXiv  Detail & Related papers  (2026-08-23T05:43:42Z)
• DFAH-Bench: Benchmarking Observable Agent Instability in Financial Decision-Making [0.7699235580548228]
  DFAH-Bench is a replay benchmark that measures observable behavioral instability in financial agent decision-making.<n>We find that frontier models can agree on decisions 95% of the time while following the same tool path only 77% of the time.<n>We identify three behavioral profiles: pattern matchers that achieve near-perfect agreement by collapsing to a single output regardless of input, stable executors with relatively consistent tool-use processes, and trajectory divergers that reach the same conclusions through materially different tool paths and evidence contacts.
  arXiv  Detail & Related papers  (2026-06-10T03:31:09Z)
• Diagnosing Live Within-Policy Instruction Conflicts in LLM Agents with Witnessed Resolution Profiles [15.65681557926802]
  LLM agents are governed by long-lived natural-collision prompt policies, but individually reasonable standing rules can interact in uninspected ways.<n>We study live intra-policy rule-conflict diagnosis: finding rule pairs inside a single prompt policy that can co-govern a realistic state.<n>We introduce WIRE, a Witnessed Intra-policy Rule Evaluation pipeline.
  arXiv  Detail & Related papers  (2026-05-27T00:09:45Z)
• Auditable Decision Models with Learned Abstention and Real-Time Steering [6.287457666346811]
  Production AI systems often operate with incomplete, conflicting, or insufficient evidence.<n>We study operational decision control for AI systems, where uncertainty must be explicitly routable.<n>We present EvaluatorDPT, a bounded decision-control model that predicts YES, NO, or TBD.
  arXiv  Detail & Related papers  (2026-05-26T23:37:56Z)
• Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring [0.0]
  Current approaches to AI compliance treat conformity as a binary, audit-time verdict.<n>We argue that this compliance fiction is structurally ill-suited to the requirements of the EU AI Act.<n>We introduce governance from metrics, a principle whereby regulatory compliance is derived as a continuous signal from behavioural observability.
  arXiv  Detail & Related papers  (2026-05-23T21:21:33Z)
• RuleSafe-VL: Evaluating Rule-Conditioned Decision Reasoning in Vision-Language Content Moderation [4.531887731522564]
  RuleSafe-VL is a benchmark for rule-conditioned decision reasoning in content moderation.<n>It formalizes 93 atomic rules and 92 typed rule relations, yielding 2,166 context-sensitive image-text cases.<n>It identifies activated rules, recover rule interactions, judge decision sufficiency, and resolve outcomes once missing context is supplied.
  arXiv  Detail & Related papers  (2026-05-08T14:05:00Z)
• PolicyBank: Evolving Policy Understanding for LLM Agents [51.86716874651299]
  PolicyBank maintains structured, tool-level policy insights and iteratively refines them.<n>PolicyBank closes up to 82% of the gap toward a human oracle.
  arXiv  Detail & Related papers  (2026-04-16T20:29:30Z)
• The Confidence Gate Theorem: When Should Ranked Decision Systems Abstain? [0.0]
  Ranked decision systems must decide when to intervene in ranked outputs and when to abstain.<n>We study when confidence-based abstention monotonically improves decision quality, and when it fails.
  arXiv  Detail & Related papers  (2026-03-10T17:44:10Z)
• Admissibility Alignment [0.0]
  We present MAP-AI, a new control-plane system architecture for aligned decision-making under uncertainty.<n>It enforces alignment through Monte Carlo estimation of outcome distributions and admissibility-controlled policy selection.<n>We show how alignment evaluation can be integrated into decision-making itself, yielding an admissibility-controlled action selection mechanism.
  arXiv  Detail & Related papers  (2026-01-05T05:58:19Z)
• Policy-Aware Generative AI for Safe, Auditable Data Access Governance [0.0]
  We present a policy aware controller that uses a large language model (LLM) to interpret natural language requests against written policies and metadata, not raw data.<n>The system, implemented with Google Gemini2.0 Flash, executes a six-stage reasoning framework with early hard policy gates and deny by default.
  arXiv  Detail & Related papers  (2025-10-27T16:10:55Z)
• TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them [58.04324690859212]
  Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks.<n>We identify two fundamental types of inconsistencies: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency.<n>We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations.
  arXiv  Detail & Related papers  (2025-09-25T13:04:29Z)
• Conformal Off-Policy Evaluation in Markov Decision Processes [53.786439742572995]
  Reinforcement Learning aims at identifying and evaluating efficient control policies from data. Most methods for this learning task, referred to as Off-Policy Evaluation (OPE), do not come with accuracy and certainty guarantees. We present a novel OPE method based on Conformal Prediction that outputs an interval containing the true reward of the target policy with a prescribed level of certainty.
  arXiv  Detail & Related papers  (2023-04-05T16:45:11Z)
• Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation [60.71312668265873]
  We develop a method to balance the need for personalization with confident predictions. We show that our method can be used to form accurate predictions of heterogeneous treatment effects.
  arXiv  Detail & Related papers  (2021-11-28T23:19:12Z)
• Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies [80.42316902296832]
  We study the estimation of policy value and gradient of a deterministic policy from off-policy data when actions are continuous. In this setting, standard importance sampling and doubly robust estimators for policy value and gradient fail because the density ratio does not exist. We propose several new doubly robust estimators based on different kernelization approaches.
  arXiv  Detail & Related papers  (2020-06-06T15:52:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.