The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don't

• URL: http://arxiv.org/abs/2605.01771v1
• Date: Sun, 03 May 2026 08:11:15 GMT
• Title: The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don't
• Abstract summary: We call this the Compliance Gap: a third, axis of AI honesty distinct from factual truthfulness and rhetorical substance.<n>Some 75 benchmarks measure outcome fidelity; none measures process fidelity.<n>We release BS-Bench: the first open benchmark for process compliance, with seven tool-call-log audit metrics and a public leaderboard.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An auditor instructs an AI assistant: "open each file individually using the Read tool -- no scripts, no agents." The AI replies "Yes" -- then issues a single batched call summarizing all fifty files at once. We call this the Compliance Gap: a third, orthogonal axis of AI honesty distinct from factual truthfulness and rhetorical substance. Three questions: does this verbal-behavioral disconnect exist (existence); can any text-only observer recover it (detectability); what infrastructure does AI deployment need (remedy)? Some 75 benchmarks (IFEval, SWE-bench, BFCL, COMPASS, SpecEval) measure outcome fidelity; none measures process fidelity. Theorem 1 shows the gap is structurally inevitable under RL that rewards text without observing behavior. Theorem 2, via the Data Processing Inequality, shows it is undetectable from text alone -- by any human or LLM observer, present or future. Thirteen experiments and 2,031 sessions on six frontier models confirm both predictions. Under default framing, all six exhibit instruction compliance rates of 0% -- Claude Sonnet 4 verbally agrees ten out of ten times then bypasses in all ten. The gap is selective: 97% compliance where rationale is rewarded (audit trails), 0-4% where it is not (file reading, privacy masking); removing delegation tools raises compliance to 75% (Cohen's d = 2.47), confirming environmental affordance rather than weight-encoded failure. Nine blinded human raters achieve Fleiss' kappa = 0.130 and correctly identify zero of fifteen compliant sessions, exactly as Theorem 2 predicts. Where humans show 47% intention-behavior gaps in psychology and 96.5pp gaps in surgical audits, RLHF-trained models approach 100% under default conditions -- a regime warranting its own measurement infrastructure. We release BS-Bench: the first open benchmark for process compliance, with seven tool-call-log audit metrics and a public leaderboard.

Related papers

• SWE-Test: Benchmarking LLM Vulnerability Discovery via Input Prediction [13.719725391379276]
  Vulnerability discovery is an important ability of large language model (LLM) agents.<n>We recast its measurement as an input-prediction task with a closed, deterministic ground truth.<n>This decomposes discovery into three task modes over 22 real-world C/C++ programs spanning 15 domains.
  arXiv  Detail & Related papers  (2026-09-05T19:04:41Z)
• When Do Internal Probes Beat Reading the Answer? Miscalibrated Readouts and Behavior-Concealed Knowledge in Language Models [0.0]
  A language model, asked to verify 1,200 logical conclusions, answers YES every time.<n>linear probes on its hidden states read the correct verdict at 0.96 AUC.<n>We ask where the verdict is lost, and find the dominant failure is a single scalar.<n>The diagnosis generalizes: across 90 semantic-label configurations of a five-model, three-family factorial, behavioral accuracy collapses onto a single function of threshold offset.
  arXiv  Detail & Related papers  (2026-09-04T00:26:45Z)
• ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices [48.36933831982682]
  We introduce ADeptS-Bench, a trustworthiness benchmark grounded in the ADEPTS capability framework.<n>The Safety stream provides paired benign/malicious tasks with threats embedded in the visual interface.<n>The Disambiguation stream evaluates whether agents seek clarification when intent is ambiguous.
  arXiv  Detail & Related papers  (2026-08-25T23:21:57Z)
• Judging Is Not Enumerating: Silent Omissions in LLM-Authored Acceptable Sets [13.85834524293015]
  We measure the capability that role assumes and find it lacking under the protocol the role is usually deployed with.<n>Models judge whether a candidate belongs far better than they author the set itself.
  arXiv  Detail & Related papers  (2026-08-02T05:00:44Z)
• The Librarian Who Refused to Code: Model-Dependent Identity Enactment in LLM Code Generation [0.0]
  We tested four prompt conditions (no persona, two engineer personas, and a research-librarian persona), 12 code-generation tasks, two frontier models, and five runs per cell.<n>Under pre-registered mixed-effects analysis, the condition-by-model interaction was significant for provider-reported output tokens.<n>On Claude Opus, the minimalist engineer persona reduced visible output by 30% without improving correctness, while the thorough engineer persona increased output without a correctness gain.
  arXiv  Detail & Related papers  (2026-07-19T21:47:58Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• FirstPass: Grounding AI Scientific Judgment in Multi-Round Editorial Outcomes [0.27998963147546146]
  We introduce FirstPass, a dataset fine-tuned model that addresses all three fronts of AI for peer review.<n>We exploit mandatory transparent peer review (instituted November 2022) and verify 100% content integrity by automated audit.<n>On generation, FirstPass produces reviews averaging 1,187 words, substantially closer to human references (2,155 words) than any baseline.
  arXiv  Detail & Related papers  (2026-06-18T15:06:36Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• TRUST: A Framework for Decentralized AI Service v.0.1 [47.384270414446604]
  Large Reasoning Models (LRMs) and Multi-Agent Systems (MAS) in high-stakes domains demand reliable verification.<n>We introduce TRUST (Transparent, Robust, and Unified Services for Trustworthy AI), a decentralized framework with three innovations.<n>We prove a Safety-Profitability Theorem ensuring honest auditors profit while malicious actors incur losses.
  arXiv  Detail & Related papers  (2026-04-29T19:32:58Z)
• AIRA: AI-Induced Risk Audit: A Structured Inspection Framework for AI-Generated Code [0.0]
  Practitioners have reported a directional pattern in AI-assisted code generation: AI-generated code tends to fail quietly.<n>This paper introduces the Reward-Shaped Failure Hypothesis - the proposal that this pattern may reflect an artifact of optimization through human feedback.<n>We present AIRA, a deterministic 15-check inspection framework designed to detect failure-untruthful patterns in code.
  arXiv  Detail & Related papers  (2026-04-19T19:32:52Z)
• Formal Architecture Descriptors as Navigation Primitives for AI Coding Agents [0.0]
  We investigate whether providing agents with formal architecture descriptors can reduce this navigational overhead.<n>We propose intent.lisp, an S-expression architecture descriptor, and open-source the Forge toolkit.
  arXiv  Detail & Related papers  (2026-04-11T00:26:31Z)
• ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Models [60.14219417402433]
  Existing memory benchmarks for LLM agents evaluate explicit recall of facts, yet overlook implicit memory where experience becomes automated behavior without conscious retrieval.<n>We introduce ImplicitMemBench, the first systematic benchmark evaluating implicit memory through three constructs drawn from standard cognitive-science accounts of non-declarative memory.<n>Our 300-item suite employs a unified Learning/Priming-Interfere-Test protocol with first-attempt scoring.
  arXiv  Detail & Related papers  (2026-04-09T10:26:32Z)
• Can Humans Tell? A Dual-Axis Study of Human Perception of LLM-Generated News [47.03825808787752]
  Can humans tell whether a news article was written by a person or a large language model (LLM)?<n>We investigate this question using JudgeGPT, a study platform that measures source attribution (human vs. machine) and authenticity judgment (legitimate vs. fake) on continuous scales.<n>We report five findings: (1) participants cannot reliably distinguish machine-generated from human-written text; (2) this inability holds across all tested models, including open-weight models with as few as 7B parameters; and (3) accuracy degrades after approximately 30 sequential evaluations due to cognitive fatigue.
  arXiv  Detail & Related papers  (2026-04-04T15:03:42Z)
• Epistemic Observability in Language Models [0.0]
  We find that models report highest confidence precisely when they are fabricating.<n>We prove, under formal assumptions, that this is not a capability gap but an observational one.<n>We construct a tensor interface that escapes the impossibility by exporting computational byproducts.
  arXiv  Detail & Related papers  (2026-03-20T21:59:34Z)
• GPT4o-Receipt: A Dataset and Human Study for AI-Generated Document Forensics [12.448893104517808]
  We present GPT4o-Receipt, a benchmark of 1,235 receipt images pairing GPT-4o-generated receipts with authentic ones from established datasets.<n>Human annotators exhibit the largest visual discrimination gap of any evaluator, yet their binary detection F1 falls well below Claude Sonnet 4 and below Gemini 2.5 Flash.<n>Our five-model evaluation reveals dramatic performance disparities and calibration differences that render simple accuracy metrics insufficient for detector selection.
  arXiv  Detail & Related papers  (2026-03-12T02:05:27Z)
• Aligning to Illusions: Choice Blindness in Human and AI Feedback [2.912535004465788]
  In a human choice blindness study, 91% of surreptitiously swapped preferences go undetected, extending choice blindness to third-person evaluative comparison of unfamiliar text.<n>We find detection relies on shallow text matching rather than genuine self-monitoring.<n>In a dose-response experiment across two architectures, one-sixth to one-third of labels must be corrupted before the reward signal halves.<n>A Best-of-N evaluation confirms this translates to downstream policy degradation.
  arXiv  Detail & Related papers  (2026-03-09T14:10:36Z)
• HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units [81.53783563025084]
  We propose an offline clustering step to provide aligned target labels for a BERT-like prediction loss. A key ingredient of our approach is applying the prediction loss over the masked regions only. HuBERT shows up to 19% and 13% relative WER reduction on the more challenging dev-other and test-other evaluation subsets.
  arXiv  Detail & Related papers  (2021-06-14T14:14:28Z)
• PRover: Proof Generation for Interpretable Reasoning over Rules [81.40404921232192]
  We propose a transformer-based model that answers binary questions over rule-bases and generates the corresponding proofs. Our model learns to predict nodes and edges corresponding to proof graphs in an efficient constrained training paradigm. We conduct experiments on synthetic, hand-authored, and human-paraphrased rule-bases to show promising results for QA and proof generation.
  arXiv  Detail & Related papers  (2020-10-06T15:47:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.