Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demands

• URL: http://arxiv.org/abs/2605.15164v1
• Date: Thu, 14 May 2026 17:54:28 GMT
• Title: Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demands
• Abstract summary: We argue that behavioural assurance, even when carefully designed, is being asked to carry safety claims it cannot verify.<n>We formalize this structural mismatch as the audit gap, the divergence between required and achievable verification access, and introduce the concept of fragile assurance.<n>We propose a technical pivot: bounding the weight of behavioral evidence in legal text and extending voluntary pre-deployment access with mechanistic-evidence classes.
• Score: 4.125187280299246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This position paper argues that behavioural assurance, even when carefully designed, is being asked to carry safety claims it cannot verify. AI governance frameworks enacted between 2019 and early 2026 require reviewable evidence of properties such as the absence of hidden objectives, resistance to loss-of-control precursors, and bounded catastrophic capability; current assurance methodologies (primarily behavioural evaluations and red-teaming) are epistemically limited to observable model outputs and cannot verify the latent representations or long-horizon agentic behaviours these frameworks presume to regulate. We formalize this structural mismatch as the audit gap, the divergence between required and achievable verification access, and introduce the concept of fragile assurance to describe cases where the evidential structure does not support the asserted safety claim. Through an analysis of a 21-instrument inventory, we identify an incentive gradient where geopolitical and industrial pressures systematically reward surface-level behavioral proxies over deep structural verification. Finally, we propose a technical pivot: bounding the weight of behavioral evidence in legal text and extending voluntary pre-deployment access with mechanistic-evidence classes, specifically linear probes, activation patching, and before/after-training comparisons.

Related papers

• Cognitive Admission Control: Risk-Conditioned Assurance for Consequential Actions in Agentic Distributed Systems [2.124730017640531]
  In agentic distributed systems, an agent may be authorized to mutate external infrastructure while lacking evidence that the mutation is ready to execute.<n> Cognitive Admission Control (CAC) makes this evidence requirement explicit.<n>A policy maps a typed action and its modeled risk to assurance obligations specifying predicates, evidence classes, scope, freshness, and witness-set constraints.<n>A successful admission produces a certificate binding the action, its witness manifest, and dispatch-time guards.
  arXiv  Detail & Related papers  (2026-09-14T20:20:54Z)
• AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation [48.55515767840701]
  "AI slop" is artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports.<n>This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage.
  arXiv  Detail & Related papers  (2026-08-26T11:48:38Z)
• Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle [65.6642776933861]
  This survey examines the protective paradigm that has grown around this intervention point.<n>We find that most protections are still validated mainly against static or weakly adaptive adversaries.<n>We close by consolidating cross-stage countermeasures and open problems for robust, composable, and deployable owner-side protection.
  arXiv  Detail & Related papers  (2026-08-05T00:46:50Z)
• Towards Trustworthy Embodied Intelligence: A Systems Framework and Graded Trustworthiness Levels [53.63220028820581]
  Embodied intelligence integrates learned perception and decision making with real-time computation, control, and physical interaction.<n>We define trustworthy embodied intelligence as the sustained capacity to execute specified tasks reliably.
  arXiv  Detail & Related papers  (2026-07-28T17:50:44Z)
• What Benchmarks Don't Measure: The Case for Evaluating Abstention Competence in Autonomous Agents [2.8340200313563773]
  Benchmarks for autonomous agents measure whether agents complete tasks, yet this framing is systematically blind to whether an agent should have proceeded at all.<n>Agents trained under human-feedback objectives develop a structural tendency to proceed even when they lack the inputs, evidence, or authorization to act safely.<n>We introduce a three-gap taxonomy of abstention-warranted scenarios, covering specification gaps where required information is absent, verification gaps where world state cannot be confirmed, and authority gaps where explicit authorization has not been given.
  arXiv  Detail & Related papers  (2026-06-01T23:52:56Z)
• Provably Secure Agent Guardrail [89.79561918065122]
  Existing defense architectures rely on empirical semantic guardrails and probabilistic large model adjudicators.<n>This paper proposes a new security paradigm for agents based on the fundamental limitations of logical reasoning.
  arXiv  Detail & Related papers  (2026-05-28T02:12:41Z)
• Referential Security as a New Paradigm for AI Evaluations [5.748110232428846]
  We propose referential security as a new paradigm for AI evaluation.<n>This approach reframes model identity as an empirically verifiable property and separates referential stability from the substantive security claims it conditions.
  arXiv  Detail & Related papers  (2026-05-25T10:24:10Z)
• PolicyBank: Evolving Policy Understanding for LLM Agents [51.86716874651299]
  PolicyBank maintains structured, tool-level policy insights and iteratively refines them.<n>PolicyBank closes up to 82% of the gap toward a human oracle.
  arXiv  Detail & Related papers  (2026-04-16T20:29:30Z)
• Authority-Level Priors: An Under-Specified Constraint in Hierarchical Predictive Processing [0.0]
  Authority-Level Priors (ALPs) are meta-structural constraints defining a regulatory-admissible subset of identity-level hypotheses.<n>ALPs are not additional representational states nor hyperpriors over precision; they constrain which hypotheses are admissible for regulatory control.<n>A computational formalisation restricts policy optimisation to policies generated by authorised hypotheses.
  arXiv  Detail & Related papers  (2026-03-19T13:27:47Z)
• GAVEL: Towards rule-based safety through activation monitoring [2.337566423505956]
  Large language models (LLMs) are increasingly paired with activation-based monitoring to detect and prevent harmful behaviors.<n>Existing activation safety approaches, trained on broad misuse datasets, struggle with poor precision, limited flexibility, and lack of interpretability.<n>This paper introduces a new paradigm: rule-based activation safety, inspired by rule-sharing practices in cybersecurity.
  arXiv  Detail & Related papers  (2026-01-27T16:31:39Z)
• Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process [66.38541693477181]
  We propose an unsupervised framework for discovering reasoning vectors, which we define as directions in the activation space that encode distinct reasoning behaviors.<n>By segmenting chain-of-thought traces into sentence-level'steps', we uncover disentangled features corresponding to interpretable behaviors such as reflection and backtracking.<n>We demonstrate the ability to control response confidence by identifying confidence-related vectors in the SAE decoder space.
  arXiv  Detail & Related papers  (2025-12-30T05:09:11Z)
• The Loss of Control Playbook: Degrees, Dynamics, and Preparedness [39.39076397908963]
  This report addresses the absence of an actionable definition for Loss of Control (LoC) in AI systems by developing a novel taxonomy and preparedness framework.<n>We propose a graded LoC taxonomy, based on the metrics of severity and persistence, that distinguishes between Deviation, Bounded LoC, and Strict LoC.<n>We put forward a plan to maintain preparedness and prevent the occurrence of LoC outcomes should a state of societal vulnerability be reached.
  arXiv  Detail & Related papers  (2025-11-19T20:10:39Z)
• Confidential Guardian: Cryptographically Prohibiting the Abuse of Model Abstention [65.47632669243657]
  A dishonest institution can exploit mechanisms to discriminate or unjustly deny services under the guise of uncertainty.<n>We demonstrate the practicality of this threat by introducing an uncertainty-inducing attack called Mirage.<n>We propose Confidential Guardian, a framework that analyzes calibration metrics on a reference dataset to detect artificially suppressed confidence.
  arXiv  Detail & Related papers  (2025-05-29T19:47:50Z)
• Advancing Neural Network Verification through Hierarchical Safety Abstract Interpretation [52.626086874715284]
  We introduce a novel problem formulation called Abstract DNN-Verification, which verifies a hierarchical structure of unsafe outputs.<n>By leveraging abstract interpretation and reasoning about output reachable sets, our approach enables assessing multiple safety levels during the formal verification process.<n>Our contributions include a theoretical exploration of the relationship between our novel abstract safety formulation and existing approaches.
  arXiv  Detail & Related papers  (2025-05-08T13:29:46Z)
• A PRISMA-Driven Bibliometric Analysis of the Scientific Literature on Assurance Case Patterns [7.930875992631788]
  Assurance cases can be used to prevent system failure. They are structured arguments that allow arguing and relaying various safety-critical systems' requirements.
  arXiv  Detail & Related papers  (2024-07-06T05:00:49Z)
• No Need to Know Physics: Resilience of Process-based Model-free Anomaly Detection for Industrial Control Systems [95.54151664013011]
  We present a novel framework to generate adversarial spoofing signals that violate physical properties of the system. We analyze four anomaly detectors published at top security conferences.
  arXiv  Detail & Related papers  (2020-12-07T11:02:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.