What Can Be Enforced? A Theory of Certified Runtime Safety for Tool-Using Agents

• URL: http://arxiv.org/abs/2607.22868v1
• Date: Fri, 24 Jul 2026 19:14:26 GMT
• Title: What Can Be Enforced? A Theory of Certified Runtime Safety for Tool-Using Agents
• Abstract summary: guardrails act before irreversible tool calls, but their guarantees depend on what policy state is representable.<n>Pearson representation attacks add a margin, so benign calibration alone does not transfer.<n>Experiments target these distinctions through static diagnostics, controlled-model enumeration, representation rewrites, and paired closed-loop reruns.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Runtime guardrails act before irreversible tool calls, but their guarantees depend on what policy state is representable, what a judge observes, and whether intervention changes future behavior. We separate three questions. First, relative to fixed oracle predicates, a deterministic gate enforces exactly the nonempty safety policies whose good prefixes its register model recognizes; policy nontriviality is undecidable with two decrementable counters but in PSPACE for a separable monotone fragment. Second, under a fixed exogenous law, Neyman-Pearson gives the exact false-block/miss frontier and conformal calibration gives a finite-sample marginal certificate, possibly via block-all. Third, once blocking changes future proposals, static scores and ungated trajectories need not identify the closed-loop frontier; a specified finite controlled model instead yields an occupancy program. Bounded representation attacks add a robustness margin, so benign calibration alone does not transfer. Experiments target these distinctions through static diagnostics, controlled-model enumeration, representation rewrites, and paired closed-loop reruns.

Related papers

• Semantic Fibers and Cross-Gram Interference: A Calculus of Safety Drift in Overcomplete Representations [0.0]
  A deployed language model may refuse a harmful request in English yet comply with its faithful translation.<n>We formalize this phenomenon through an audited equivalence relation.<n>We show that, for a declared quotient, representation, metric, feature dictionary, scoring head, threshold, and contrast model, the resulting safety drift admits an exact linear-algebraic characterization.
  arXiv  Detail & Related papers  (2026-09-14T00:23:32Z)
• Certifying Plans under Model Mismatch: A Trilemma for Reachability from Scarce Data [5.455772242709494]
  Sim-to-real policies are designed under nominal dynamics, but target-system trials may yield only a few isolated one-step transitions.<n>We study pre-execution certification of a fixed control sequence, such as an action chunk produced by a learned policy.
  arXiv  Detail & Related papers  (2026-08-03T16:28:26Z)
• When Can Safe Controllers Adapt? Information before Commitment [26.556002437630426]
  We study safe adaptive control under a safety guarantee on the learning trajectory itself.<n>Performance is measured against a safe oracle that knows the realized model.<n>We prove recovery in special cases and derive semidefinite upper certificates for deterministic linear-Gaussian systems.
  arXiv  Detail & Related papers  (2026-07-18T17:30:41Z)
• Falsifiable Release Gates for Self-Improving Systems [0.0]
  We describe falsifiable release gates, and a methodology to build and validate such systems.<n>Every new capability must pass a pre-specified, machine-verifiable acceptance suite before it ships.<n>No action goes to an effector without a safety-critical property capability token minted by a control ring.
  arXiv  Detail & Related papers  (2026-07-11T06:06:34Z)
• Greedy Coordinate Diffusion: Effective and Semantically Coherent Adversarial Attacks via Diffusion Guidance [48.34904668359272]
  Adrial attacks on large language models have limited practical impact despite extensive research.<n>We introduce Greedy Coordinate Diffusion (GCD), a novel framework that efficiently generates adversarial attacks against safety-aligned models.<n>GCD maintains low perplexity and high semantic adherence to the adversary's original intent.
  arXiv  Detail & Related papers  (2026-06-14T01:18:53Z)
• Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RL [76.45061154544568]
  Self-play reinforcement learning trains language models on their own generated tasks, co-evolving a proposer and solver without human labels.<n>Recent systems report strong reasoning gains, but collapse and instability are widely observed and poorly understood.<n>We argue instead that self-play stability is governed by two distinct levers: a data-level gate that decides which proposer-generated tasks enter the training pool, and the reward signal that updates the policy on tasks already admitted.
  arXiv  Detail & Related papers  (2026-05-21T09:19:23Z)
• Governing What You Cannot Observe: Adaptive Runtime Governance for Autonomous AI Agents [0.0]
  Autonomous AI agents can remain fully authorized and still become unsafe as behavior drifts, adversaries adapt, and decision patterns shift without any code change.<n>We propose the textbfInformational Viability Principle: governing an agent reduces to estimating a bound on unobserved risk.<n>textbfRiskGate instantiates the framework with dedicated statistical estimators (KL divergence, segment-vs-rest $z$-tests, sequential pattern matching), a fail-secure monotonic pipeline, and a closed-loop Autopilot.
  arXiv  Detail & Related papers  (2026-04-27T16:46:15Z)
• Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors [48.881343993730844]
  Safety-aligned large language models (LLMs) are increasingly deployed in real-world pipelines.<n> adversaries can distribute backdoored checkpoints that behave normally under standard evaluation but jailbreak when a hidden trigger is present.<n>Recent post-hoc weight-editing methods offer an efficient approach to injecting such backdoors.
  arXiv  Detail & Related papers  (2026-04-14T06:48:33Z)
• Conformal Policy Control [50.46542384484142]
  We show how to use any safe reference policy as a probabilistic regulator for any optimized but untested policy.<n>Unlike conservative optimization methods, we do not assume the user has identified the correct model class.<n>Our experiments on applications ranging from natural language question answering to biomolecular engineering show that safe exploration is possible from the first moment of deployment.
  arXiv  Detail & Related papers  (2026-03-02T18:54:36Z)
• Conformal Tradeoffs: Guarantees Beyond Coverage [0.02648566468224904]
  Deployed conformal predictors are long-lived decision infrastructure operating over finite operational windows.<n> Marginal coverage does not determine deployment-facing quantities.<n>We provide a framework for operational certification and planning beyond coverage.
  arXiv  Detail & Related papers  (2026-02-20T07:58:25Z)
• LEC: Linear Expectation Constraints for False-Discovery Control in Selective Prediction and Routing Systems [95.35293543918762]
  Large language models (LLMs) often generate unreliable answers, while uncertainty methods fail to fully distinguish correct from incorrect predictions.<n>We address this issue through the lens of false discovery rate (FDR) control, ensuring that among all accepted predictions, the proportion of errors does not exceed a target risk level.<n>We propose LEC, which reinterprets selective prediction as a constrained decision problem by enforcing a Linear Expectation Constraint.
  arXiv  Detail & Related papers  (2025-12-01T11:27:09Z)
• Core Safety Values for Provably Corrigible Agents [2.6451153531057985]
  We introduce the first implementable framework for corrigibility, with provable guarantees in multi-step, partially observed environments.<n>Our framework replaces a single reward with five *structurally separate* utility heads.<n>For open-ended settings where adversaries can modify the agent, we prove that deciding whether an arbitrary post-hack agent will ever violate corrigibility is undecidable.
  arXiv  Detail & Related papers  (2025-07-28T16:19:25Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Distribution-free uncertainty quantification for classification under label shift [105.27463615756733]
  We focus on uncertainty quantification (UQ) for classification problems via two avenues. We first argue that label shift hurts UQ, by showing degradation in coverage and calibration. We examine these techniques theoretically in a distribution-free framework and demonstrate their excellent practical performance.
  arXiv  Detail & Related papers  (2021-03-04T20:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.