The Irreversibility Budget: Fleet-Level Risk Accounting and Admission Control for Agent Operating Systems

• URL: http://arxiv.org/abs/2609.00275v1
• Date: Mon, 31 Aug 2026 19:18:09 GMT
• Title: The Irreversibility Budget: Fleet-Level Risk Accounting and Admission Control for Agent Operating Systems
• Abstract summary: Current controls check one effect at a time, so a fleet of individually authorized agents can overdraw its principal's risk under a shared trigger.<n>We propose the irreversibility budget, a cumulative account of residual value-at-risk that a trusted runtime maintains for each principal.<n>Treating irreversibility as a first-class resource, the runtime charges each effect its residual loss below the agent and denies the marginal effect once the aggregate would overdraw the budget.
• Score: 1.8001251270607181
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fleets of LLM agents now externalize effects that cannot be fully undone: they move money, deploy code, delete data, and disclose information. Current controls check one effect at a time, so a fleet of individually authorized agents can overdraw its principal's risk under a shared trigger while every local gate stays correct. We propose the irreversibility budget, a cumulative account of residual value-at-risk that a trusted runtime maintains for each principal across agents, workflows, and tenants. Treating irreversibility as a first-class resource, the runtime charges each effect its residual loss below the agent and denies the marginal effect once the aggregate would overdraw the budget. Getting the price right is hard, because effects are heterogeneous, adversarially declared, and correlated. We perform a controlled study in which per-effect gates admit fleet-level overdraws of up to 48 times the tenant's risk limit while the budget holds every correctly charged run within that limit. Conservative, dependency-aware pricing remains the central open problem for a deployable design.

Related papers

• The Missing Boundary: How Autonomous Agents Lose Control [7.908421631813947]
  Existing studies often attribute such failures to adversarial instructions, malicious environments, or conflicting objectives.<n>We study this question by independently manipulating three factors: goal pressure, control degradation, and executable unsafe opportunity.<n>Across 1,800 unique trajectories, we find that neither degraded control nor unsafe opportunity alone produces substantial loss of control.
  arXiv  Detail & Related papers  (2026-09-10T03:02:27Z)
• FRAMES: Guarded and Dual-Objective Skill Evolution for Agents in Policy-Governed Enterprise Workflows [10.830716991438878]
  FRAMES is a closed-loop framework that cold-starts deployable skills from existing assets and then evolves them.<n>It attains the best accuracy-cost trade-off among baselines, with the same gains reproduced on tau-bench.
  arXiv  Detail & Related papers  (2026-08-03T06:46:47Z)
• BRA-Audit: Budgeted Runtime Auditing for LLM Multi-Agent Systems via Cumulative-Exposure Audit-Point Placement [5.4775608854724345]
  Multi-agent systems (LLM-MAS) solve complex tasks through specialized collaboration, but inter-agent dependencies can propagate hallucinated or malicious outputs into system-level failures.<n>We propose BRA-Audit, a budget-aware runtime auditing framework that models MAS execution as a dynamic dependency graph and formulates audit scheduling as audit-point placement under a fixed audit-call budget to minimize cumulative unchecked exposure.<n>Across structured coordination, complex reasoning, and open-ended tasks, BRA-Audit restores performance close to the clean setting, remains competitive with heavy guard methods and reduces end-to-end token consumption by (
  arXiv  Detail & Related papers  (2026-08-02T14:35:43Z)
• AdaTurn: Budget-Aware Test-Time Scaling for Active Visual Perception Agents [43.827500763103586]
  AdaTurn is a budget-aware framework for visual agent training.<n>FA-DAPO converts an over-budget event into a trainable final-decision step.<n>AdaTurn substantially improves low-budget accuracy, for example raising VisualProbe-Medium from 36.7% to 47.6% at four turns.
  arXiv  Detail & Related papers  (2026-07-16T04:07:22Z)
• OCELOT: Inference-Leakage Budgets for Privacy-Preserving LLM Agents [32.5466552510287]
  Leakage is cumulative, as individually innocuous releases accumulate across honest-but-curious or colluding sinks.<n>We present OCELOT, a runtime mediator that budgets how much an adversary's belief about a secret may improve.<n>OCELOT attains significantly lower leakage at higher task utility, resists adaptive injection, jailbreak, cumulative inference, and sink collusion.
  arXiv  Detail & Related papers  (2026-06-10T17:13:35Z)
• Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability [79.08785366532287]
  We propose a lightweight risk-gated reinforcement learning approximation for risk-sensitive control under partial observability.<n>We evaluate the approach in two safety-critical partially observable domains: automated glucose regulation and safety-constrained navigation.
  arXiv  Detail & Related papers  (2026-05-14T01:23:09Z)
• When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making [68.12864562049957]
  Embodied robotic systems increasingly rely on large language model (LLM)-based agents to support high-level reasoning.<n>This raises a fundamental question for embodied agents: when should the agent reason, and when should it act?<n>We propose RARRL (Resource-Aware Reasoning via Reinforcement Learning), a hierarchical framework for resource-aware orchestration of embodied agents.
  arXiv  Detail & Related papers  (2026-03-17T15:38:50Z)
• Sovereign-OS: A Charter-Governed Operating System for Autonomous AI Agents with Verifiable Fiscal Discipline [7.232369984903087]
  Sovereign-OS is a governance-first operating system that places every agent action under constitutional control.<n>A CEO (Strategist) decomposes goals into dependency-aware task DAGs.<n>A CFO gates each expenditure against budget caps, daily burn limits, and profitability floors via an auction-based bidding engine.<n>Workers operate under earned-autonomy governed by a dynamic TrustScore.
  arXiv  Detail & Related papers  (2026-03-14T16:25:31Z)
• Conformal Thinking: Risk Control for Reasoning on a Compute Budget [60.65072883773352]
  Reasoning Large Language Models (LLMs) enable test-time scaling, with dataset-level accuracy improving as the token budget increases.<n>We re-frame the budget setting problem as risk control, limiting the error rate while minimizing compute.<n>Our framework introduces an upper threshold that stops reasoning when the model is confident and a novel lower threshold that preemptively stops unsolvable instances.
  arXiv  Detail & Related papers  (2026-02-03T18:17:22Z)
• PRISM: Festina Lente Proactivity -- Risk-Sensitive, Uncertainty-Aware Deliberation for Proactive Agents [11.440330367799513]
  PRISM is a novel framework that couples a decision-theoretic gate with a dual-process reasoning architecture.<n>Inspired by festina lente (Latin: "make haste slowly"), we gate by an acceptance-calibrated, cost-derived threshold.<n>On ProactiveBench, PRISM reduces false alarms by 22.78% and improves F1 by 20.14% over strong baselines.
  arXiv  Detail & Related papers  (2026-02-02T01:56:29Z)
• Criticality and Safety Margins for Reinforcement Learning [53.10194953873209]
  We seek to define a criticality framework with both a quantifiable ground truth and a clear significance to users.<n>We introduce true criticality as the expected drop in reward when an agent deviates from its policy for n consecutive random actions.<n>We also introduce the concept of proxy criticality, a low-overhead metric that has a statistically monotonic relationship to true criticality.
  arXiv  Detail & Related papers  (2024-09-26T21:00:45Z)
• Safety Margins for Reinforcement Learning [53.10194953873209]
  We show how to leverage proxy criticality metrics to generate safety margins. We evaluate our approach on learned policies from APE-X and A3C within an Atari environment.
  arXiv  Detail & Related papers  (2023-07-25T16:49:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.