Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents

• URL: http://arxiv.org/abs/2606.22528v2
• Date: Sat, 27 Jun 2026 08:35:20 GMT
• Title: Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents
• Abstract summary: We show that in-context governance constraints reliably obey while visible can be silently removed by compaction.<n>Across 1,323 episodes, violation rises from 0% with the policy in full context to 30% after compaction.<n>We propose Constraint Pinning, a training-free mitigation that quarantines governance constraints from lossy compaction.
• Score: 0.4855342112648282
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern LLM agents increasingly rely on context compaction, summarization, or eviction to keep long-running sessions within a token budget. We show that this context-management layer is a safety-critical failure surface: in-context governance constraints that agents reliably obey while visible can be silently removed by compaction, causing the same agent to perform prohibited tool actions later in the session. We call this failure mode Governance Decay. We introduce ConstraintRot, a benchmark of long-horizon agent scenarios with deterministic tool-call grading, and measure compaction-induced violations across seven model families. Across 1,323 episodes, violation rises from 0% with the policy in full context to 30% after compaction, reaching 59% for some models; when the constraint survives the summary, violation remains 0%, but when it is dropped, violation reaches 38%. We further study a Compaction-Eviction Attack, in which adversarial in-context content biases the summarizer to omit a legitimate policy, and show that optimized injections defeat every evaluated model. Finally, we propose Constraint Pinning, a simple training-free mitigation that quarantines governance constraints from lossy compaction and restores violation to 0% in our benchmark. These results identify context management as a first-class governance surface for deployed LLM agents.

Related papers

• The Missing Boundary: How Autonomous Agents Lose Control [7.908421631813947]
  Existing studies often attribute such failures to adversarial instructions, malicious environments, or conflicting objectives.<n>We study this question by independently manipulating three factors: goal pressure, control degradation, and executable unsafe opportunity.<n>Across 1,800 unique trajectories, we find that neither degraded control nor unsafe opportunity alone produces substantial loss of control.
  arXiv  Detail & Related papers  (2026-09-10T03:02:27Z)
• Game-Theoretic Multi-Agent Control for Robust Contextual Reasoning in LLMs [6.335875360476155]
  Large Language Models (LLMs) in multi-turn interactions maintain evolving context rather than generating isolated responses.<n>We introduce the Game-Theoretic Secure Model Context Protocol (GT-MCP), a controller-driven multi-agent method that treats context management as a closed-loop dynamical process.<n>When instability is detected, a rollback-based self-healing mechanism restores the validated context and prevents unsupported fragments from propagating.
  arXiv  Detail & Related papers  (2026-06-09T02:18:44Z)
• Ghost in the Context: Measuring Policy-Carriage Failures in Decision-Time Assembly [0.0]
  LLM agents do not act on raw interaction history; they act on a bounded decision state assembled by truncation, summarization, reordering, and rewriting.<n>We study this failure mode over local Llama 3.1 8B, Qwen 2.5 7B, and Mistral 7B using judged exact constraint respect and direct audits of assembled-state visibility.
  arXiv  Detail & Related papers  (2026-05-02T18:07:42Z)
• Beyond Compaction: Structured Context Eviction for Long-Horizon Agents [0.0]
  We present Context Window Lifecycle (CWL), a context-management scheme that gives long-horizon LLM agents an effectively unbounded working horizon.<n>As a session accumulates history, CWL keeps the context within budget through graduated, semantically-aware eviction.<n>We describe the annotation protocol, the episode graph, the eviction policy, and the token-accounting loop, and evaluate CWL on long-horizon agentic benchmarks.
  arXiv  Detail & Related papers  (2026-05-01T18:39:02Z)
• PolicyBank: Evolving Policy Understanding for LLM Agents [51.86716874651299]
  PolicyBank maintains structured, tool-level policy insights and iteratively refines them.<n>PolicyBank closes up to 82% of the gap toward a human oracle.
  arXiv  Detail & Related papers  (2026-04-16T20:29:30Z)
• CCTU: A Benchmark for Tool Use under Complex Constraints [66.87622847854337]
  We introduce CCTU, a benchmark for evaluating large language models (LLMs) under complex constraints.<n>The benchmark comprises 200 carefully curated and challenging test cases across diverse tool-use scenarios.<n>We develop an executable constraint validation module that performs step-level validation and enforces compliance.
  arXiv  Detail & Related papers  (2026-03-16T14:05:13Z)
• Improving Search Agent with One Line of Code [68.58667107354253]
  Tool-based Agentic Reinforcement Learning (TARL) has emerged as a promising paradigm for training search agents to interact with external tools.<n>We propose textbfSearch textbfAgent textbfPolicy textbfOptimization (textbfSAPO), which stabilizes training via a conditional token-level KL constraint.
  arXiv  Detail & Related papers  (2026-03-10T04:07:39Z)
• Safe and Compliant Cross-Market Trade Execution via Constrained RL and Zero-Knowledge Audits [0.5586191108738564]
  We present a cross-market algorithmic trading system that balances execution quality with rigorous compliance enforcement.<n>The architecture comprises a high-level planner, a reinforcement learning execution agent, and an independent compliance agent.<n>We report effects at the 95% confidence level using paired t-tests and examine tail risk via CVaR.
  arXiv  Detail & Related papers  (2025-10-06T15:52:12Z)
• Criticality and Safety Margins for Reinforcement Learning [53.10194953873209]
  We seek to define a criticality framework with both a quantifiable ground truth and a clear significance to users.<n>We introduce true criticality as the expected drop in reward when an agent deviates from its policy for n consecutive random actions.<n>We also introduce the concept of proxy criticality, a low-overhead metric that has a statistically monotonic relationship to true criticality.
  arXiv  Detail & Related papers  (2024-09-26T21:00:45Z)
• Hallucinated Adversarial Control for Conservative Offline Policy Evaluation [64.94009515033984]
  We study the problem of conservative off-policy evaluation (COPE) where given an offline dataset of environment interactions, we seek to obtain a (tight) lower bound on a policy's performance. We introduce HAMBO, which builds on an uncertainty-aware learned model of the transition dynamics. We prove that the resulting COPE estimates are valid lower bounds, and, under regularity conditions, show their convergence to the true expected return.
  arXiv  Detail & Related papers  (2023-03-02T08:57:35Z)
• Penalized Proximal Policy Optimization for Safe Reinforcement Learning [68.86485583981866]
  We propose Penalized Proximal Policy Optimization (P3O), which solves the cumbersome constrained policy iteration via a single minimization of an equivalent unconstrained problem. P3O utilizes a simple-yet-effective penalty function to eliminate cost constraints and removes the trust-region constraint by the clipped surrogate objective. We show that P3O outperforms state-of-the-art algorithms with respect to both reward improvement and constraint satisfaction on a set of constrained locomotive tasks.
  arXiv  Detail & Related papers  (2022-05-24T06:15:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.