Forgetting Without Restarting: Execution-State Unlearning for Stateful LLM Agents

• URL: http://arxiv.org/abs/2609.04875v1
• Date: Fri, 04 Sep 2026 08:31:40 GMT
• Title: Forgetting Without Restarting: Execution-State Unlearning for Stateful LLM Agents
• Abstract summary: Provenance-Guided Selective Replay attains bound as a cross-layer contract spanning prompt, compressed memory, and cache.<n>We audit with elicitation, cache, and string-free behavioral tests across three agent suites, nine baselines, and three model families.
• Score: 12.985773309621964
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-running LLM agents are stateful: beyond the transcript they accrete compressed summaries, plaintext memory, pending tool plans, and, under every serving API, a KV cache. Yet today's "forget" operations delete a plaintext memory record and stop, leaving every artifact derived from the revoked information intact. We formalize execution-state unlearning: after a forget request, the agent must behave as if it had never observed the target. Modeling the runtime as a deterministic transition system, we prove that the pre-target trajectory prefix is shared with this counterfactual world for free, that the post-target suffix is irreducibly tainted without token-level attribution, and that exact unlearning requires at least $T-τ+1$ recomputed transitions, where $τ$ is the target's injection step. Provenance-Guided Selective Replay attains this bound as a cross-layer contract spanning prompt, compressed memory, and cache: a provenance graph locates the injection point, checkpoint restoration reduces to cropping the KV cache, and sanitized replay regenerates the counterfactual suffix. Audited with elicitation, stochastic, and string-free behavioral tests across three agent suites, nine baselines, and three model families, memory deletion leaves leakage unchanged, instruction-based forgetting collapses under elicitation (Leak@probes = 1.00), and source redaction still acts on a revoked preference in 80% of episodes, while selective replay is indistinguishable from a full reset at up to 9x fewer recomputed tokens.

Related papers

• What to Forget in Unlearning? Forget Set Curation for Language Models [24.23267387545857]
  We show how a requester may ask a model to stop reproducing a song or book without knowing which spans, documents, quotations, or near-duplicates support that behavior.<n>We introduce CleanSlate, a benchmark for verbatim output suppression over songs and books, with model-specific extraction profiles, content-grounded QA, and capability-retention evaluations.
  arXiv  Detail & Related papers  (2026-08-14T19:51:53Z)
• From Faulty Memories to Corrected Actions: Dependency-Guided Rollback Repair for Memory-Augmented Agents [19.125716338799545]
  textbfdependency-guided rollback repair builds a typed memory-to-action graph from provenance, traces explicit downstream dependencies, and selectively replays only answer-relevant affected computation.<n>We evaluate this approach on a 150-case controlled benchmark spanning three tool-use domains and four memory failure types.
  arXiv  Detail & Related papers  (2026-08-11T05:19:55Z)
• Resume Means Resume: A Machine-Checked Conformance Contract for Checkpoint, Interrupt, and Resume Semantics in Workflow Persistence Layers [0.0]
  A framework that persists execution state so a run can be interrupted, survive a crash, and continue must decide what a resume means for effects that already happened.<n>Five widely deployed agent workflow frameworks answer differently, none exposes a machine-checkable contract.<n>LangGraph 1.2.9 records a second resume value and never consults it, persists-invalid state silently, and re-executes durably recorded work after a real SIGKILL.
  arXiv  Detail & Related papers  (2026-08-04T15:45:31Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation [72.54604107217669]
  textscpass@$k$ nearly vanishes after compression, yet textscpass@$k$ recovers substantially under repeated sampling.<n>Recovery should train on the compressed model's own on-policy states with dense token-level supervision.<n>We propose textbfshortopd, a short-to-long OPD schedule that detects teacher-confirmed repetitive suffixes.
  arXiv  Detail & Related papers  (2026-07-14T17:50:50Z)
• Deployment-Time Memorization in Foundation-Model Agents [18.40535884093806]
  We study agent memory as a privacy-utility frontier measured by Personalization Recall (PR) and Adrial Extraction Rate (AER)<n>Key-fact summarization reduces canary extraction by 76% on Gemma 3 12B and 64% on GPT-4o-mini while preserving nearly all personalization recall.<n>On LongMemEval, key-fact summarization reduces canary extraction by 76% on Gemma 3 12B and 64% on GPT-4o-mini while preserving nearly all personalization recall.
  arXiv  Detail & Related papers  (2026-06-08T18:38:41Z)
• Scaling Self-Evolving Agents via Parametric Memory [69.96398842169002]
  Existing memory-augmented LLM agents store past experience exclusively in prompt space.<n>We introduce textttTMEM, a self-evolving parametric memory framework.<n>textttTMEM consistently outperforms summary-based and retrieval-based baselines across different model scales.
  arXiv  Detail & Related papers  (2026-06-03T07:18:31Z)
• From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents [56.31499185764872]
  Supervised fine-tuning (SFT) on long teacher trajectories is the dominant way to instill investigation and reasoning in open software-engineering (SWE) agents.<n>We propose Patches-to-Trajectories (P2T), which uses $pstar$ as privileged information during curation and formulates trajectory construction as bi-objective optimization over per-step effectiveness and trajectory length.
  arXiv  Detail & Related papers  (2026-05-21T04:54:55Z)
• MemLineage: Lineage-Guided Enforcement for LLM Agent Memory [3.058164432275431]
  We introduce MemLineage, a defense for LLM agent memory that attaches cryptographic provenance and LLM-mediated lineage to every entry.<n>Recent and concurrent work shows that untrusted content can be written into persistent agent state and re-enter later sessions as an instruction.<n>MemLineage treats this as a chain-of-custody problem rather than a filtering problem.
  arXiv  Detail & Related papers  (2026-05-14T06:07:54Z)
• AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Management [47.49917373646469]
  Existing defenses treat bloated memory as given and focus on remaining resilient.<n>We present AgentSys, a framework that defends against indirect prompt injection through explicit memory management.
  arXiv  Detail & Related papers  (2026-02-07T06:28:51Z)
• Gumbel Reranking: Differentiable End-to-End Reranker Optimization [61.16471123356738]
  RAG systems rely on rerankers to identify relevant documents.<n> fine-tuning these models remains challenging due to the scarcity of annotated query-document pairs.<n>We propose Gumbel Reranking, an end-to-end training framework for rerankers aimed at minimizing the training-inference gap.
  arXiv  Detail & Related papers  (2025-02-16T13:23:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.