Regimes: An Auditable, Held-Out-Gated Improvement Loop Demonstrated on LongMemEval with ActiveGraph

• URL: http://arxiv.org/abs/2606.10241v1
• Date: Mon, 08 Jun 2026 23:04:35 GMT
• Title: Regimes: An Auditable, Held-Out-Gated Improvement Loop Demonstrated on LongMemEval with ActiveGraph
• Abstract summary: We show that an event-sourced agent runtime turns controlled improvement into a first-class workflow.<n>When the agent's state is a deterministic projection of an append-only event log, failures are recorded, a run replays exactly from its log, and candidate patches scope to typed pipeline seams, gates are auditable.<n>We demonstrate this with Regimes, a loop on the ActiveGraph runtime that diagnoses failed evaluations, proposes a repair at a pipeline point, and promotes it only after static checks, sandbox execution, in-sample evaluation, and held-out validation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous improvement loops are hard to trust because the improvement process is usually external scaffolding bolted onto the agent: failures go unlogged, diagnoses cannot be replayed, and promote-or-discard decisions land in a side database rather than the agent's own history. We show that an event-sourced agent runtime removes that friction and turns controlled improvement into a first-class workflow. When the agent's state is a deterministic projection of an append-only event log, failures are recorded, a run replays exactly from its log, candidate patches scope to typed pipeline seams, gates are auditable, and every promotion or discard is itself an event. We demonstrate this with Regimes, a loop on the ActiveGraph runtime that diagnoses failed evaluations, proposes a repair at a pipeline point, and promotes it only after static checks, sandbox execution, in-sample evaluation, and held-out validation. The loop is target-agnostic: the same control flow runs against different tasks through a common interface. On LongMemEval-S the dominant failure is not retrieval but reconciliation: the evidence is already in the assembled context, yet the reader answers incorrectly. Across five seeded held-out splits, Regimes discovers reader-prompt repairs that improve final held-out accuracy by +0.05 to +0.10 in four splits and +0.01 in one over-promotion split; two splits are individually significant (seed 5 unadjusted for its sequential promotion structure), and the pooled count is descriptive only, since the splits share one 500-question pool. The durable contributions are ActiveGraph as an auditable substrate that makes controlled improvement loops tractable, the held-out-gated loop it supports, the failure-regime taxonomy routing each failure to a pipeline location (whose marginal value over an unrouted baseline is the primary open question), and the prompt-as-discovery-probe hypothesis.

Related papers

• TROVE: Adaptive Agent Skill Orchestration via Trace-Grounded Route Validation and Editing [14.424308190797683]
  We propose Trace-grounded Route Orchestration via Validation and Editing (TROVE)<n> Offline, TROVE distills evaluated workflow-search traces into atomic and composite skills and an outcome-conditioned transition graph.<n>Online, it treats a planned route as provisional, after committing one top-level skill, the controller retains a valid continuation, inserts a trace-supported local response, or replaces only the invalid suffix.
  arXiv  Detail & Related papers  (2026-09-04T11:38:13Z)
• Disagree to Explore, Agree to Commit: Routing-Guided Test-Time Scaling for Software Agents [60.650808962786364]
  We study whether native MoE router traces can guide steering and selection without an external judge or selection-time test execution.<n>Our analysis shows that routing provides a robust behavioral role signal; token-granular readouts and decision-matched comparison sets turn it into effective control.<n>We evaluate on SWE-bench Verified using open-weight sparse MoE agents across scales and reasoning-effort settings.
  arXiv  Detail & Related papers  (2026-08-23T03:07:03Z)
• Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Framework [49.16055123488827]
  coding agents powered by large language models can now generate code patches directly from bug reports.<n>No mechanism independently verifies whether it truly resolves the reported problem.<n>We propose RETRACE, a training-free post-generation verification framework.
  arXiv  Detail & Related papers  (2026-08-09T22:59:13Z)
• OrchestraBench: Evaluating Multi-Agent Orchestration Failure Modes, Recovery, and Decomposition Quality [0.0]
  OrchestraBench evaluates failure, recovery, and decomposition through a controlled, seed-reproducible failure-injection harness over templated enterprise.<n>It introduces cascade radius and per-failure-mode recovery as primary metrics and compares routing policies with bootstrap confidence intervals and paired tests.
  arXiv  Detail & Related papers  (2026-08-05T17:27:15Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• What Resolve Rate Hides: Trajectory Structure Diagnostics for Coding Agents [14.988334585821939]
  Coding agents are ranked almost entirely by resolve rate.<n>A single pass/fail label says nothing about why a run failed or why an accepted run spent extra steps, time, or tokens.<n>We present TraceProbe, a trajectory-diagnostic framework that recovers what resolve rate hides.
  arXiv  Detail & Related papers  (2026-07-07T12:09:46Z)
• Self-Evolving Agents with Anytime-Valid Certificates [1.2691047660244335]
  We present textbfSEA, an architecture that confines self-modification to a small steering adapter and a versioned harness around a emphfrozen base model.<n>Five loop controllers compose published guarantees; because such gates can only emphselect among behaviors the frozen base already produces, five verifier-in-the-loop mechanisms supply the dense, grader-free signal the gates require.<n>Results are single-run on expensive evaluations; confirming run-to-run variance and adapting the per-task algorithm mix are future work.
  arXiv  Detail & Related papers  (2026-07-01T12:34:52Z)
• AuditWeave: A Tamper-Evident, Auditor-Navigable Evidence Layer for AI-Assisted and Data-Transformation Workflows [0.0]
  AuditWeave is a lightweight Python library that records the steps of AI-assisted and data-transformation into a single hash-chained ledger.<n>We describe the design and evaluate recording overhead, scalability, and tamper-detection correctness on the reference implementation.
  arXiv  Detail & Related papers  (2026-06-14T21:46:04Z)
• REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces [10.98846592145896]
  Large language model (LLM) agents solve complex tasks through long plan-and-execution traces, yet the ability to locate errors in a completed traces still lags far behind.<n>We propose methodname, a method that closes this gap by diagnosing a candidate error step, testing it through controlled replay with a diagnosis-specific patch, and using the verified outcome flip as contrastive evidence to refine the final attribution.
  arXiv  Detail & Related papers  (2026-06-08T06:11:57Z)
• Auditable Graph-Guided Root Cause Analysis for Kubernetes Incidents [1.116726665785374]
  We present Graph Traversal Agent, a graph-guided RCA agent that combines LLM reasoning with specialized tools.<n>We map operational constraints, including read-only evidence collection, propagation-aware diagnosis, bounded execution, and independently validated verdicts.<n>On ITBench snapshots scored by one fixed qwen-plus judge, the audited system raises root-cause-entity F1 over an earlier iteration of the same system.
  arXiv  Detail & Related papers  (2026-06-07T12:05:09Z)
• Self-Healing Agentic Orchestrators for Reliable Tool-Augmented Large Language Model Systems [0.10210859604701106]
  Self-healing agentic orchestrator treats reliability as a runtime bounded control problem.<n>Self-healing achieves 98.8% task success, compared with 94.5% for retry-only and 93.8% for full replanning.
  arXiv  Detail & Related papers  (2026-05-31T19:27:22Z)
• Benchmarking Recursive-Collapse Warning Claims Under Matched False-Positive Control [0.0]
  Recursive systems can enter collapse-like regimes before overt failure becomes visible.<n>We introduce Loopzero, a claim-bounded benchmark framework for testing whether failures follow a directional telemetry pattern.<n>We evaluate the bridge on two frozen public-artifact benchmarks.
  arXiv  Detail & Related papers  (2026-05-29T20:12:42Z)
• TraceGraph: Shared Decision Landscapes for Diagnosing and Improving Agent Trajectories [61.51984029109709]
  We introduce TraceGraph, a graph-based framework that turns released multi-model agent trajectories into shared decision landscapes.<n>For each task, TraceGraph builds a graph over observable action-observation states from pooled rollouts before model identity is introduced.<n>It then overlays outcome-informed productive cores and trap regions, and summarizes each rollout with three events: Access, Trap exposure, and Repair.
  arXiv  Detail & Related papers  (2026-05-29T13:40:31Z)
• Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RL [76.45061154544568]
  Self-play reinforcement learning trains language models on their own generated tasks, co-evolving a proposer and solver without human labels.<n>Recent systems report strong reasoning gains, but collapse and instability are widely observed and poorly understood.<n>We argue instead that self-play stability is governed by two distinct levers: a data-level gate that decides which proposer-generated tasks enter the training pool, and the reward signal that updates the policy on tasks already admitted.
  arXiv  Detail & Related papers  (2026-05-21T09:19:23Z)
• ExComm: Exploration-Stage Communication for Error-Resilient Agentic Test-Time Scaling [57.42714978834704]
  ExComm is a communication protocol for exploration-stage agentic test-time scaling.<n>We show that ExComm consistently outperforms strong test-time scaling baselines.
  arXiv  Detail & Related papers  (2026-05-21T07:38:44Z)
• From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents [56.31499185764872]
  Supervised fine-tuning (SFT) on long teacher trajectories is the dominant way to instill investigation and reasoning in open software-engineering (SWE) agents.<n>We propose Patches-to-Trajectories (P2T), which uses $pstar$ as privileged information during curation and formulates trajectory construction as bi-objective optimization over per-step effectiveness and trajectory length.
  arXiv  Detail & Related papers  (2026-05-21T04:54:55Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.