TraceCompiler: Skill-Guided Mining and Compilation of LLM Agent Traces into Mostly Deterministic Workflows

• URL: http://arxiv.org/abs/2608.02680v1
• Date: Mon, 03 Aug 2026 01:05:06 GMT
• Title: TraceCompiler: Skill-Guided Mining and Compilation of LLM Agent Traces into Mostly Deterministic Workflows
• Abstract summary: We present TraceCompiler, a skill-guided system that compiles noisy traces into trajectories.<n>Every hard edge carries an auditable evidence, and ambiguous relations are marked suspected.<n>We measure call reduction but not offline compilation cost, so we claim no net efficiency result.
• Score: 2.435006380732194
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tool-using language-model agents repeatedly rediscover procedures they have already executed, producing traces that mix reusable structure with retries, exploration, accidental ordering, and repeated lookups. We present TraceCompiler, a skill-guided system that mines clusters of noisy agent traces and compiles them into executable, mostly deterministic workflows. It admits an inter-tool dependency only when a consumer argument contains a value attributable uniquely to an earlier producer; every hard edge carries an auditable evidence tuple, and ambiguous relations are marked suspected and impose no ordering constraint. Bindings are classified as constants, user inputs, copied outputs, transforms, or residual LLM decisions. On T1, a mechanized form of the rule recovers producer-consumer dependencies at 0.928 precision and 0.943 recall over 15,775 def-use edges of its training split, against 0.711 F1 for adjacency and 0.712 for a frequency-thresholded directly-follows measure on identical data; the compiler skill run blind reaches 0.992 on 250 of those edges. On AppWorld we replay released trajectories in the deterministic simulator to recover masked return values and measure the rule against 563 token edges at 0.993 precision - a self-consistency check, since replay injects tokens by a related heuristic. We compile two recurring intents: a Venmo money-request intent reduces 34 observed API calls to 11 runtime calls and, under leave-one-out execution against the benchmark's own state tests, passes 15 of 21, the failing fold escalating rather than acting because its required branch was never observed; and a Spotify/Todoist intent the compiler correctly refuses to compile, because an irreversible side effect is under-determined. We measure call reduction but not offline compilation cost, so we claim no net efficiency result.

Related papers

• Guardrailed Meta-Agent Loops: Stress-Testing Policy Pinning, Budget Bounds, and Crash Recovery [7.414694666903067]
  GuardrailLoop is a simulation-based testbed that makes three operational contracts jointly testable.<n>A hash-pinned policy fixes goals, scope, evaluation identity, budget, and release conditions.<n>It separates useful adaptation, state recovery, and repeated execution.
  arXiv  Detail & Related papers  (2026-09-10T21:22:08Z)
• Threshold Choice, Not Sample Size, Bounds Trustless Verification of Nondeterministic Compound AI Workflows [0.0]
  Compound AI pipelines chain LLM calls, retrievers, and tools are nondeterministic.<n>We give a protocol covering all three: it commits digests of each stage's inputs, outputs, and context.<n>On a synthetic HotpotQA pipeline a calibrated fixed threshold accepts 44 of 45 honest reproductions and rejects 104 of 105 divergent pairs.
  arXiv  Detail & Related papers  (2026-09-07T22:10:54Z)
• Parsing the Stream: A Live Trace Model for Long-Horizon Agents and Their Observers [4.008486665405814]
  A long-horizon agent's trace outgrows both of its consumers: the human observer monitoring the run, and the agent itself.<n>We present a live trace model, an append-only event ledger folded incrementally into typed run state, and evaluate it for both consumers.<n>For the observer side, evaluated with an LLM reader as proxy, the compiled view answers monitoring questions using approximately 14x and 15x fewer input tokens.
  arXiv  Detail & Related papers  (2026-09-01T16:03:54Z)
• The Recall Trap: A Recall-Maximizing Retriever Configuration Reduces Issue Resolution in Fixed-Budget Code Context [45.88028371034407]
  We report a controlled case study in code repair, not a new phenomenon.<n>We toggle one flag (one-chunk-perfile deduplication) on an otherwise identical stack.<n>We map where it holds: it reverses on a lexical BM25 (3.2pp, significant cross-paradigm interaction)<n>We show that at a tight fixed budget, do not hard-deduplicate by file, and A/B packing policies against the task, not the metric flag was tuned.
  arXiv  Detail & Related papers  (2026-08-14T19:22:50Z)
• Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees [13.145367841870412]
  Computer-use agents often fail on transient GUI events because they produce the correct action only after the relevant window has closed.<n>We propose Adaptive Anticipatory Policy Trees (AAPT) which eliminates this delay without modifying the underlying model.<n>A lightweight observer matches change-gated frames to a prepared branch and immediately executes the corresponding action without generating new text.
  arXiv  Detail & Related papers  (2026-07-30T15:50:10Z)
• Operational Proto-Introspection in Looped Language Models: Process-Quality Taps, Executable Branching, and the Readout-Control Boundary [0.0]
  Can a language model read the quality of its ongoing computation?<n>Can an external intervention turn that readout into better outcomes?<n>We test both questions in a frozen 2.6B looped transformer, Ouro-RLTT.
  arXiv  Detail & Related papers  (2026-07-20T22:40:36Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• Auto: The AGI Compiler [0.0]
  Auto is a compiler that records live agent behavior, measures which parts are secretly deterministic, extracts them into verified programs or distilled specialists, and emits cognition binaries.<n>We use "AGI compiler" in one narrow, testable sense: a system that autonomously converts novel experience into permanent, verified, near-free skill while measuring what it does not know.
  arXiv  Detail & Related papers  (2026-07-05T23:09:24Z)
• Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents [99.67681154381803]
  Test-time Rule Acquisition and Compiled Enforcement (TRACE) is a skill-layer pipeline for coding-agent runtimes.<n>Unlike runtime checks written ahead of time by developers, TRACE skills come from the user's own chat corrections.
  arXiv  Detail & Related papers  (2026-06-11T10:43:40Z)
• Trace2Policy: From Expert Behavior Traces to Self-Evolving Decision Agents [5.689042186242701]
  Decision rules that enterprise experts apply tacitly can be systematically recovered and improved through iterative error analysis.<n>We present textbfTrace2Policy, whose core mechanism is textbfEISR.<n>Each round executes the rules on a validation set, clusters errors by root cause into MISSING, WRONG, or CONFLICT types, applies targeted patches, and commits only those that pass a regression gate.
  arXiv  Detail & Related papers  (2026-06-09T06:05:29Z)
• Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures [0.0]
  Causal Agent Replay (CAR) models an agent run as a structural causal model.<n>It applies a do-operation to a step, and re-executes a trajectory forward under the same policy.<n>CAR is open source and runs on hosted or free local models.
  arXiv  Detail & Related papers  (2026-06-06T17:44:23Z)
• Benchmarking Recursive-Collapse Warning Claims Under Matched False-Positive Control [0.0]
  Recursive systems can enter collapse-like regimes before overt failure becomes visible.<n>We introduce Loopzero, a claim-bounded benchmark framework for testing whether failures follow a directional telemetry pattern.<n>We evaluate the bridge on two frozen public-artifact benchmarks.
  arXiv  Detail & Related papers  (2026-05-29T20:12:42Z)
• Reliable Extraction of Clinical Follow-Up Instructions: A Hybrid Neural-Symbolic Pipeline [43.0484058393522]
  We test a hybrid neural-symbolic pipeline against direct generation.<n>We evaluate on a 2,000-note synthetic outpatient corpus with action-disjoint splits.
  arXiv  Detail & Related papers  (2026-05-26T05:14:33Z)
• Constraint-Guided Multi-Agent Decompilation for Executable Binary Recovery [47.704311990064554]
  Decompilation is essential for security analysis, malware reverse engineering, and legacy software maintenance.<n>We present a multi-agent framework that transforms decompiled code into re-executable source through Multi-level Constraint-Guided Decompilation (MCGD)<n>Our framework achieves 84-97% re-executability, improving baseline decompiler output by 28-89 percentage points.
  arXiv  Detail & Related papers  (2026-04-27T01:28:11Z)
• AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents [50.481033105867205]
  We introduce AgentProcessBench, the first benchmark dedicated to evaluating step-level effectiveness in realistic, tool-augmented trajectories.<n>The benchmark comprises 1,000 diverse trajectories and 8,509 human-labeled step annotations with 89.1% inter-annotator agreement.<n>It features a ternary labeling scheme to capture exploration and an error propagation rule to reduce labeling ambiguity.
  arXiv  Detail & Related papers  (2026-03-15T16:13:58Z)
• SemGuard: Real-Time Semantic Evaluator for Correcting LLM-Generated Code [46.20378145112059]
  Post-hoc repair pipelines detect such faults only after execution.<n>We present SemGuard, a semantic-evaluator-driven framework that performs real-time, line-level semantic supervision.
  arXiv  Detail & Related papers  (2025-09-29T09:21:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.