TRACE: TRajectory Attribution for Automated Context Engineering

• URL: http://arxiv.org/abs/2608.09153v1
• Date: Mon, 10 Aug 2026 06:01:03 GMT
• Title: TRACE: TRajectory Attribution for Automated Context Engineering
• Abstract summary: We present TRACE, an automated feedback loop that mines historical agent trajectories to diagnose and remediate context failures.<n>Unlike model fine-tuning, TRACE operates on the context layer, enabling rapid iteration without retraining.<n>On 60 dissatisfaction traces spanning three complexity tiers, TRACE achieves 72.7% root cause attribution and 82% end-to-end fix effectiveness.
• Score: 2.332848751256598
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Production AI agents fail when their context sources -- system prompts, knowledge bases, tool descriptions, and procedural skills -- contain errors or gaps. Current maintenance relies on manual log review and ad-hoc debugging, creating a scalability bottleneck as interaction volume grows. We present TRACE (TRajectory Attribution for Automated Context Engineering), an automated feedback loop that mines historical agent trajectories to diagnose and remediate context failures. Our key insight is that trajectories are rich with implicit dissatisfaction signals -- user corrections, rephrasing, abandonment cues -- that reveal precisely where context sources failed, without explicit feedback collection. Unlike model fine-tuning, TRACE operates on the context layer, enabling rapid iteration without retraining. We make four contributions: (1) a trajectory mining framework that systematically extracts diagnostic information from historical agent executions; (2) multi-component causal attribution that extends textual gradients from monolithic prompt optimization to heterogeneous context sources (skills, knowledge bases, tools, prompts); (3) exploratory verification, where agents actively read context sources to distinguish content gaps requiring CREATE from stale content requiring UPDATE, achieving 96% operation accuracy; and (4) a reusable simulation methodology and verifiable benchmark addressing the absence of open datasets for context debugging, with a six-category fault taxonomy, ground truth annotations, and a cross-layer verification protocol. On 60 dissatisfaction traces spanning three complexity tiers (up to 16 execution nodes), TRACE achieves 72.7% root cause attribution and 82% end-to-end fix effectiveness, showing that over 80% of context-layer failures can be automatically diagnosed and remediated by mining historical trajectories, an overlooked resource in production systems.

Related papers

• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing [81.51044612408793]
  SkillAudit is a framework for evolving agent skills without ground-truth feedback.<n>To turn behavioral differences into edit guidance, SkillAudit uses Process-Aligned Contrastive Evaluation.<n>It routes edits through two pipelines: Refine removes noisy or irrelevant guidance from broadly useful skills, while Repair replaces passages that conflict with the task.
  arXiv  Detail & Related papers  (2026-06-12T08:20:09Z)
• TrajAudit: Automated Failure Diagnosis for Agentic Coding Systems [18.833806809109536]
  Agentic systems have been widely studied to automate software engineering jobs such as bug fixing.<n>Existing automated failure diagnosis approaches leverage task execution trajectories, yet their effectiveness degrades as trajectory length and complexity increase.<n>For repository-level coding tasks specifically, trajectories are laden with noise, such as redundant program structure and verbose code context.<n>We propose TrajAudit, the first failure diagnosis framework for repository-level coding trajectories.
  arXiv  Detail & Related papers  (2026-05-26T05:24:37Z)
• RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations [51.43574078961796]
  We introduce RepoMirage, a two-stage evaluation suite built on SWE-Bench Verified.<n>RepoMirage-Perturb applies three types of semantics-preserving repository-level perturbations, revealing a clear performance drop when correct solving requires broader context access.<n>We propose RepoAnchor, a structure-first prototype workflow that separates repository exploration from downstream problem solving.
  arXiv  Detail & Related papers  (2026-05-25T06:26:43Z)
• BLAgent: Agentic RAG for File-Level Bug Localization [2.2917707112773593]
  BLAgent is a novel agentic RAG framework for file-level bug localization.<n> BLAgent attains over 78% Top-1 accuracy with open-source models.<n> BLAgent improves end-to-end repair success by over 20%.
  arXiv  Detail & Related papers  (2026-05-18T07:20:13Z)
• Knowledge Graphs, the Missing Link in Agentic AI-based Formal Verification [1.9085643829335266]
  This work builds a verification-centric Knowledge Graph (KG) constructed from structured Intermediate Representations (IRs) extracted from the specification, RTL, and formal-tool feedback.<n>The KG links requirements, design hierarchy, signals, assumptions, and properties to provide traceable, design-grounded context for generation.<n> Evaluation across seven benchmark designs indicates that KG-based context retrieval improves specification-to-RTL grounding and consistently produces compilable SVAs.
  arXiv  Detail & Related papers  (2026-05-07T15:35:53Z)
• Bian Que: An Agentic Framework with Flexible Skill Arrangement for Online System Operations [19.829321356625428]
  Bian Que is an agentic framework for monitoring, alert response, and root cause analysis.<n>It was deployed on the e-commerce search engine of KuaiShou, the major short-video platform in China.<n>It reduces alert volume by 75%, achieves 80% root-cause analysis accuracy, and cuts mean time to resolution by over 50%.
  arXiv  Detail & Related papers  (2026-04-29T15:35:01Z)
• Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generation [52.3707788779464]
  We introduce a novel Jensen-Shannon Divergence driven method to Attribute Response to Context (ARC-JSD)<n>ARC-JSD enables efficient and accurate identification of essential context sentences without additional fine-tuning, gradient-calculation or surrogate modelling.<n> Evaluations on a wide range of RAG benchmarks, such as TyDi QA, Hotpot QA, and Musique, using instruction-tuned LLMs in different scales demonstrate superior accuracy and significant computational efficiency improvements.
  arXiv  Detail & Related papers  (2025-05-22T09:04:03Z)
• Learning to Filter Context for Retrieval-Augmented Generation [75.18946584853316]
  Generation models are required to generate outputs given partially or entirely irrelevant passages. FILCO identifies useful context based on lexical and information-theoretic approaches. It trains context filtering models that can filter retrieved contexts at test time.
  arXiv  Detail & Related papers  (2023-11-14T18:41:54Z)
• Gradient Imitation Reinforcement Learning for General Low-Resource Information Extraction [80.64518530825801]
  We develop a Gradient Reinforcement Learning (GIRL) method to encourage pseudo-labeled data to imitate the gradient descent direction on labeled data. We also leverage GIRL to solve all IE sub-tasks (named entity recognition, relation extraction, and event extraction) in low-resource settings.
  arXiv  Detail & Related papers  (2022-11-11T05:37:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.