FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents

• URL: http://arxiv.org/abs/2608.08570v1
• Date: Sun, 09 Aug 2026 08:22:57 GMT
• Title: FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents
• Abstract summary: FailForge is an agentic framework that converts failed rollouts into training signal.<n>FailForge recovers over 26% of previously failed instances at marginal additional cost.
• Score: 49.519184792774304
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Rejection sampling fine-tuning (RFT) is widely used to train code agents by generating trajectories on verifiable software engineering tasks, retaining those that pass the tests, and fine-tuning on the successful rollouts. However, even strong code agents repeatedly fail on a substantial fraction of such tasks, and standard RFT simply discards these failures. The discarded samples are precisely the hardest and most informative ones, drawn from verifiable instances that are costly to curate. Stronger base models may reduce the number of failures, but the remaining hard cases still define the frontier for further improvement. We propose FailForge, an agentic framework that converts failed rollouts into training signal. For each failed instance, an agent diagnoses the failure from error feedback and execution traces, distills the diagnosis into a concise and actionable skill, and injects the skill into the agent context for a guided second attempt. Trajectories that succeed under skill guidance are folded back into the RFT corpus. Crucially, the skill is removed at training time, so the model internalizes the recovered behavior rather than relying on external hints at inference. FailForge recovers over 26% of previously failed instances at marginal additional cost, and training Qwen3.5-4B on the augmented corpus improves the SWE-bench Verified resolve rate by 6.6 points over a strong RFT baseline, with gains concentrated on the hardest problems.

Related papers

• TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents [51.523913302114266]
  TRACE (Turn-level Reward Assignment via Credit Estimation) is a dense credit-assignment method for agentic reinforcement learning.<n>It represents rollouts as state transitions at tool-call boundaries, obtains gold-answer log-probabilities from a frozen reference model, transforms them into log-ratio state values, and derives per-action rewards as Temporal-Difference changes in those values.<n>It substantially improves base-model tool-use ability using pure RL, without a cold-start supervised fine-tuning stage, an agentic mid-training stage, or training on live-web data.
  arXiv  Detail & Related papers  (2026-07-15T16:16:42Z)
• Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents [80.03824805228719]
  A major challenge in developing computer-use agents is collecting large-scale, high-quality trajectories.<n>We propose a data-centric paradigm that turns failed trajectories into agent improvements.<n>We validate this approach with the state-of-the-art OpenCUA-72B model on the OSWorld benchmark.
  arXiv  Detail & Related papers  (2026-06-30T07:44:37Z)
• Failed Reasoning Traces Tell You What Is Fixable (But Not by Reading Them) [25.626296075137372]
  We propose that failed traces encode recoverability structure: the inference-time signature of which test-time interventions can rescue a given failure.<n>Three problem-level trajectory features, derived from the structure of available interventions, recover this structure from the distributional signature of failed rollouts.<n>The same three features thus convert failed traces from discarded data into a diagnostic object, supporting test-time routing and post-training analysis without training-time or weight-space access.
  arXiv  Detail & Related papers  (2026-06-03T17:50:26Z)
• TD-Grokking: Learning from Zero-Reward Problems by Training-Time Decomposition [10.882920019894451]
  We propose TD-Grokking, a training-time decomposition framework for zero-reward problems.<n>We show that TD-Grokking outperforms vanilla GRPO as well as all baseline approaches.
  arXiv  Detail & Related papers  (2026-06-03T06:40:04Z)
• Trace2Skill: Verifier-Guided Skill Evolution for Long-Context EDA Agents [0.3733676450456031]
  We present Trace2Skill, a test-time scaling framework that improves a hardware agent without RTL-specialized model fine-tuning.<n>Rather than training a new model or only sampling more candidate solutions, Trace2Skill treats the agent's natural-language skill as an evolvable policy.<n>It mines repeated rollout traces for success and failure modes, converts them into dense diagnostics and oracle lessons, and uses an oracle, mutator, and selector loop to produce task-specific skills.
  arXiv  Detail & Related papers  (2026-05-20T23:10:49Z)
• Towards Robust LLM Post-Training: Automatic Failure Management for Reinforcement Fine-Tuning [44.75827047453428]
  We propose an automatic failure management framework for reinforcement fine-tuning.<n>RFT-FaultBench is the first benchmark for fine-grained failures in reinforcement fine-tuning.<n>RFT-FM shows strong capability in detecting, diagnosing, and mitigating RFT failures.
  arXiv  Detail & Related papers  (2026-05-06T02:50:00Z)
• Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis [68.28714988482703]
  Process Reward Models (PRMs) have achieved remarkable success in augmenting the reasoning capabilities of Large Language Models (LLMs)<n>We present a study revealing that general-domain PRMs struggle to supervise data analysis agents.<n>We introduce DataPRM, a novel environment-aware generative process reward model.
  arXiv  Detail & Related papers  (2026-04-27T09:00:30Z)
• AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents [50.481033105867205]
  We introduce AgentProcessBench, the first benchmark dedicated to evaluating step-level effectiveness in realistic, tool-augmented trajectories.<n>The benchmark comprises 1,000 diverse trajectories and 8,509 human-labeled step annotations with 89.1% inter-annotator agreement.<n>It features a ternary labeling scheme to capture exploration and an error propagation rule to reduce labeling ambiguity.
  arXiv  Detail & Related papers  (2026-03-15T16:13:58Z)
• R$^3$L: Reflect-then-Retry Reinforcement Learning with Language-Guided Exploration, Pivotal Credit, and Positive Amplification [44.99719889905381]
  Reinforcement learning drives recent advances in reasoning and agentic capabilities, yet current approaches struggle with both exploration and exploitation.<n>We propose R$3$L, Reflect-then-Retry Reinforcement Learning with Language-Guided Exploration, Pivotal Credit, and Positive Amplification.<n>Experiments on agentic and reasoning tasks demonstrate 5% to 52% relative improvements over baselines while maintaining training stability.
  arXiv  Detail & Related papers  (2026-01-07T09:04:52Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• Exploring Expert Failures Improves LLM Agent Tuning [74.0772570556016]
  We propose Exploring Expert Failures (EEF), which identifies beneficial actions from failed expert trajectories.<n>EEF successfully solves some previously unsolvable subtasks and improves agent tuning performance.
  arXiv  Detail & Related papers  (2025-04-17T17:53:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.