CRAFT: Learn the Schema, Execute the Plan

• URL: http://arxiv.org/abs/2607.22642v1
• Date: Wed, 24 Jun 2026 00:12:03 GMT
• Title: CRAFT: Learn the Schema, Execute the Plan
• Abstract summary: We present CRAFT, a two-stage post-training recipe for schema-grounded coding agents.<n>First, schema-stripped PLAN supervised fine-tuning learns domain-structured plans and executable behaviors from validated trajectories.<n>Second, execution-shaped reinforcement learning aligns the policy for tool selection, code quality, plan-code consistency, and recovery from failed executions.
• Score: 1.4711904351495848
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Enterprise coding agents translate natural-language analytical requests into executable code over proprietary APIs, schemas, and metric definitions. Yet the prevailing deployment pattern injecting exhaustive schema and tool documentation into each prompt increases inference overhead, complicates schema evolution, and undermines reliability in multi-turn analysis. We investigate whether stable schema knowledge and tool-use behavior can instead be acquired through post-training while preserving the consistency required for production-facing analytics. We present CRAFT, a two-stage post-training recipe for schema-grounded coding agents. First, schema-stripped PLAN supervised fine-tuning learns domain-structured plans and executable behaviors from validated trajectories without exhaustive prompt-time schema injection. Second, execution-shaped reinforcement learning aligns the policy for tool selection, code quality, plan-code consistency, and recovery from failed executions. Training trajectories are curated through a Tri-Gate filter combining execution validation, data-integrity checks, and LLM-judge reasoning audit. We evaluate CRAFT for planned rollout in advertising analytics, covering campaign performance analysis, metric drill-downs, entity-level performance analysis, and multi-turn analytical refinement. The enterprise evaluation environment incorporates beta APIs as the agent-facing tool surface and spans 25 schema-linked core entities and 30 agentic workflows. Relative to a schema-stuffed baseline, CRAFT improves composite Agent Score by +9.6 pp, consistency by +4.1 pp, and multi-turn coherence by +4.2 pp, while reducing input-token burden by approximately 9x and schema-discovery loops by up to 5x. We further report deployment tradeoffs, reward-shaping limitations, and training-infrastructure extensions required for multi-turn tool-use reinforcement learning in enterprise settings.

Related papers

• Graph-Based Agentic AI with LangGraph: Workflow Pathways for Long-Running Stateful Business Processes [0.0]
  We present three executable recipes to show how typed state, conditional routing, deterministic tools, retries, interrupts, checkpoints, and traces fit together.<n>LangGraph is positioned by workflow-complexity fit, not as a universal default.
  arXiv  Detail & Related papers  (2026-07-21T17:07:13Z)
• TIER: Trajectory-Invariant Execution Rewards for Multi-Step Tool Composition [62.56752617853322]
  Outcome-based rewards provide only sparse feedback, while trajectory-supervised rewards depend on annotated reference solutions.<n>We propose TIER: Trajectory-Invariant Execution Rewards, a reward framework that derives supervision directly from function schemas and runtime execution.
  arXiv  Detail & Related papers  (2026-05-16T03:47:26Z)
• Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning [79.88942231770629]
  Reinforcement learning (RL) has become a central post-training tool for improving the reasoning abilities of large language models (LLMs)<n>We formalize rollout pipelines with unified notation and introduce Generate-Filter-Control-Replay (GFCR)<n>We synthesize methods spanning RL with verifiable rewards, process supervision, judge-based gating, guided and tree/segment rollouts, adaptive compute allocation, early-exit and partial rollouts, throughput optimization, and replay/recomposition for self-improvement.
  arXiv  Detail & Related papers  (2026-04-08T00:53:29Z)
• Understanding by Reconstruction: Reversing the Software Development Process for LLM Pretraining [66.89012795621349]
  Large Language Models (LLMs) often struggle with the deep, long-horizon reasoning required for complex software engineering.<n>We propose a novel paradigm: understanding via reconstruction.<n>We introduce a framework that synthesizes latent agentic trajectories using a multi-agent simulation.
  arXiv  Detail & Related papers  (2026-03-11T09:23:20Z)
• Agentic Planning with Reasoning for Image Styling via Offline RL [66.10749901925941]
  Direct prompt-based editing fails on complex transformations because vague and subjective prompts often require nuanced understanding of what should be changed in the image.<n>We present a tool-based agentic RL post-training framework that addresses this through structured planning with chain-of-thought reasoning.
  arXiv  Detail & Related papers  (2026-03-07T11:14:37Z)
• Tool-Aware Planning in Contact Center AI: Evaluating LLMs through Lineage-Guided Query Decomposition [2.8180871881371456]
  We present a domain-grounded framework and benchmark for tool-aware plan generation in contact centers.<n>Our contributions are threefold: (i) a reference-based plan evaluation framework operating in two modes - a metric-wise evaluator and a one-shot evaluator, and (ii) a data methodology that iteratively refines plans via an evaluator->optimizer loop.
  arXiv  Detail & Related papers  (2026-02-16T17:36:05Z)
• GAP: Graph-Based Agent Planning with Parallel Tool Use and Reinforcement Learning [20.75113227786218]
  Graph-based Agent Planning (GAP) is a novel framework that explicitly models inter-task dependencies through graph-based planning.<n>Our approach trains agent foundation models to decompose complex tasks into dependency-aware sub-task graphs.<n>This dependency-aware orchestration achieves substantial improvements in both execution efficiency and task accuracy.
  arXiv  Detail & Related papers  (2025-10-29T09:35:55Z)
• CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning [67.18702329644526]
  CoT Referring enhances model reasoning across modalities through a structured, chain-of-thought training data structure.<n>We restructure the training data to enforce a new output form, providing new annotations for existing datasets.<n>We also integrate detection and segmentation capabilities into a unified MLLM framework, training it with a novel adaptive weighted loss to optimize performance.
  arXiv  Detail & Related papers  (2025-10-03T08:50:21Z)
• Schema Lineage Extraction at Scale: Multilingual Pipelines, Composite Evaluation, and Language-Model Benchmarks [3.3705400036304205]
  "Semantic drift" compromises data and governance, and impairs the utility of services like text-to-RAG.<n>This paper proposes a novel framework for the automated extraction of fine-grained schema lineage from multilingual enterprise pipeline scripts.<n>Result: A 32B open-source model, using a single reasoning trace, can achieve performance comparable to the GPT series under standard prompting.
  arXiv  Detail & Related papers  (2025-08-10T05:04:32Z)
• SPARE: Single-Pass Annotation with Reference-Guided Evaluation for Automatic Process Supervision and Reward Modelling [58.05959902776133]
  We introduce Single-Pass.<n>with Reference-Guided Evaluation (SPARE), a novel structured framework that enables efficient per-step annotation.<n>We demonstrate SPARE's effectiveness across four diverse datasets spanning mathematical reasoning (GSM8K, MATH), multi-hop question answering (MuSiQue-Ans), and spatial reasoning (SpaRP)<n>On ProcessBench, SPARE demonstrates data-efficient out-of-distribution generalization, using only $sim$16% of training samples compared to human-labeled and other synthetically trained baselines.
  arXiv  Detail & Related papers  (2025-06-18T14:37:59Z)
• Benchmarking Agentic Workflow Generation [80.74757493266057]
  We introduce WorfBench, a unified workflow generation benchmark with multi-faceted scenarios and intricate graph workflow structures.<n>We also present WorfEval, a systemic evaluation protocol utilizing subsequence and subgraph matching algorithms.<n>We observe that the generated can enhance downstream tasks, enabling them to achieve superior performance with less time during inference.
  arXiv  Detail & Related papers  (2024-10-10T12:41:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.