TRACE: Business Rule-Grounded Reasoning Curriculum for Knowledge-Preserving Parametric Tool Retrieval in Enterprise LLMs

• URL: http://arxiv.org/abs/2607.22639v1
• Date: Mon, 22 Jun 2026 23:15:35 GMT
• Title: TRACE: Business Rule-Grounded Reasoning Curriculum for Knowledge-Preserving Parametric Tool Retrieval in Enterprise LLMs
• Abstract summary: Parametric retrieval enables LLMs to retrieve tools implicitly by assigning each API a unique virtual token and training the model to generate it via constrained beam search.<n>Toolsense shows that this regime has two critical drawbacks: it destroys parametric tool knowledge during training, and its beamsearch decoding is too slow for real-time deployment.<n>We introduce TRACE (Tool Retrieval via Augmented Chain-of-thought and Enterprise rules), a two-stage curriculum that resolves this dissociation.
• Score: 1.4680035572775534
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Parametric retrieval enables LLMs to retrieve tools implicitly by assigning each API a unique virtual token and training the model to generate it via constrained beam search. Toolsense shows that this regime has two critical drawbacks: it destroys parametric tool knowledge during training, and its beam-search decoding is too slow for real-time deployment. We introduce TRACE (Tool Retrieval via Augmented Chain-of-thought and Enterprise rules), a two-stage curriculum that resolves this dissociation. Stage 1 reuses the multi-format memorization SFT from ToolSense to seed tool knowledge with LoRA. Stage 2 is our core contribution: the model is trained to emit a thinking trace before producing a JSON list of tool tokens, using two data sources -- RRB pairs from ToolSense and queries synthesized to target business rules curated by domain experts -- both augmented with reasoning traces. This training objective preserves Stage 1 MCQ and QA probing accuracy while enabling single-beam greedy decoding at production latency. Evaluated on a combined enterprise catalog of 8,300+ tools across two enterprise product lines, TRACE training for Stage 2 not only preserves but improves tool understanding: MCQ accuracy gains +3.2 pp and QA probing gains +9 pp over Stage 1. On retrieval, TRACE achieves ~86% recall on Domain A and ~60% on Domain B -- compared to embedding baseline performance of ~27% & ~52% -- both with single-beam greedy decoding, making it directly deployable at production latency.

Related papers

• Forgotten in Weights, Recovered by Tools: Agentic Tool Unlearning for LLM Agents [54.050506620820784]
  Agentic Tool Unlearning (ATU) aims to reduce both parametric recall and tool-mediated recovery while preserving normal tool use for retained knowledge.<n>ATU achieves a better balance between target forgetting and retained utility, making unlearning more robust under tool-augmented agent deployment.
  arXiv  Detail & Related papers  (2026-08-21T18:34:48Z)
• Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancement [16.192937389387982]
  We propose AKBE (Agentic Knowledge Boundary Enhancement), an on-policy method that dynamically probes the model's intrinsic knowledge boundary.<n> Experiments on seven QA benchmarks demonstrate that AKBE improves task accuracy by +1.85 on average and reduces tool calls by 18% over standard agentic RL.
  arXiv  Detail & Related papers  (2026-05-26T12:42:23Z)
• ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning [91.51460129144654]
  We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling.<n>We show that ParaVT improves over the Qwen3-VL baseline by +7.9% on average, with PARA-GRPO lifting training-time format compliance from 0.13 to 0.64.
  arXiv  Detail & Related papers  (2026-05-19T18:01:26Z)
• SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understanding [41.98672557723593]
  We introduce SWEQA-Pro, a benchmark constructed from diverse, long-tail repositories with executable environments.<n>We also propose a scalable synthetic data pipeline that powers a two-stage training recipe: Supervised Fine-Tuning (SFT) followed by Reinforcement Learning from AI Feedback (RLAIF)<n> Empirically, a Qwen3-8B model trained with our recipe surpasses GPT-4o by 2.3 points on SWE-QA-Pro and substantially narrows the gap to state-of-the-art models.
  arXiv  Detail & Related papers  (2026-03-17T05:12:48Z)
• Machine Learning as a Tool (MLAT): A Framework for Integrating Statistical ML Models as Callable Tools within LLM Agent Workflows [0.152622865871084]
  We introduce Machine Learning as a Tool (MLAT), a design pattern in which pre-trained statistical machine learning models are exposed as callable tools within large language model (LLM) agent.<n>Unlike conventional pipelines that treat ML inference as a static preprocessing step, MLAT positions the model as a first-class tool alongside web search, database queries, and APIs.<n>We present PitchCraft, a pilot production system that converts discovery call recordings into professional proposals with ML-predicted pricing.
  arXiv  Detail & Related papers  (2026-02-15T20:00:28Z)
• ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibration [68.89572566071575]
  ETAgent is a training framework for calibrating agent's tool-use behavior.<n>It is designed to progressively calibrate erroneous behavioral patterns to optimal behaviors.
  arXiv  Detail & Related papers  (2026-01-11T11:05:26Z)
• VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use [78.29315418819074]
  We introduce VerlTool, a unified and modular framework that addresses limitations through systematic design principles.<n>Our framework formalizes ARLT as multi-turn trajectories with multi-modal observation tokens (text/image/video), extending beyond single-turn RLVR paradigms.<n>The modular plugin architecture enables rapid tool integration requiring only lightweight Python definitions.
  arXiv  Detail & Related papers  (2025-09-01T01:45:18Z)
• RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Use [50.52940111891476]
  Large language models excel at basic reasoning but struggle with tasks that require interaction with external tools.<n>We present RLFactory, a plug-and-play reinforcement learning framework for multi-round tool use.
  arXiv  Detail & Related papers  (2025-08-31T16:47:31Z)
• R1-Code-Interpreter: LLMs Reason with Code via Supervised and Multi-stage Reinforcement Learning [23.795932850992816]
  We present R1-Code-Interpreter, an extension of a text-only Large Language Models (LLMs) trained via multi-turn supervised fine-tuning (SFT) and reinforcement learning (RL)<n>We show that training a general-purpose Code Interpreter across 144 diverse reasoning and planning tasks presents significant challenges due to task heterogeneity and scarcity of effective samples.<n>Our final model, R1-CI-14B, improves average accuracy on the 37 test tasks from 44.1% to 72.4%, outperforming text-only GPT-4o (58.6%) and GPT-4o with Code Interpreter (70.9%).
  arXiv  Detail & Related papers  (2025-05-27T18:47:33Z)
• ReTool: Reinforcement Learning for Strategic Tool Use in LLMs [27.07998056454784]
  ReTool enhances long-form reasoning with tool-integrated learning.<n>Model achieves 67% accuracy with 400 training steps.<n>Remarkably, ReTool-32B attains 72.5% accuracy in extended settings.
  arXiv  Detail & Related papers  (2025-04-15T18:10:22Z)
• TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use [72.32614703504122]
  Large language models (LLMs) achieve remarkable advancements by leveraging tools to interact with environments.<n>Standard supervised fine-tuning approach, which relies on large-scale datasets, often overlooks task-specific characteristics in tool use.<n>We propose TL-Training, a task-feature-based framework that mitigates the effects of suboptimal training data.
  arXiv  Detail & Related papers  (2024-12-20T02:21:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.