AgentBrew: Offline Tool-Use Agent Learning from Raw Real-World Trajectories

• URL: http://arxiv.org/abs/2609.05837v1
• Date: Sat, 05 Sep 2026 03:04:23 GMT
• Title: AgentBrew: Offline Tool-Use Agent Learning from Raw Real-World Trajectories
• Abstract summary: Real-world applications provide no pre-defined tasks or verifiers, no faithful simulators, and limited budget for large-scale environment interaction.<n>We propose textbfAgentBrew, an offline training framework that learns effective tool-use policies from a single batch of raw interaction trajectories.<n>AgentBrew improves Qwen3-32B by +8.7 Acc / +9.7 Score on average, surpassing Qwen3-235B (+2.3 / +4.4) and outperforming rejection sampling.
• Score: 36.7302669545557
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-based agents are increasingly deployed in real-world applications through tool-use APIs, yet training them for specific environments remains fundamentally difficult: real-world applications provide no pre-defined tasks or verifiers, no faithful simulators, and limited budget for large-scale environment interaction. In this paper, we propose \textbf{AgentBrew}, an offline training framework that learns effective tool-use policies from a single batch of raw interaction trajectories, without task verifiers or iterative on-policy rollouts. The agent first explores the target environment to collect a raw trajectory corpus without quality filtering. To extract training signal from this noisy corpus, \emph{retrospective task inference} reconstructs an aligned instruction for each trajectory based on its actual outcome, and \emph{PMI-Based credit assignment} decomposes the trajectory's total information about the inferred instruction into additive per-action credits via pointwise mutual information (PMI). These credits weight the policy training objective, amplifying informative actions while suppressing ineffective ones. On three real-world MCP applications (GitHub, Notion, PostgreSQL), AgentBrew improves Qwen3-32B by +8.7 Acc / +9.7 Score on average, surpassing Qwen3-235B (+2.3 / +4.4) and outperforming rejection sampling (+5.9 / +10.3). These results demonstrate that fine-grained offline learning can recover useful supervision from raw trajectories that filtering-based approaches would discard. The code is available at https://github.com/alphatogo/AgentBrew

Related papers

• PRIMAL3: Pathfinding via Reinforcement and Imitation Multi-Agent Learning - Leveraging LaCAM3 [19.66694371656982]
  PRIMAL3 is an ultra-large-scale framework for multi-agent pathfinding (MAPF)<n>It integrates reinforcement learning, topology-aware communication, LaCAM3-guided training, and PIBT-based action refinement.<n> Experiments demonstrate that PRIMAL3 substantially outperforms state-of-the-art learning-based baselines.
  arXiv  Detail & Related papers  (2026-08-05T14:33:59Z)
• Retrieval is Enough: Training-Free Interpretability with a Tool-Using Agent [56.93673666934672]
  Training-based methods are typically more powerful, in part because they leverage large activation datasets during training.<n>This raises a natural question - do they actually surface insights that go beyond what is recoverable from the training dataset itself?<n>We equip an LLM agent with a vector database of activations paired with their textual contexts, along with tools for manipulating activations.<n>The agent iteratively queries the database, forms hypotheses from the retrieved samples, and validates them by constructing linear probes.
  arXiv  Detail & Related papers  (2026-07-17T18:47:31Z)
• Beyond Action Residuals: Real-World Robot Policy Steering via Bottleneck Latent Reinforcement Learning [42.74142065376427]
  We propose Z-Perturbation Reinforcement Learning (ZPRL) to steer pretrained policies through a compact bottleneck latent.<n>In the real world, ZPRL improves the average success rate on four tasks by 33.7% over imitation base policies.
  arXiv  Detail & Related papers  (2026-05-19T14:43:26Z)
• What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents [70.6980022118038]
  Reinforcement learning can train LLM agents from sparse task rewards, but long-horizon credit assignment remains challenging.<n>Existing methods rely on trajectory-level rewards or proxy signals, without fully leveraging per-step environmental feedback.<n>We introduce SERL, a selective environment-reweighted learning framework.
  arXiv  Detail & Related papers  (2026-05-19T07:00:55Z)
• Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents [52.30603055218294]
  Trajectory2Task is a verifiable data generation pipeline for studying tool use at scale under three realistic user scenarios.<n>It converts valid tool-call trajectories into user-facing tasks with controlled intent adaptations.<n>We benchmark seven state-of-the-art LLMs on the generated complex user scenario tasks and observe frequent failures.
  arXiv  Detail & Related papers  (2026-01-28T00:36:13Z)
• Emerging from Ground: Addressing Intent Deviation in Tool-Using Agents via Deriving Real Calls into Virtual Trajectories [24.35781391952214]
  "Real-to-Virtual" method designed to mitigate intent deviation.<n>We introduce RISE, a "Real-to-Virtual" method designed to mitigate intent deviation.
  arXiv  Detail & Related papers  (2026-01-21T15:58:54Z)
• ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibration [68.89572566071575]
  ETAgent is a training framework for calibrating agent's tool-use behavior.<n>It is designed to progressively calibrate erroneous behavioral patterns to optimal behaviors.
  arXiv  Detail & Related papers  (2026-01-11T11:05:26Z)
• Solving Robotics Tasks with Prior Demonstration via Exploration-Efficient Deep Reinforcement Learning [0.688204255655161]
  This paper proposes an exploration-efficient Deep Reinforcement Learning with Reference policy (DRLR) framework for learning robotics tasks that incorporates demonstrations.<n>The DRLR framework is developed based on an algorithm called Imitation Bootstrapped Reinforcement Learning (IBRL)
  arXiv  Detail & Related papers  (2025-09-04T10:02:32Z)
• Reinforcement Learning for Long-Horizon Interactive LLM Agents [56.9860859585028]
  Interactive digital agents (IDAs) leverage APIs of stateful digital environments to perform tasks in response to user requests.<n>We present a reinforcement learning (RL) approach that trains IDAs directly in their target environments.<n>We derive LOOP, a data- and memory-efficient variant of proximal policy optimization.
  arXiv  Detail & Related papers  (2025-02-03T18:35:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.