AgentBrew: Lifelong Knowledge Brewing from Strong Teachers to Weak LLM Agents

• URL: http://arxiv.org/abs/2607.16851v1
• Date: Sat, 18 Jul 2026 15:23:45 GMT
• Title: AgentBrew: Lifelong Knowledge Brewing from Strong Teachers to Weak LLM Agents
• Abstract summary: LLM agents typically require a compact test-time student, even if a stronger teacher is available during training.<n>This setting poses two challenges: environments provide only sparse, binary feedback, and teacher-authored notes must be inherently tailored to be concretely executable by a substantially weaker student.<n>We propose AgentBrew, comprising two coupled components. First, a failure-triggered teacher--Ralph Loops sparse feedback by transforming student failures into environment-validated notes.<n>Second, student-aware synthesis calibrates teacher knowledge to the weak executor's operational granularity, yielding model-specific, actionable guidance.
• Score: 8.847692192802343
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deploying LLM agents typically requires a compact test-time student, even if a stronger teacher is available during training. We study knowledge brewing: distilling a teacher's interactive experience into a persistent external memory for the student. Crucially, this requires no weight updates, expert demonstrations, ground-truth labels, or test-time teacher access. This setting poses two challenges: environments provide only sparse, binary feedback, and teacher-authored notes must be inherently tailored to be concretely executable by a substantially weaker student. To address these hurdles, we propose AgentBrew, comprising two coupled components. First, a failure-triggered teacher--Ralph Loop mitigates sparse feedback by transforming student failures into environment-validated notes. Second, student-aware synthesis calibrates teacher knowledge to the weak executor's operational granularity, yielding model-specific, actionable guidance. Extensive evaluations and comprehensive ablations across coding, math, and tool-use tasks demonstrate that this asymmetric, training-free brewing paradigm produces highly capable yet deployable LLM agents.

Related papers

• Multi-Turn On-Policy Distillation with Prefix Replay [73.10000453999088]
  We propose Replayed-Prefix On-Policy Distillation (ReOPD) for agentic tasks.<n>ReOPD reuses pre-collected teacher trajectories as replayed prefixes.<n>It preserves or improves OPD-level accuracy, uses zero tool calls during student training, and is at least 4$times$ faster per training step than OPD.
  arXiv  Detail & Related papers  (2026-07-06T07:56:53Z)
• Evaluating LLMs for Answering Student Questions in Introductory Programming Courses [0.0]
  This study investigates the capabilities of Large Language Models to safely and effectively assist educators in answering student questions within a CS1 programming course.<n>Because traditional text-matching metrics are insufficient for evaluating open-ended educational responses, we developed and validated a custom LLM-as-a-Judge metric.<n>Our findings demonstrate that models, such as Gemini 3 flash, can surpass the quality baseline of typical educator responses, achieving high alignment with expert pedagogical standards.
  arXiv  Detail & Related papers  (2026-03-30T11:22:58Z)
• Masking Teacher and Reinforcing Student for Distilling Vision-Language Models [50.619420197124356]
  Large-scale vision-language models (VLMs) have recently achieved remarkable multimodal understanding.<n>This raises the need for compact yet capable VLMs that can efficiently learn from powerful large teachers.<n>We propose Masters (Masking Teacher and Reinforcing Student), a mask-progressive reinforcement learning framework.
  arXiv  Detail & Related papers  (2025-12-23T14:40:38Z)
• AdaSwitch: Adaptive Switching Generation for Knowledge Distillation [58.647880811071495]
  Small language models (SLMs) are crucial for applications with strict latency and computational constraints.<n>We propose AdaSwitch, a novel approach that combines on-policy and off-policy generation at the token level.<n>AdaSwitch consistently improves accuracy, offering a practical and effective method for distilling SLMs with acceptable additional overhead.
  arXiv  Detail & Related papers  (2025-10-09T06:38:37Z)
• Distilling LLM Agent into Small Models with Retrieval and Code Tools [65.73762766854192]
  Agent Distillation is a framework for transferring reasoning capability and task-solving behavior from large language models into small language models.<n>Our results show that sLMs as small as 0.5B, 1.5B, 3B parameters can achieve performance competitive with next-tier larger 1.5B, 3B, 7B models.
  arXiv  Detail & Related papers  (2025-05-23T08:20:15Z)
• Alice: Proactive Learning with Teacher's Demonstrations for Weak-to-Strong Generalization [69.96794098855938]
  Weak-to-strong generalization (W2SG) offers a promising framework for supervising increasingly capable language models (LLMs)<n>Traditional W2SG methods rely on passive learning, where a weak teacher provides noisy demonstrations to train a strong student.<n>We introduce Alice, a framework that leverages complementary knowledge between teacher and student to enhance the learning process.
  arXiv  Detail & Related papers  (2025-04-09T22:33:06Z)
• Better than Your Teacher: LLM Agents that learn from Privileged AI Feedback [12.61197377492141]
  Large language models (LLMs) show impressive decision-making abilities. Current methods lack a mechanism for automatic self-improvement from errors during task execution. We propose LEAP, an iterative fine-tuning framework that continually improves LLM agents using feedback from AI expert teachers.
  arXiv  Detail & Related papers  (2024-10-07T18:55:53Z)
• Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarios [3.818273633647809]
  We propose a three-component framework leveraging three signal types. The first signal is the student's self-consistency (consistency of student multiple outputs), which is a proxy of the student's confidence. We show that our proposed two-stage framework brings a relative improvement of up to 20.79% compared to fine-tuning without any signals across datasets.
  arXiv  Detail & Related papers  (2024-06-08T02:17:43Z)
• Adaptive Teaching in Heterogeneous Agents: Balancing Surprise in Sparse Reward Scenarios [3.638198517970729]
  Learning from Demonstration can be an efficient way to train systems with analogous agents. However, naively replicating demonstrations that are out of bounds for the Student's capability can limit efficient learning. We present a Teacher-Student learning framework specifically tailored to address the challenge of heterogeneity between the Teacher and Student agents.
  arXiv  Detail & Related papers  (2024-05-23T05:52:42Z)
• Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts [81.37287967870589]
  We propose to harness a diverse set of specialized teachers, instead of a single generalist one, that collectively supervises the strong student. Our approach resembles the classical hierarchical mixture of experts, with two components tailored for co-supervision. We validate the proposed method through visual recognition tasks on the OpenAI weak-to-strong benchmark and additional multi-domain datasets.
  arXiv  Detail & Related papers  (2024-02-23T18:56:11Z)
• Large Language Model as a Policy Teacher for Training Reinforcement Learning Agents [16.24662355253529]
  Large Language Models (LLMs) can address sequential decision-making tasks through the provision of high-level instructions. LLMs lack specialization in tackling specific target problems, particularly in real-time dynamic environments. We introduce a novel framework that addresses these challenges by training a smaller, specialized student RL agent using instructions from an LLM-based teacher agent.
  arXiv  Detail & Related papers  (2023-11-22T13:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.