Learning on the Job: Continual Learning from Deployment Feedback for Frozen-Weights Agents

• URL: http://arxiv.org/abs/2607.22157v1
• Date: Fri, 24 Jul 2026 10:01:00 GMT
• Title: Learning on the Job: Continual Learning from Deployment Feedback for Frozen-Weights Agents
• Abstract summary: AI agents encounter learning opportunities in every episode they run, and discard nearly all of them.<n>We show that this feedback is a sufficient signal for continual learning when the frozen model is paired with an external memory.<n>We show that learning from the one-bit outcome verdict lifts single-trial success to 1.6$times$ the baseline, and learning from corrections to 2.6$times$, converting 22 of the 84 tasks the baseline never solves.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI agents encounter learning opportunities in every episode they run, and discard nearly all of them: the underlying models are frozen at deployment, so an agent that resolves a difficult request today starts from zero when it recurs tomorrow. Yet ordinary operation already produces feedback, in the form of outcome verdicts and after-the-fact corrections. We show that this feedback is a sufficient signal for continual learning when the frozen model is paired with an external memory that distils each episode into retrievable natural-language rules. On the banking domain of $τ$-bench, against a static-RAG control retrieving over the complete policy corpus, learning from the one-bit outcome verdict lifts single-trial success to 1.6$\times$ the baseline, and learning from corrections to 2.6$\times$, converting 22 of the 84 tasks the baseline never solves. The result spans the deployment spectrum, measured on Mistral Large, an open-weights model that organisations with data sovereignty requirements can self-host, and replicated on a frontier model, Claude Sonnet 5. The accumulated memory also transfers: each model, reading the store built by the other, rises above its own no-memory baseline. The harness, protocol, and data are released.

Related papers

• UpgradeBench: A Decision-Centric Benchmark for Upgrading Fine-Tuned LLM Specialists [10.23869027501654]
  We present Upgrade, a decision-driven longitudinal benchmark covering four consecutive Qwen releases.<n>Upgrade disentangles three core questions: whether a new checkpoint improves fixed-recipe retrained specialist performance, whether assets transfer across versions, and what recovery resources are usable.
  arXiv  Detail & Related papers  (2026-08-21T09:37:47Z)
• Training-Free Inference-Time Self-Reflection and Cost-Bounded Early Stopping for Large Language Models [15.961083103517872]
  EvoResearcher is a training-free, inference-time protocol that adds cost-bounded self-reflection to a single frozen LLM backbone.<n>Four self-reflective meta-reward components act as design principles instantiated as prompt-level mechanisms.
  arXiv  Detail & Related papers  (2026-08-19T13:09:57Z)
• Living-Harness Is an Interactive-Agent Evolver [15.081919387954578]
  Large language model (LLM) agents may recover from a failure within an episode or after a retry, yet the same execution failure can recur in later tasks.<n>We propose $textbfLiving-Harness, a self-evolving agent harness that converts each completed trajectory and its evaluator signals into posterior evidence for bounded harness updates.
  arXiv  Detail & Related papers  (2026-07-29T08:20:11Z)
• Procedural Memory Distillation: Online Reflection for Self-Improving Language Models [51.938248694868584]
  Reinforcement learning with verifiable rewards (RLVR) evaluates each rollout against a verifier and updates the policy from that episode-level signal.<n>Across episodes and epochs, the model repeatedly encounters related problems under a changing policy, producing cross-episode signals that episode-local updates cannot capture.<n>We propose Procedural Memory Distillation (PMD), which converts these crossepisode signals into reusable procedural memory and distills it into the policy's weights during training.
  arXiv  Detail & Related papers  (2026-07-01T21:20:57Z)
• Inverting the Bellman Equation: From $Q$-Values to World Models [57.827849584133425]
  We prove that value-based agents trained on a sufficiently rich set of reward functions implicitly encode a unique and accurate world model.<n>We find that policies trained exclusively on a ttReacher agent's implicit world model are quasi-optimal on out-of-distribution, velocity-based goals despite position-only training.
  arXiv  Detail & Related papers  (2026-06-19T07:26:14Z)
• Self-CTRL: Self-Consistency Training with Reinforcement Learning [45.9704217495657]
  Language models (LMs) that faithfully describe their own behavior can more easily be audited, understood, and trusted by users.<n>This paper describes Self-Consistency Training with Reinforcement Learning (Self-CTRL), a method that optimize for consistency between a LM's self-explanations and behavior on related inputs.
  arXiv  Detail & Related papers  (2026-06-16T17:59:40Z)
• Scaling Self-Evolving Agents via Parametric Memory [69.96398842169002]
  Existing memory-augmented LLM agents store past experience exclusively in prompt space.<n>We introduce textttTMEM, a self-evolving parametric memory framework.<n>textttTMEM consistently outperforms summary-based and retrieval-based baselines across different model scales.
  arXiv  Detail & Related papers  (2026-06-03T07:18:31Z)
• Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis [68.28714988482703]
  Process Reward Models (PRMs) have achieved remarkable success in augmenting the reasoning capabilities of Large Language Models (LLMs)<n>We present a study revealing that general-domain PRMs struggle to supervise data analysis agents.<n>We introduce DataPRM, a novel environment-aware generative process reward model.
  arXiv  Detail & Related papers  (2026-04-27T09:00:30Z)
• Probing for Knowledge Attribution in Large Language Models [45.47366023067617]
  Large language models (LLMs) often generate fluent but unfounded claims, or hallucinations.<n> Proper mitigation depends on knowing whether a model's answer is based on the prompt or its internal weights.<n>We show that a probe, a simple linear classifier trained on model hidden representations, can reliably predict contributive attribution.
  arXiv  Detail & Related papers  (2026-02-26T09:21:12Z)
• MERGETUNE: Continued fine-tuning of vision-language models [77.8627788911249]
  Fine-tuning vision-language models (VLMs) often leads to catastrophic forgetting of pretrained knowledge.<n>We introduce a novel paradigm, continued fine-tuning (CFT), which seeks to recover pretrained knowledge after a zero-shot model has already been adapted.
  arXiv  Detail & Related papers  (2026-01-15T15:15:53Z)
• Memento: Fine-tuning LLM Agents without Fine-tuning LLMs [36.3424780932712]
  We introduce a novel learning paradigm for Adaptive Large Language Model (LLM) agents.<n>Our method enables low-cost continual adaptation via memory-based online reinforcement learning.<n>We instantiate our agent model in the deep research setting, namely emphMemento, which attains top-1 on GAIA validation.
  arXiv  Detail & Related papers  (2025-08-22T07:25:30Z)
• Harmonic LLMs are Trustworthy [3.8119386967826294]
  We introduce an intuitive method to test the robustness of any black-box LLM in real-time via its local deviation from harmoniticity, denoted as $gamma$. We measure $gamma$ in 10 popular LLMs across thousands of queries in three objective domains: WebQA, ProgrammingQA, and TruthfulQA.
  arXiv  Detail & Related papers  (2024-04-30T17:00:32Z)
• Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning [54.585428241509234]
  We propose R$3$: Learning Reasoning through Reverse Curriculum Reinforcement Learning (RL) RL employs only outcome supervision to achieve the benefits of process supervision for large language models.
  arXiv  Detail & Related papers  (2024-02-08T16:46:26Z)
• ReLU Regression with Massart Noise [52.10842036932169]
  We study the fundamental problem of ReLU regression, where the goal is to fit Rectified Linear Units (ReLUs) to data. We focus on ReLU regression in the Massart noise model, a natural and well-studied semi-random noise model. We develop an efficient algorithm that achieves exact parameter recovery in this model.
  arXiv  Detail & Related papers  (2021-09-10T02:13:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.