FORGE: Self-Evolving Agent Memory With No Weight Updates via Population Broadcast

• URL: http://arxiv.org/abs/2605.16233v1
• Date: Fri, 15 May 2026 17:42:49 GMT
• Title: FORGE: Self-Evolving Agent Memory With No Weight Updates via Population Broadcast
• Abstract summary: FORGE wraps a Reflexion-style inner loop, where a dedicated reflection agent converts failed trajectories into reusable knowledge artifacts.<n>We evaluate on CybORG CAGE-2 a network-defense POMDP at a 30-step horizon against the B-line attacker.<n>Compared against both a zero-shot baseline and a Reflexion baseline (isolated single-stream learning), FORGE improves average evaluation return by 1.7-7.7$times$ over zero-shot.
• Score: 3.774094352572544
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Can LLM agents improve decision-making through self-generated memory without gradient updates? We propose FORGE (Failure-Optimized Reflective Graduation and Evolution), a staged, population-based protocol that evolves prompt-injected natural-language memory for hierarchical ReAct agents. FORGE wraps a Reflexion-style inner loop, where a dedicated reflection agent (using the same underlying LLM, no distillation from a stronger model) converts failed trajectories into reusable knowledge artifacts: textual heuristics (Rules), few-shot demonstrations (Examples), or both (Mixed), with an outer loop that propagates the best-performing instance's memory to the population between stages and freezes converged instances via a graduation criterion. We evaluate on CybORG CAGE-2, a stochastic network-defense POMDP at a 30-step horizon against the B-line attacker, where all four tested LLM families (Gemini-2.5-Flash-Lite, Grok-4-Fast, Llama-4-Maverick, Qwen3-235B) exhibit strongly negative, heavy-tailed zero-shot rewards. Compared against both a zero-shot baseline and a Reflexion baseline (isolated single-stream learning), FORGE improves average evaluation return by 1.7-7.7$\times$ over zero-shot and by 29-72% over Reflexion in all 12 model-representation conditions, reducing major-failure rates (below $-100$) to as low as $\sim$1%. We find that (1) population broadcast is critical mechanism, with a no-graduation ablation confirming that broadcast carries the performance gains while graduation primarily saves compute; (2) Examples achieves the strongest returns for three of four models, Rules offers the best cost-reliability profile with $\sim$40% fewer tokens; and (3) weaker baseline models benefit disproportionately, suggesting FORGE may mitigate capability gaps rather than amplify strong models. All evidence is confined to CAGE-2 B-line; cross-family findings are directional evidence.

Related papers

• GRASP: Gated Regression-Aware Skill Proposer for Self-Improving LLM Agents [17.26555663132631]
  GRASP (Gated Regression-Aware Skill Proposer) treats agent improvement as a sequence of edits to a bounded skill library.<n>We evaluate GRASP across five base models on clinical benchmarks.
  arXiv  Detail & Related papers  (2026-05-28T09:30:09Z)
• ResRL: Boosting LLM Reasoning via Negative Sample Projection Residual Reinforcement Learning [60.411097231348975]
  Reinforcement Learning with Verifiable Rewards (RLVR) enhances reasoning of Large Language Models (LLMs)<n>This paper proposes negative sample projection Residual Reinforcement Learning (ResRL) that decouples similar semantic distributions among positive and negative responses.
  arXiv  Detail & Related papers  (2026-05-01T03:57:44Z)
• Agentic Adversarial Rewriting Exposes Architectural Vulnerabilities in Black-Box NLP Pipelines [34.37967130277969]
  We propose a two-agent evasion framework operating in a semantic perturbation space.<n>Evasion rates are 19.95 to 40.34% on modern large language model (LLM) based systems.<n>A pattern-informed defense reduces the evasion rate by up to 65.18%.
  arXiv  Detail & Related papers  (2026-04-26T00:58:36Z)
• REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge [83.2858110368572]
  We propose textbfREAL (underlineREgression-underlineAware Reinforcement underlineLThought), a principled RL framework designed to optimize regression rewards.<n>We show that REAL consistently outperforms both regression-aware SFT baselines and standard RL methods.
  arXiv  Detail & Related papers  (2026-03-17T21:19:08Z)
• RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedback [54.39884046754265]
  RetroAgent is an online RL framework that enables agents to master complex interactive environments.<n>Our experiments show that RetroAgent achieves state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2026-03-09T16:23:33Z)
• Small Reward Models via Backward Inference [100.59075794599768]
  FLIP (FLipped Inference for Prompt Reconstruction) is a reference-free and rubric-free reward modeling approach.<n>It reformulates reward modeling through backward inference: inferring the instruction that would most plausibly produce a given response.
  arXiv  Detail & Related papers  (2026-02-14T01:55:39Z)
• R^3: Replay, Reflection, and Ranking Rewards for LLM Reinforcement Learning [32.16683059021539]
  Large reasoning models (LRMs) aim to solve diverse and complex problems through structured reasoning.<n>Recent advances in group-based policy optimization methods have shown promise in enabling stable advantage estimation without reliance on process-level annotations.<n>We propose a reinforcement learning mechanism named emphtextbfR3 that along three directions: (1) a emphcross-context underlinetextbfReplay strategy that maintains the intra-group advantage, (2) an emphin-context self-underlinetextbfReflection mechanism
  arXiv  Detail & Related papers  (2026-01-27T13:55:34Z)
• LLMs can Compress LLMs: Adaptive Pruning by Agents [0.0]
  Post-training pruning has emerged as a promising approach to reduce computational costs while preserving performance.<n>We introduce agent-guided pruning, where a foundation model acts as an adaptive pruning agent.<n>We evaluate our approach on Q3 models (4B and 8B parameters) at approximately 45% sparsity, demonstrating substantial improvements over structured pruning baselines.
  arXiv  Detail & Related papers  (2026-01-14T18:45:36Z)
• Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models [50.99097734404912]
  We show that RL-friendly models exhibit intra-class compactness and inter-class separation in their probability assignments to correct vs. incorrect responses.<n>Experiments across six mathematical benchmarks show consistent improvements across all model families, with gains up to 5.9 points on AIME24.
  arXiv  Detail & Related papers  (2026-01-11T13:34:44Z)
• RefineBench: Evaluating Refinement Capability of Language Models via Checklists [71.02281792867531]
  We evaluate two refinement modes: guided refinement and self-refinement.<n>In guided refinement, both proprietary LMs and large open-weight LMs can leverage targeted feedback to refine responses to near-perfect levels within five turns.<n>These findings suggest that frontier LMs require breakthroughs to self-refine their incorrect responses.
  arXiv  Detail & Related papers  (2025-11-27T07:20:52Z)
• Squeeze the Soaked Sponge: Efficient Off-policy Reinforcement Finetuning for Large Language Model [56.92219181993453]
  We propose Reincarnating Mix-policy Proximal Policy Gradient (ReMix) to enable on-policyRFT methods like PPO and GRPO to leverage off-policy data.<n>ReMix consists of three major components: (1) Mix-policy proximal policy gradient with an increased Update-To-Data (UTD) ratio for efficient training; (2) KL-Convex policy constraint to balance the trade-off between stability and flexibility; (3) Policy reincarnation to achieve a seamless transition from efficient early-stage learning to steady improvement.
  arXiv  Detail & Related papers  (2025-07-09T14:29:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.