AEL: Agent Evolving Learning for Open-Ended Environments

• URL: http://arxiv.org/abs/2604.21725v1
• Date: Thu, 23 Apr 2026 14:29:25 GMT
• Title: AEL: Agent Evolving Learning for Open-Ended Environments
• Abstract summary: We introduce emphAgent Evolving Learning (ael), a two-timescale framework that addresses this obstacle.<n>ael achieves a Sharpe ratio of 2.13$pm$0.47, outperforming five published self-improving methods.<n>This demonstrates that the bottleneck in agent self-improvement is emphself-diagnosing how to use experience rather than adding architectural complexity.
• Score: 43.56685432981852
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM agents increasingly operate in open-ended environments spanning hundreds of sequential episodes, yet they remain largely stateless: each task is solved from scratch without converting past experience into better future behavior. The central obstacle is not \emph{what} to remember but \emph{how to use} what has been remembered, including which retrieval policy to apply, how to interpret prior outcomes, and when the current strategy itself must change. We introduce \emph{Agent Evolving Learning} (\ael{}), a two-timescale framework that addresses this obstacle. At the fast timescale, a Thompson Sampling bandit learns which memory retrieval policy to apply at each episode; at the slow timescale, LLM-driven reflection diagnoses failure patterns and injects causal insights into the agent's decision prompt, giving it an interpretive frame for the evidence it retrieves. On a sequential portfolio benchmark (10 sector-diverse tickers, 208 episodes, 5 random seeds), \ael{} achieves a Sharpe ratio of 2.13$\pm$0.47, outperforming five published self-improving methods and all non-LLM baselines while maintaining the lowest variance among all LLM-based approaches. A nine-variant ablation reveals a ``less is more'' pattern: memory and reflection together produce a 58\% cumulative improvement over the stateless baseline, yet every additional mechanism we test (planner evolution, per-tool selection, cold-start initialization, skill extraction, and three credit assignment methods) \emph{degrades} performance. This demonstrates that the bottleneck in agent self-improvement is \emph{self-diagnosing how to use} experience rather than adding architectural complexity. Code and data: https://github.com/WujiangXu/AEL.

Related papers

• Scaling Self-Evolving Agents via Parametric Memory [69.96398842169002]
  Existing memory-augmented LLM agents store past experience exclusively in prompt space.<n>We introduce textttTMEM, a self-evolving parametric memory framework.<n>textttTMEM consistently outperforms summary-based and retrieval-based baselines across different model scales.
  arXiv  Detail & Related papers  (2026-06-03T07:18:31Z)
• Unified Context Evolution for LLM Agents [19.55736346580243]
  Unified Context Evolution (UCE) is a framework that externalizes agent experience into an evolving library typed Evolvable Context Units (ECUs)<n>UCE decomposes experience into four complementary types (Memory, Strategy, Skill)<n>A scheduling module allocates each cycle's generation budget toward the types where the library is weakest.
  arXiv  Detail & Related papers  (2026-06-01T14:25:29Z)
• $\nabla$-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space [71.23672814629448]
  $nabla$-Reasoner is an iterative generation framework that integrates differentiable optimization over token logits into the decoding loop.<n>$nabla$-Reasoner achieves over 20% accuracy improvement on a challenging mathematical reasoning benchmark.
  arXiv  Detail & Related papers  (2026-03-05T08:42:54Z)
• R^3: Replay, Reflection, and Ranking Rewards for LLM Reinforcement Learning [32.16683059021539]
  Large reasoning models (LRMs) aim to solve diverse and complex problems through structured reasoning.<n>Recent advances in group-based policy optimization methods have shown promise in enabling stable advantage estimation without reliance on process-level annotations.<n>We propose a reinforcement learning mechanism named emphtextbfR3 that along three directions: (1) a emphcross-context underlinetextbfReplay strategy that maintains the intra-group advantage, (2) an emphin-context self-underlinetextbfReflection mechanism
  arXiv  Detail & Related papers  (2026-01-27T13:55:34Z)
• Process-Tensor Tomography of SGD: Measuring Non-Markovian Memory via Back-Flow of Distinguishability [1.078600700827543]
  We build a simple model-agnostic witness of training memory based on emphback-flow of distinguishability.<n>We observe consistent positive back-flow with tight bootstrap confidence intervals, amplification under higher momentum, and more micro-steps.<n>We position this as a principled diagnostic and empirical evidence that practical SGD deviates from the Markov idealization.
  arXiv  Detail & Related papers  (2026-01-23T09:03:25Z)
• Boosting Micro-Expression Analysis via Prior-Guided Video-Level Regression [15.099304324307434]
  Micro-expressions (MEs) are involuntary, low-intensity, and short-duration facial expressions.<n>Most existing ME analysis methods rely on window-level classification with fixed window sizes and hard decisions.<n>We propose a prior-guided video-level regression method for ME analysis.
  arXiv  Detail & Related papers  (2025-08-26T09:13:36Z)
• Memento: Fine-tuning LLM Agents without Fine-tuning LLMs [36.3424780932712]
  We introduce a novel learning paradigm for Adaptive Large Language Model (LLM) agents.<n>Our method enables low-cost continual adaptation via memory-based online reinforcement learning.<n>We instantiate our agent model in the deep research setting, namely emphMemento, which attains top-1 on GAIA validation.
  arXiv  Detail & Related papers  (2025-08-22T07:25:30Z)
• Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments [54.67512489842682]
  Large language models (LLMs) have demonstrated strong planning and decision-making capabilities in complex embodied environments.<n>We take a first step toward exploring the early-exit behavior for LLM-based agents.
  arXiv  Detail & Related papers  (2025-05-23T08:23:36Z)
• FREE: Faster and Better Data-Free Meta-Learning [77.90126669914324]
  Data-Free Meta-Learning (DFML) aims to extract knowledge from a collection of pre-trained models without requiring the original data.<n>We introduce the Faster and Better Data-Free Meta-Learning framework, which contains: (i) a meta-generator for rapidly recovering training tasks from pre-trained models; and (ii) a meta-learner for generalizing to new unseen tasks.
  arXiv  Detail & Related papers  (2024-05-02T03:43:19Z)
• Large-scale Fully-Unsupervised Re-Identification [78.47108158030213]
  We propose two strategies to learn from large-scale unlabeled data. The first strategy performs a local neighborhood sampling to reduce the dataset size in each without violating neighborhood relationships. A second strategy leverages a novel Re-Ranking technique, which has a lower time upper bound complexity and reduces the memory complexity from O(n2) to O(kn) with k n.
  arXiv  Detail & Related papers  (2023-07-26T16:19:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.