Unified Context Evolution for LLM Agents

• URL: http://arxiv.org/abs/2606.02304v1
• Date: Mon, 01 Jun 2026 14:25:29 GMT
• Title: Unified Context Evolution for LLM Agents
• Abstract summary: Unified Context Evolution (UCE) is a framework that externalizes agent experience into an evolving library typed Evolvable Context Units (ECUs)<n>UCE decomposes experience into four complementary types (Memory, Strategy, Skill)<n>A scheduling module allocates each cycle's generation budget toward the types where the library is weakest.
• Score: 19.55736346580243
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-based agents can solve multi-step interactive tasks by combining reasoning with environment feedback, yet each episode starts from the same fixed context and any useful strategy discovered along the way is lost once the task ends. Existing approaches either limit learning to the current task or pool all experience into a single untyped store, without distinguishing knowledge types, tracking quality through use, or balancing what the library still lacks. We introduce Unified Context Evolution (UCE), a gradient-free framework that externalizes agent experience into an evolving library of typed Evolvable Context Units (ECUs). UCE decomposes experience into four complementary types (Memory, Strategy, Workflow, and Skill), each generated from trajectories under type-specific conditions, retrieved at decision time, scored through repeated usage outcomes, and pruned when no longer valuable. A scheduling module allocates each cycle's generation budget toward the types where the library is weakest. Across two interactive benchmarks, UCE raises ALFWorld success from 75.4% to 96.3% and WebShop task score from 45.1% to 61.3%, and the accumulated library transfers to alternative actor backbones without retraining.

Related papers

• From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use Agents [48.50635499112162]
  We present an online skill-evolution framework that converts interaction trajectories and evaluator feedback into a persistent, versioned library of reusable procedures.<n>We compare the full evolving-library system with a configuration-matched empty-library control across four OSWorld application domains.<n>In GIMP, provenance-aware analysis reveals retrieval across task-of-origin boundaries and revision, where repeated accepted edits fail to recover the originating task.
  arXiv  Detail & Related papers  (2026-09-04T08:29:45Z)
• IntHQ: Task-Interactive Hierarchical Query on Dual-Stream Representations for Generative Recommendation [20.546679348797905]
  We show that existing multi-taskers are inherently prone to a threefold collapse.<n>We propose InHQ, a multi-task generative recommender with three components each alleviating one collapse.<n>In offline evaluations, InHQ consistently outperforms competitive backbones under four representative task-head configurations.
  arXiv  Detail & Related papers  (2026-08-10T14:13:47Z)
• AEL: Agent Evolving Learning for Open-Ended Environments [43.56685432981852]
  We introduce emphAgent Evolving Learning (ael), a two-timescale framework that addresses this obstacle.<n>ael achieves a Sharpe ratio of 2.13$pm$0.47, outperforming five published self-improving methods.<n>This demonstrates that the bottleneck in agent self-improvement is emphself-diagnosing how to use experience rather than adding architectural complexity.
  arXiv  Detail & Related papers  (2026-04-23T14:29:25Z)
• SLEA-RL: Step-Level Experience Augmented Reinforcement Learning for Multi-Turn Agentic Training [2.291770711277359]
  We propose SLEA-RL (Step-Level Experience-Augmented Reinforcement Learning), a framework that retrieves relevant experiences at each decision step conditioned on the current observation.<n>SLEA-RL operates through three components: (i) step-level observation clustering that groups structurally equivalent environmental states for efficient cluster-indexed retrieval; (ii) a self-evolving experience library that distills successful strategies and failure patterns through score-based admission and rate-limited extraction; and (iii) policy optimization with step-level credit assignment for fine-grained advantage estimation across multi-turn episodes.
  arXiv  Detail & Related papers  (2026-03-18T07:16:18Z)
• Autonomous Continual Learning of Computer-Use Agents for Environment Adaptation [57.65688895630163]
  We introduce ACuRL, an Autonomous Curriculum Reinforcement Learning framework that continually adapts agents to specific environments with zero human data.<n>Our method effectively enables both intra-environment and cross-environment continual learning, yielding 4-22% performance gains without forgetting existing environments.
  arXiv  Detail & Related papers  (2026-02-10T23:06:02Z)
• Fine-tuning with RAG for Improving LLM Learning of New Skills [8.825427873545063]
  Large language model (LLM) agents frequently fail in predictable ways.<n>We propose a simple pipeline that converts inference-time retrieval into learned competence through distillation.
  arXiv  Detail & Related papers  (2025-10-01T19:03:48Z)
• Reinforcement Learning for Long-Horizon Interactive LLM Agents [56.9860859585028]
  Interactive digital agents (IDAs) leverage APIs of stateful digital environments to perform tasks in response to user requests.<n>We present a reinforcement learning (RL) approach that trains IDAs directly in their target environments.<n>We derive LOOP, a data- and memory-efficient variant of proximal policy optimization.
  arXiv  Detail & Related papers  (2025-02-03T18:35:42Z)
• Loop Improvement: An Efficient Approach for Extracting Shared Features from Heterogeneous Data without Central Server [16.249442761713322]
  "Loop Improvement" (LI) is a novel method enhancing this separation and feature extraction without necessitating a central server or data interchange among participants. In personalized federated learning environments, LI consistently outperforms the advanced FedALA algorithm in accuracy across diverse scenarios. LI's adaptability extends to multi-task learning, streamlining the extraction of common features across tasks and obviating the need for simultaneous training.
  arXiv  Detail & Related papers  (2024-03-21T12:59:24Z)
• OverPrompt: Enhancing ChatGPT through Efficient In-Context Learning [49.38867353135258]
  We propose OverPrompt, leveraging the in-context learning capability of LLMs to handle multiple task inputs. Our experiments show that OverPrompt can achieve cost-efficient zero-shot classification without causing significant detriment to task performance.
  arXiv  Detail & Related papers  (2023-05-24T10:08:04Z)
• Reinforcement Learning for Branch-and-Bound Optimisation using Retrospective Trajectories [72.15369769265398]
  Machine learning has emerged as a promising paradigm for branching. We propose retro branching; a simple yet effective approach to RL for branching. We outperform the current state-of-the-art RL branching algorithm by 3-5x and come within 20% of the best IL method's performance on MILPs with 500 constraints and 1000 variables.
  arXiv  Detail & Related papers  (2022-05-28T06:08:07Z)
• Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems [42.973910399533054]
  We introduce a curriculum learning algorithm, Variational Automatic Curriculum Learning (VACL), for solving cooperative multi-agent reinforcement learning problems. Our VACL algorithm implements this variational paradigm with two practical components, task expansion and entity progression. Experiment results show that VACL solves a collection of sparse-reward problems with a large number of agents.
  arXiv  Detail & Related papers  (2021-11-08T16:35:08Z)
• PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning [102.36450942613091]
  We propose an inverse reinforcement learning algorithm, called emphinverse temporal difference learning (ITD) We show how to seamlessly integrate ITD with learning from online environment interactions, arriving at a novel algorithm for reinforcement learning with demonstrations, called $Psi Phi$-learning.
  arXiv  Detail & Related papers  (2021-02-24T21:12:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.