PRACTICE: From Experience to Expertise in Self-Evolving Embodied Agents
- URL: http://arxiv.org/abs/2608.30760v1
- Date: Mon, 31 Aug 2026 13:26:16 GMT
- Title: PRACTICE: From Experience to Expertise in Self-Evolving Embodied Agents
- Abstract summary: We train a skill learner to discover and maintain a persistent skill library from past interaction trajectories.<n>We apply online skill-edit distillation to align the skill learner with a stronger teacher on its current edit distribution.<n>Experiments demonstrate that a compact skill learner delivers consistent performance improvements across successive library-update rounds.
- Score: 8.367852606377419
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Recent studies have shown that multimodal large language models (MLLMs) can serve as embodied agents, translating language instructions and visual observations into executable plans. However, building agents that can continually improve through interaction and rapidly adapt to their environments remains challenging. Summing up experience from past interaction trajectories provides a promising solution, but existing experience-based methods often rely on manually designed prompting workflows to extract and update skills. Such fixed procedures may struggle to learn updated skills from new and diverse experiences. We introduce PRACTICE, which trains a skill learner to discover and maintain a persistent skill library from past interaction trajectories while keeping the task executor frozen. Given the historical accumulated skills and incoming trajectories, the skill learner produces structured batch-edits that add, refine, merge, or remove skills, and then hierarchical consolidate all collected edits into a consistent updated skill library. We train the learner with a two-stage curriculum. First, it learns basic skill generation and library maintenance from oracle trajectories. Then, by contrasting successful and failed trajectories from heterogeneous executors on the same tasks, it learn to identify invalid action patterns and recovery strategies. Finally, we apply online skill-edit distillation to align the skill learner with a stronger teacher on its current edit distribution to further improves the policy. Experiments demonstrate that a compact skill learner delivers consistent performance improvements across successive library-update rounds for multiple frozen executors. On EB-ALFRED and EB-Habitat, PRACTICE further outperforms the strongest experience-based baselines. Project resources are publicly available at: https://baai-agents.github.io/PRACTICE
Related papers
- Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents [67.97365535829098]
Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.<n>We propose Evo-Harness, which distills noisy, single-shot executions into reusable skill harnesses for cross-domain and topic-level adaptation.<n>Our analysis demonstrates the effectiveness of Evo-Harness and provides a principled understanding of how LLM agents can effectively learn on the fly.
arXiv Detail & Related papers (2026-08-15T06:43:56Z) - Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
arXiv Detail & Related papers (2026-07-24T17:59:22Z) - Learning New Tasks via Reusable Skills: Skill-Compositional Experts for Embodied Continual Learning [60.37693496953694]
Embodied Continual Learning (ECL) aims to enable robots to continually acquire new manipulation tasks.<n>ECL suffers from more severe catastrophic forgetting.<n>Key challenge in ECL lies in structured skill reuse across continually evolving tasks.
arXiv Detail & Related papers (2026-06-14T09:02:49Z) - SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization [99.18305770138677]
Agent skills are reusable strategies that guide agent reasoning and action.<n>Current skill construction methods treat the problem as one-shot extraction.<n>We introduce SkillComposer, a framework that decomposes skill construction into three learnable operations.
arXiv Detail & Related papers (2026-06-04T12:16:08Z) - CODESKILL: Learning Self-Evolving Skills for Coding Agents [13.599750520978988]
Coding agents produce rich trajectories while solving software-engineering tasks.<n>To enable agent self-evolution, these trajectories can be distilled into reusable procedural skills that compactly encode experience to guide future behavior.<n>We propose CODESKILL, an LLM-based framework that reformulates skill extraction and skill-bank maintenance as a learnable management policy.
arXiv Detail & Related papers (2026-05-25T05:12:49Z) - SkillEvolBench: Benchmarking the Evolution from Episodic Experience to Procedural Skills [31.23929961213889]
Large language model (LLM) agents accumulate rich episodic trajectories while solving real-world tasks.<n>We introduce SkillEvolBench, a diagnostic benchmark for evaluating this step from experience reuse to skill formation.<n>We find that current agents often adapt locally but rarely form robust reusable skills.
arXiv Detail & Related papers (2026-05-22T18:23:31Z) - SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphs [70.1970574147839]
We propose SKILLGRAPH, a framework that represents reusable skills as nodes in a directed graph.<n>SKILLGRAPH retrieves not just individual skills, but an ordered skill subgraph that can guide multi-step decision making.<n> Experiments show that SKILLGRAPH achieves state-of-the-art performance against memory-augmented RL methods.
arXiv Detail & Related papers (2026-05-12T12:21:49Z) - SkillOS: Learning Skill Curation for Self-Evolving Agents [67.94374107466957]
We propose SkillOS, an experience-driven RL training recipe for learning skill curation in self-evolving agents.<n>SkillOS pairs a frozen agent executor that retrieves and applies skills with a trainable skill curator that updates an external SkillRepo from accumulated experience.<n>We show that SkillOS consistently outperforms memory-free and strong memory-based baselines in both effectiveness and efficiency.
arXiv Detail & Related papers (2026-05-07T17:31:50Z) - Reinforcement Learning for Self-Improving Agent with Skill Library [14.717149089634718]
Large Language Model (LLM)-based agents have demonstrated remarkable capabilities in complex reasoning and multi-turn interactions.<n>One promising approach is implementing skill libraries that allow agents to learn, validate, and apply new skills.<n>We propose a Reinforcement Learning (RL)-based approach to enhance agents' self-improvement capabilities with a skill library.
arXiv Detail & Related papers (2025-12-18T21:58:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.