PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraft

• URL: http://arxiv.org/abs/2605.27762v2
• Date: Mon, 01 Jun 2026 19:20:53 GMT
• Title: PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraft
• Abstract summary: We present PEAM, a Parametric Embodied Agent Memory framework in Minecraft.<n>PEAM transforms agent memory from inference-time retrieval into parameter-resident skills internalized through experience.<n>We show that PEAM improves long-horizon task performance, mitigates forgetting on previously consolidated skills, and improves parametric-versus-retrieval efficiency.
• Score: 59.80301645617478
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present PEAM, a Parametric Embodied Agent Memory framework in Minecraft that transforms agent memory from inference-time retrieval into parameter-resident skills internalized through experience. PEAM pairs a slow deliberative LLM for open-ended reasoning with a fast parametric module for reflexive execution of consolidated skills. The fast module is a multimodal Mixture-of-Experts LoRA architecture with per-category physically isolated adapters, enabling parameter-level continual learning without catastrophic forgetting. We treat failure as a first-class training signal: failure--correction trajectory pairs are internalized through a joint behavioral-cloning and contrastive objective, so the agent learns not only what succeeds but also how corrected actions differ from failed ones. To govern consolidation, PEAM introduces a parameterization-worthiness score for deciding which experience should be internalized, and a scale-free self-triggered consolidation mechanism for deciding when to internalize without task-specific hand-tuned thresholds, making the agent self-evolving as the trigger transfers across task distributions without re-tuning. Experiments in Minecraft show that PEAM improves long-horizon task performance, mitigates forgetting on previously consolidated skills, and improves parametric-versus-retrieval efficiency over retrieval-based embodied agents and parametric memory variants.

Related papers

• EVAF: A Test-Retest Protocol for Selective Parametric Consolidation [0.0]
  We introduce EVAF, an Echo-Valence Attractor Field mechanism for gated LoRA consolidation.<n>Across GPT-2 and TinyLlama, EVAF preferentially consolidates high-valence, high-surprise experiences.<n>Test-retest measurements show stronger post-interference behavioral persistence than frozen, retrieval-only, and ungated continual-update baselines.
  arXiv  Detail & Related papers  (2026-06-29T07:53:50Z)
• EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning [39.60899845728349]
  Multimodal Large Language Model (MLLM)-driven image restoration agent demonstrates effectiveness in degradation coupling scenarios.<n>Training-based methods embed intrinsic experience into parameters, achieving high inference efficiency but lacking compatibility with new tools or degradation.<n>We propose EvoIR-Agent, which first systematically formulates the experience components of a training-free image restoration agent.
  arXiv  Detail & Related papers  (2026-05-21T09:14:25Z)
• Dynamic Mixture of Latent Memories for Self-Evolving Agents [57.20419419302731]
  MoLEM is a generative mixture of latent memory framework based on a dynamic mixture-of-experts (MoE)<n>We train the framework on continual-learning sequences spanning math, science, and code domains.<n>After the full continual-learning sequence, our method improves the average accuracy by 10.40% over the Vanilla pretrained baseline.
  arXiv  Detail & Related papers  (2026-05-21T03:35:10Z)
• AffectAgent: Collaborative Multi-Agent Reasoning for Retrieval-Augmented Multimodal Emotion Recognition [62.16431420189863]
  LLM-based multimodal emotion recognition relies on static parametric memory and often hallucinates when interpreting nuanced affective states.<n>We introduce AffectAgent, an affect-oriented multi-agent retrieval-augmented generation framework.<n>AffectAgent comprises three jointly optimized agents, namely a query planner, an evidence filter, and an emotion generator.
  arXiv  Detail & Related papers  (2026-04-14T13:49:19Z)
• MP-ISMoE: Mixed-Precision Interactive Side Mixture-of-Experts for Efficient Transfer Learning [12.196384894535827]
  We propose a novel Mixed-Precision Interactive Side Mixture-of-Experts framework (MP-ISMoE)<n>MP-ISMoE remarkably promotes accuracy compared to state-of-the-art METL approaches, while maintaining comparable parameter and memory efficiency.
  arXiv  Detail & Related papers  (2026-04-10T08:00:28Z)
• ProcMEM: Learning Reusable Procedural Memory from Experience via Non-Parametric PPO for LLM Agents [31.24386037416211]
  LLM-driven agents often rely on on-the-fly reasoning, re-deriving solutions even in recurring scenarios.<n>This insufficient experience reuse leads to computational redundancy and execution instability.<n>We propose ProcMEM, a framework that enables agents to autonomously learn procedural memory from interaction experiences without parameter updates.
  arXiv  Detail & Related papers  (2026-02-02T09:43:12Z)
• Optimal Brain Iterative Merging: Mitigating Interference in LLM Merging [11.708743111945727]
  Large Language Models (LLMs) have demonstrated impressive capabilities, but their high computational costs pose challenges for customization.<n>Model merging offers a cost-effective alternative, yet existing methods suffer from interference among parameters, leading to performance degradation.<n>We propose Optimal Brain Iterative Merging, a novel method designed to mitigate both intra-model and inter-model interference.
  arXiv  Detail & Related papers  (2025-02-17T09:07:49Z)
• ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Experts [71.91042186338163]
  ALoRE is a novel PETL method that reuses the hypercomplex parameterized space constructed by Kronecker product to Aggregate Low Rank Experts.<n>Thanks to the artful design, ALoRE maintains negligible extra parameters and can be effortlessly merged into the frozen backbone.
  arXiv  Detail & Related papers  (2024-12-11T12:31:30Z)
• SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning [63.93193829913252]
  We propose an innovative METL strategy called SHERL for resource-limited scenarios. In the early route, intermediate outputs are consolidated via an anti-redundancy operation. In the late route, utilizing minimal late pre-trained layers could alleviate the peak demand on memory overhead.
  arXiv  Detail & Related papers  (2024-07-10T10:22:35Z)
• Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers [107.3726071306935]
  We propose a new plug-and-play training framework, SMoE-Dropout, to enable scaling transformers to better accuracy in their full capacity without collapse. SMoE-Dropout consists of a randomly and fixed router network to activate experts and gradually increases the activated expert number as training progresses over time. Our experiments demonstrate the superior performance and substantial computation savings of SMoE-Dropout, compared to dense training baselines with equivalent parameter counts.
  arXiv  Detail & Related papers  (2023-03-02T22:12:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.