ELITE: Experiential Learning and Intent-Aware Transfer for Self-improving Embodied Agents

• URL: http://arxiv.org/abs/2603.24018v1
• Date: Wed, 25 Mar 2026 07:25:51 GMT
• Title: ELITE: Experiential Learning and Intent-Aware Transfer for Self-improving Embodied Agents
• Abstract summary: Vision-language models (VLMs) have shown remarkable general capabilities, yet embodied agents built on them fail at complex tasks.<n>We introduce ELITE, an embodied agent framework with Experiential Learning and Intent-aware Transfer.
• Score: 17.131524956793935
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vision-language models (VLMs) have shown remarkable general capabilities, yet embodied agents built on them fail at complex tasks, often skipping critical steps, proposing invalid actions, and repeating mistakes. These failures arise from a fundamental gap between the static training data of VLMs and the physical interaction for embodied tasks. VLMs can learn rich semantic knowledge from static data but lack the ability to interact with the world. To address this issue, we introduce ELITE, an embodied agent framework with {E}xperiential {L}earning and {I}ntent-aware {T}ransfer that enables agents to continuously learn from their own environment interaction experiences, and transfer acquired knowledge to procedurally similar tasks. ELITE operates through two synergistic mechanisms, \textit{i.e.,} self-reflective knowledge construction and intent-aware retrieval. Specifically, self-reflective knowledge construction extracts reusable strategies from execution trajectories and maintains an evolving strategy pool through structured refinement operations. Then, intent-aware retrieval identifies relevant strategies from the pool and applies them to current tasks. Experiments on the EB-ALFRED and EB-Habitat benchmarks show that ELITE achieves 9\% and 5\% performance improvement over base VLMs in the online setting without any supervision. In the supervised setting, ELITE generalizes effectively to unseen task categories, achieving better performance compared to state-of-the-art training-based methods. These results demonstrate the effectiveness of ELITE for bridging the gap between semantic understanding and reliable action execution.

Related papers

• Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents [67.97365535829098]
  Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.<n>We propose Evo-Harness, which distills noisy, single-shot executions into reusable skill harnesses for cross-domain and topic-level adaptation.<n>Our analysis demonstrates the effectiveness of Evo-Harness and provides a principled understanding of how LLM agents can effectively learn on the fly.
  arXiv  Detail & Related papers  (2026-08-15T06:43:56Z)
• SEARL: Joint Optimization of Policy and Tool Graph Memory for Self-Evolving Agents [35.45207852488779]
  We introduce a Tool-Memory based self-evolving agentic framework SEARL.<n>Unlike approaches that directly utilize interaction experiences, our method constructs a structured experience memory that integrates planning with execution.<n>We evaluate our framework on knowledge reasoning and mathematics tasks, demonstrating its effectiveness in achieving more practical and efficient learning.
  arXiv  Detail & Related papers  (2026-04-09T04:38:47Z)
• Secure Forgetting: A Framework for Privacy-Driven Unlearning in Large Language Model (LLM)-Based Agents [20.770088583807148]
  This paper initiates research on unlearning in large language model (LLM)-based agents.<n>We propose a novel and comprehensive framework that categorizes unlearning scenarios into three contexts.<n>Within this framework, we introduce a natural language-based unlearning method that trains a conversion model to transform high-level unlearning requests into actionable unlearning prompts.
  arXiv  Detail & Related papers  (2026-04-01T03:17:35Z)
• Real-Time Procedural Learning From Experience for AI Agents [2.543194442104227]
  We propose Procedural Recall for Agents with eXperiences Indexed by State (PRAXIS)<n>PRAXIS stores the consequences of actions and retrieves them by jointly matching environmental and internal states of past episodes to the current state.<n> PRAXIS augments agentic action selection with retrieved state-action-result exemplars that are generated in real time.
  arXiv  Detail & Related papers  (2025-11-27T03:51:49Z)
• Grounded in Reality: Learning and Deploying Proactive LLM from Offline Logs [72.08224879435762]
  textttLearn-to-Ask is a simulator-free framework for learning and deploying proactive dialogue agents.<n>Our approach culminates in the successful deployment of LLMs into a live, large-scale online AI service.
  arXiv  Detail & Related papers  (2025-10-29T12:08:07Z)
• Learning Affordances at Inference-Time for Vision-Language-Action Models [50.93181349331096]
  In robotics, Vision-Language-Action models (VLAs) offer a promising path towards solving complex control tasks.<n>We introduce Learning from Inference-Time Execution (LITEN), which connects a VLA low-level policy to a high-level VLM that conditions on past experiences.<n>Our approach iterates between a reasoning phase that generates and executes plans for the low-level VLA, and an assessment phase that reflects on the resulting execution.
  arXiv  Detail & Related papers  (2025-10-22T16:43:29Z)
• Weakly-supervised VLM-guided Partial Contrastive Learning for Visual Language Navigation [36.17444261325021]
  Visual Language Navigation (VLN) is a fundamental task within the field of Embodied AI, focusing on the ability of agents to navigate complex environments based on natural language instructions.<n>Existing methods rely on pre-trained backbone models for visual perception, which struggle with the dynamic viewpoints in VLN scenarios.<n>We propose Weakly-supervised Partial Contrastive Learning (WPCL), a method that enhances an agent's ability to identify objects from dynamic viewpoints in VLN scenarios without requiring VLM fine-tuning.
  arXiv  Detail & Related papers  (2025-06-18T11:43:50Z)
• Improving LLM Agent Planning with In-Context Learning via Atomic Fact Augmentation and Lookahead Search [48.348209577994865]
  Large Language Models (LLMs) are increasingly capable but often require significant guidance or extensive interaction history to perform effectively in complex, interactive environments.<n>We introduce a novel LLM agent framework that enhances planning capabilities through in-context learning.<n>Our agent learns to extract task-critical atomic facts'' from its interaction trajectories.
  arXiv  Detail & Related papers  (2025-06-10T18:36:31Z)
• Underlying Semantic Diffusion for Effective and Efficient In-Context Learning [113.4003355229632]
  Underlying Semantic Diffusion (US-Diffusion) is an enhanced diffusion model that boosts underlying semantics learning, computational efficiency, and in-context learning capabilities.<n>We present a Feedback-Aided Learning (FAL) framework, which leverages feedback signals to guide the model in capturing semantic details.<n>We also propose a plug-and-play Efficient Sampling Strategy (ESS) for dense sampling at time steps with high-noise levels.
  arXiv  Detail & Related papers  (2025-03-06T03:06:22Z)
• Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
  AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions.<n>Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes.<n>We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
  arXiv  Detail & Related papers  (2025-02-18T17:12:26Z)
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty [52.72790059506241]
  Open Information Extraction (OIE) task aims at extracting structured facts from unstructured text. Despite the potential of large language models (LLMs) like ChatGPT as a general task solver, they lag behind state-of-the-art (supervised) methods in OIE tasks.
  arXiv  Detail & Related papers  (2023-09-07T01:35:24Z)
• Automated Relational Meta-learning [95.02216511235191]
  We propose an automated relational meta-learning framework that automatically extracts the cross-task relations and constructs the meta-knowledge graph. We conduct extensive experiments on 2D toy regression and few-shot image classification and the results demonstrate the superiority of ARML over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-01-03T07:02:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.