Joint Agent Memory and Exploration Learning via Novelty Signals

• URL: http://arxiv.org/abs/2606.01528v1
• Date: Mon, 01 Jun 2026 01:20:26 GMT
• Title: Joint Agent Memory and Exploration Learning via Novelty Signals
• Abstract summary: In open-ended environments, exploration is fundamental for autonomous agents.<n>We introduce textbfJoint textbfAgent textbfMemory and textbfExploration textbfLearning (textbfJAMEL)<n>textbfJAMEL is a framework that trains agentic memory and exploration policy together through novelty-driven interaction.
• Score: 46.41306111727712
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In open-ended environments, exploration is fundamental for autonomous agents, yet current language model agents struggle with this. Effective exploration requires memory, but retaining raw interaction histories is computationally expensive over long trajectories. While latent memory offers a solution to compress interaction histories, its training lacks reliable supervisory signals. We introduce \textbf{J}oint \textbf{A}gent \textbf{M}emory and \textbf{E}xploration \textbf{L}earning (\textbf{JAMEL}), a framework that trains agentic memory and exploration policy together through novelty-driven interaction. We observe that memory and exploration form a mutually dependent loop: sustained exploration requires memory to distinguish exhausted behaviors from unseen ones, while novelty-seeking interaction provides the supervision needed to make memory useful for future exploration. By utilizing deterministic and persistent novelty signals such as code coverage in the GUI domain, we provide natural, annotation-free supervision for the memory module. Empirical evaluations demonstrate that \ours successfully generalizes to unseen environments. Its exploration capability outperforms open-weight baselines and rivals the exploration depth of a closed-source model while reducing token consumption. Our code and model are open-sourced at https://github.com/MobileLLM/JAMEL.

Related papers

• Active Exploring like a Pigeon: Reinforcing Spatial Reasoning via Agentic Vision-Language Models [17.24700274670692]
  Existing approaches treat spatial language models (VLMs) as passive observers, which is difficult for real-world applications.<n>Inspired by pigeons' building and exploiting cognitive maps for navigation, we propose a novel agentic pipeline for spatial reasoning.<n>Experiments on VisionCube benchmark demonstrate state-of-the-art performance with emph80.5% overall accuracy, outperforming the best current method by emph29.5 accuracy points.
  arXiv  Detail & Related papers  (2026-06-01T16:30:56Z)
• SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent [51.274445160155864]
  Long-horizon agentic reasoning requires large language models to act over long interaction histories containing thoughts, tool calls, observations, and partial conclusions.<n>Existing approaches address this difficulty by truncating the interaction history, compressing it into shorter surrogates, or retrieving selected parts of it for reuse.<n>We propose State-Adaptive Memory, a standalone framework that consolidates ongoing interaction into compact memory cues while preserving raw trajectory pages for intent-driven recall.
  arXiv  Detail & Related papers  (2026-05-23T08:37:16Z)
• AnchorMem: Anchored Facts with Associative Contexts for Building Memory in Large Language Models [22.304693716449112]
  Existing memory methods place excessive emphasis on organizing interactions by frequently rewriting them.<n>We introduce AnchorMem, a novel memory framework inspired by the Proust Phenomenon in cognitive science.<n>Our method reconciles fine-grained retrieval with the contextual integrity of interactions.
  arXiv  Detail & Related papers  (2026-04-19T11:02:08Z)
• PASK: Toward Intent-Aware Proactive Agents with Long-Term Memory [83.06095498971682]
  We first propose DD-MM-PAS (Demand Detection, Memory Modeling, Proactive Agent System) as a general paradigm for streaming proactive AI agent.<n>We instantiate this paradigm in Pask, with streaming IntentFlow model for DD, a hybrid memory (workspace, user, global) for long-term MM, PAS infra framework.<n>We also introduce LatentNeeds-Bench, a real-world benchmark built from user-consented data and refined through thousands of rounds of human editing.
  arXiv  Detail & Related papers  (2026-04-09T09:06:13Z)
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents [49.63422082885992]
  We propose AdaMem, an adaptive user-centric memory framework for long-horizon dialogue agents.<n>AdaMem organizes dialogue history into working, episodic, persona, and graph memories.<n>We evaluate AdaMem on the LoCoMo and PERSONAMEM benchmarks for long-horizon reasoning and user modeling.
  arXiv  Detail & Related papers  (2026-03-17T13:22:54Z)
• Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers [0.42061757959666934]
  Large language model (LLM) agents increasingly operate in settings where a single context window is too small to capture what has happened.<n>Memory is what turns a stateless text generator into a genuinely adaptive agent.<n>This survey offers a structured account of how memory is designed, implemented, and evaluated in modern LLM-based agents.
  arXiv  Detail & Related papers  (2026-03-08T15:08:01Z)
• Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMs [13.390425803544785]
  We introduce implicit memory, the ability of a model to carry state across otherwise independent interactions.<n>As a demonstration, we introduce a new class of temporal backdoors, which we call time bombs.<n>We analyze broader implications of implicit memory, including covert inter-agent communication, benchmark contamination, targeted manipulation, and training-data poisoning.
  arXiv  Detail & Related papers  (2026-02-09T12:01:32Z)
• Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Exploration [52.35887679314727]
  Long-term Memory Embodied Exploration aims to unify the agent's exploratory cognition and decision-making behaviors.<n>To enhance the agent's memory recall and proactive exploration capabilities, we propose MemoryExplorer.
  arXiv  Detail & Related papers  (2026-01-11T16:23:22Z)
• Memory Matters More: Event-Centric Memory as a Logic Map for Agent Searching and Reasoning [55.251697395358285]
  Large language models (LLMs) are increasingly deployed as intelligent agents that reason, plan, and interact with their environments.<n>To effectively scale to long-horizon scenarios, a key capability for such agents is a memory mechanism that can retain, organize, and retrieve past experiences.<n>We propose CompassMem, an event-centric memory framework inspired by Event Theory.
  arXiv  Detail & Related papers  (2026-01-08T08:44:07Z)
• Neuro-Inspired Fragmentation and Recall to Overcome Catastrophic Forgetting in Curiosity [31.396929282048916]
  Deep reinforcement learning methods exhibit impressive performance on a range of tasks but struggle on hard exploration tasks in large environments with sparse rewards. Prediction-based intrinsic rewards can help agents solve hard exploration tasks, but they can suffer from catastrophic forgetting. We propose a new method FARCuriosity, inspired by how humans and animals learn.
  arXiv  Detail & Related papers  (2023-10-26T16:28:17Z)
• METRA: Scalable Unsupervised RL with Metric-Aware Abstraction [69.90741082762646]
  Metric-Aware Abstraction (METRA) is a novel unsupervised reinforcement learning objective. By learning to move in every direction in the latent space, METRA obtains a tractable set of diverse behaviors. We show that METRA can discover a variety of useful behaviors even in complex, pixel-based environments.
  arXiv  Detail & Related papers  (2023-10-13T06:43:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.