Mem-W: Latent Memory-Native GUI Agents

• URL: http://arxiv.org/abs/2605.09317v1
• Date: Sun, 10 May 2026 04:31:23 GMT
• Title: Mem-W: Latent Memory-Native GUI Agents
• Abstract summary: We introduce Mem-W, a series of latent-memory-native GUI agents that treat memory as part of the agent's continuous context.<n>Across four web and mobile navigation benchmarks, Mem-W consistently improves diverse backbones and memory-enhanced baselines.
• Score: 50.87647372904382
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: GUI agents are beginning to operate the web, mobile, and desktop as interactive worlds, where successful control depends on carrying forward visual, procedural, and task-level evidence beyond the fleeting present screen. Yet most agents still treat memory as an external, human-readable artifact: histories are summarized, categorized, retrieved, and reinserted as text or structured records before being encoded again by the policy. This creates a mismatch between the representational form in which experience is stored and the latent embedding sequence over which modern GUI policies actually act. We introduce Mem-W, a series of latent-memory-native GUI agents that treat memory as part of the agent's continuous context rather than as an auxiliary symbolic scaffold. Mem-W weaves both historical trajectories (as experiential memory) and in-session segments (as working memory) into compact memory tokens through a shared trajectory-to-latent compressor. These tokens are woven with the current GUI observation and local context into one continuous embedding sequence, allowing the agent to read successes, failures, and unfinished progress through the same machine-native interface. Mem-W is trained with self-distillation and outcome-aware supervision to preserve decision-relevant state while filtering memory toward evidence that truly supports task success. Across four web and mobile navigation benchmarks, Mem-W consistently improves diverse backbones and memory-enhanced baselines, with gains of up to $+30.0$, suggesting that latent-context-native memory can serve as a scalable foundation for long-horizon GUI agency.

Related papers

• FocusMem: Factorizing Content, Readout, and Trust in Latent GUI Memory [13.853612752899442]
  We introduce FocusMem, which separates responsibilities within a compact latent-memory interface.<n>A role-aware content basis encourages episodic memory to retain reusable experience and working memory to retain task progress.<n>A state-conditioned readout generates a decision-specific view of the same stored evidence, while a lightweight trust gate can suppress memory blocks that appear irrelevant to the current step.
  arXiv  Detail & Related papers  (2026-08-05T07:02:30Z)
• Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents [72.3752981234916]
  Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.<n>We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process.<n>MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5-20%.
  arXiv  Detail & Related papers  (2026-07-15T08:32:40Z)
• Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents [51.30250860677378]
  We propose MRAgent, a framework that combines an associative memory graph with an active reconstruction mechanism.<n>Running on a Cue-Tag-Content graph, our active reconstruction mechanism integrates reasoning directly into memory access.<n>Experiments on the LoCoMo benchmark and LongMemEval benchmark demonstrate significant improvements over strong baselines.
  arXiv  Detail & Related papers  (2026-06-04T11:29:46Z)
• STAMP: Training Explicit Memory for Mobile GUI Agents in Controllable and Scalable Virtual Environments [63.39393178045112]
  Mobile agents excel at immediate reactive control but frequently fail in realistic, longhorizon tasks that require memory.<n>We present STAMP, a framework that trains explicit memory in mobile agents through controllable virtual environments.<n>The resulting StampGUI agent sets a new high watermark on our Memory-World benchmark, demonstrating exceptional memory accuracy and task resilience.
  arXiv  Detail & Related papers  (2026-05-28T04:00:13Z)
• MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents [47.19679323562172]
  We introduce textbfMementoGUI, a plug-in agentic memory framework for MLLM-based GUI agents.<n>MementoCore is a learned controller for online memory selection, compression, and retrieval.<n>Experiments show MementoGUI consistently improves GUI agents over no-history, history-replay, and text-only memory baselines.
  arXiv  Detail & Related papers  (2026-05-18T16:57:36Z)
• SAGE: A Self-Evolving Agentic Graph-Memory Engine for Structure-Aware Associative Memory [40.68349164818526]
  We introduce SAGE, a Self-evolving Agentic Graph-memory Engine that models graph memory as a dynamic long-term memory substrate.<n> SAGE couples two roles: a memory writer that incrementally constucts structured graph memory from interaction histories, and a Graph Foundation Model-based memory reader to perform retrieval and provide feedback to the memory writer.
  arXiv  Detail & Related papers  (2026-05-12T12:47:43Z)
• MemMA: Coordinating the Memory Cycle through Multi-Agent Reasoning and In-Situ Self-Evolution [52.29303869050117]
  Memory-augmented LLM agents maintain external memory banks to support long-horizon interaction.<n>MemMA is a plug-and-play multi-agent framework that coordinates the memory cycle along both the forward and backward paths.
  arXiv  Detail & Related papers  (2026-03-19T10:15:59Z)
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents [49.63422082885992]
  We propose AdaMem, an adaptive user-centric memory framework for long-horizon dialogue agents.<n>AdaMem organizes dialogue history into working, episodic, persona, and graph memories.<n>We evaluate AdaMem on the LoCoMo and PERSONAMEM benchmarks for long-horizon reasoning and user modeling.
  arXiv  Detail & Related papers  (2026-03-17T13:22:54Z)
• MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks [55.145729491377374]
  Existing evaluations of agents with memory typically assess memorization and action in isolation.<n>We introduce MemoryArena, a unified evaluation gym for benchmarking agent memory in multi-session Memory-Agent-Environment loops.<n> MemoryArena supports evaluation across web navigation, preference-constrained planning, progressive information search, and sequential formal reasoning.
  arXiv  Detail & Related papers  (2026-02-18T09:49:14Z)
• TraceMem: Weaving Narrative Memory Schemata from User Conversational Traces [9.654990538033362]
  Sustaining long-term interactions remains a bottleneck for Large Language Models.<n>We propose TraceMem, a framework that weaves structured, narrative memory schemata from user conversational traces.<n>TraceMem achieves state-of-the-art performance with a brain-inspired architecture.
  arXiv  Detail & Related papers  (2026-02-10T12:14:58Z)
• Graph-based Agent Memory: Taxonomy, Techniques, and Applications [63.70340159016138]
  Memory emerges as the core module in the Large Language Model (LLM)-based agents for long-horizon complex tasks.<n>Among diverse paradigms, graph stands out as a powerful structure for agent memory due to the intrinsic capabilities to model relational dependencies.<n>This survey presents a comprehensive review of agent memory from the graph-based perspective.
  arXiv  Detail & Related papers  (2026-02-05T13:49:05Z)
• EverMemOS: A Self-Organizing Memory Operating System for Structured Long-Horizon Reasoning [42.339841548168565]
  Large Language Models (LLMs) are increasingly deployed as long-term interactive agents, yet their limited context windows make it difficult to sustain coherent behavior over extended interactions.<n>We introduce EverMemOS, a self-organizing memory operating system that implements an engram-inspired lifecycle for computational memory.<n>EverMemOS achieves state-of-the-art performance on memory-augmented reasoning tasks.
  arXiv  Detail & Related papers  (2026-01-05T14:39:43Z)
• Chain-of-Memory: Enhancing GUI Agents for Cross-Application Navigation [6.815990151030097]
  Chain-of-Memory (CoM) is a novel approach for explicitly modeling short-term and long-term memory in Graphical User Interface (GUI) agents.<n>CoM enables GUI agents to better understand task states and retain critical historical information persistently.
  arXiv  Detail & Related papers  (2025-06-22T20:17:46Z)
• Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions [55.19217798774033]
  Memory is a fundamental component of AI systems, underpinning large language models (LLMs)-based agents.<n>In this survey, we first categorize memory representations into parametric and contextual forms.<n>We then introduce six fundamental memory operations: Consolidation, Updating, Indexing, Forgetting, Retrieval, and Compression.
  arXiv  Detail & Related papers  (2025-05-01T17:31:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.