Spatial Memory Agent: Experience-Grounded Procedure Memory for Spatial Intelligence

• URL: http://arxiv.org/abs/2608.12743v1
• Date: Thu, 13 Aug 2026 02:42:35 GMT
• Title: Spatial Memory Agent: Experience-Grounded Procedure Memory for Spatial Intelligence
• Abstract summary: We present textbfSpatial Memory Agent (SMA), an textbfexperience-grounded runtime framework that converts verified spatial experience into reusable transferable lessons.<n>In a verifiable spatial environment, SMA queries the frozen VLM, obtains a predicted answer and reward, and uses textbfverifier-guided to distill compact transferable lessons from spatial experience.<n>SMA achieves the highest macro average in every base-model block and the best accuracy among the evaluated methods in most of the 20 evaluations.
• Score: 8.471537051766001
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spatial intelligence is becoming a foundation for embodied agents, robotic planning, and multimodal assistants. To improve the spatial reasoning ability of VLM agents, existing work has mainly followed two lines. One line uses post-training methods, such as supervised fine-tuning and reinforcement learning. Another line adopts an agentic paradigm in which the model calls external spatial tools, such as depth estimation and 3D reconstruction tools, to gather intermediate spatial evidence. We study a complementary and underexplored route: Can a frozen VLM agent improve its spatial reasoning through \textbf{parameter-update-free self-evolution}, without depending on external expert spatial tools at inference time? We present \textbf{Spatial Memory Agent (SMA)}, an \textbf{experience-grounded runtime framework} that converts verified spatial experience into reusable transferable lessons. In a verifiable spatial environment, SMA queries the frozen VLM, obtains a predicted answer and reward, and uses \textbf{verifier-guided reflection} to distill compact transferable lessons from spatial experience. SMA further assigns each lesson a \textbf{Transfer Reliability Score (TRS)}, which is initialized uniformly and calibrated from later retrieval outcomes as visit evidence of future transfer reliability. During \textbf{read-only deployment}, SMA retrieves lessons by semantic filter and similarity-TRS combined ranking, allowing the retrieved memory to guide frozen model inference. Across five representative spatial benchmarks and four base VLMs, SMA achieves the highest macro average in every base-model block and the best accuracy among the evaluated methods in most of the 20 evaluations, establishing a practical parameter-update-free path for spatial self-evolution across the evaluated frozen model scales and environments.

Related papers

• QR-Erase: Efficient Subspace-Based Machine Unlearning with Layer Localization [7.530890774798437]
  Machine unlearning seeks to remove targeted information from trained models without requiring costly retraining.<n>We introduce QR-Erase, a subspace-based framework that uses Pivoted QR decomposition to identify and remove task-specific representations.<n>We show that Pivoted QR provides accurate subspace recovery with bounded error, and that under a mild spectral gap condition, the recovered subspace approaches the optimal SVD solution.
  arXiv  Detail & Related papers  (2026-08-02T18:08:56Z)
• Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning [57.237835833636154]
  We propose MetrIc Learning with Expandable Subspace (Miles) to harness the prior information within pre-trained knowledge.<n>Miles achieves state-of-the-art performance in various CIL settings.
  arXiv  Detail & Related papers  (2026-07-20T06:22:02Z)
• S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligence [77.5121349100339]
  textscS-Agent reshapes spatial perception into scene-centric understanding beyond frame-centric recognition.<n>textscS-Agent casts the VLM as a semantic planner that decides what evidence is needed.<n>textscS-Agent consistently improves both open-source and closed-source VLMs in a training-free manner.
  arXiv  Detail & Related papers  (2026-06-18T17:34:55Z)
• SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning [99.49739831030388]
  We study how the design of this interface shapes the agent's capacity for open-ended spatial reasoning.<n>We propose SpatialClaw, a training-free framework for spatial reasoning that adopts code as the action interface.
  arXiv  Detail & Related papers  (2026-06-11T17:59:36Z)
• Reasmory: 3D Reconstruction as Explicit Memory for VLMs Spatial Reasoning [43.950190960277865]
  Vision-Language Models (VLMs) exhibit emerging spatial reasoning capabilities, yet they remain unreliable on tasks requiring precise spatial understanding.<n>We propose textbfReasmory, a framework that formulates spatial reasoning as structured program execution over reconstructed spatial memory.<n>Reasmory constructs explicit 3D memory, augments it with semantically grounded 3D object instances, and introduces a lightweight Domain-Specific Language.
  arXiv  Detail & Related papers  (2026-05-31T02:36:57Z)
• Which Pretraining Paradigm Better Serves Spatial Intelligence? An Empirical Comparison of Vision-Language and Video Generation Models [31.851717131965824]
  We present the first systematic frozen-feature probing study of Vision-Language Models (VLMs) and Video Generation Models (VGMs)<n>Using the lightweight probe, our framework enables a controlled comparison of what information is already encoded in frozen representations from two model families.<n>Results reveal a clear complementarity: VLMs are stronger at semantic tagging and instance grouping, while VGMs provide more accessible signals for dense geometry and camera motion.
  arXiv  Detail & Related papers  (2026-05-27T08:20:04Z)
• Dynamic Mixture of Latent Memories for Self-Evolving Agents [57.20419419302731]
  MoLEM is a generative mixture of latent memory framework based on a dynamic mixture-of-experts (MoE)<n>We train the framework on continual-learning sequences spanning math, science, and code domains.<n>After the full continual-learning sequence, our method improves the average accuracy by 10.40% over the Vanilla pretrained baseline.
  arXiv  Detail & Related papers  (2026-05-21T03:35:10Z)
• Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data [52.80567461420316]
  We propose a Memory-Augmented LLM-based Multi-Agent System (textbfMALMAS) for automated feature generation.<n>MALMAS decomposes the generation process into agents with distinct responsibilities, and a Router Agent activates an appropriate subset of agents per iteration.<n>We further integrate a memory module comprising procedural memory, feedback memory, and conceptual memory, enabling iterative refinement that adaptively guides subsequent feature generation.
  arXiv  Detail & Related papers  (2026-04-22T07:09:30Z)
• Towards Continuous Intelligence Growth: Self-Training, Continual Learning, and Dual-Scale Memory in SuperIntelliAgent [10.571643330948858]
  SuperIntelliAgent is an agentic learning framework that couples a trainable small diffusion model (the learner) with a frozen large language model (the verifier)<n>Unlike conventional supervised fine-tuning, SuperIntelliAgent learns autonomously without annotation.<n>We posit that pairing a trainable learner with a reasoning-capable verifier forms a minimal reliable unit of growing intelligence.
  arXiv  Detail & Related papers  (2025-11-28T18:32:49Z)
• Maximally-Informative Retrieval for State Space Model Generation [59.954191072042526]
  We introduce Retrieval In-Context Optimization (RICO) to minimize model uncertainty for a particular query at test-time.<n>Unlike traditional retrieval-augmented generation (RAG), which relies on externals for document retrieval, our approach leverages direct feedback from the model.<n>We show that standard top-$k$ retrieval with model gradients can approximate our optimization procedure, and provide connections to the leave-one-out loss.
  arXiv  Detail & Related papers  (2025-06-13T18:08:54Z)
• Multitask Adaptation by Retrospective Exploration with Learned World Models [77.34726150561087]
  We propose a meta-learned addressing model called RAMa that provides training samples for the MBRL agent taken from task-agnostic storage. The model is trained to maximize the expected agent's performance by selecting promising trajectories solving prior tasks from the storage.
  arXiv  Detail & Related papers  (2021-10-25T20:02:57Z)
• Exploring Dynamic Context for Multi-path Trajectory Prediction [33.66335553588001]
  We propose a novel framework, named Dynamic Context Network (DCENet) In our framework, the spatial context between agents is explored by using self-attention architectures. A set of future trajectories for each agent is predicted conditioned on the learned spatial-temporal context.
  arXiv  Detail & Related papers  (2020-10-30T13:39:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.