GaussMemory: Task-Driven 3D Gaussian Scene Memory for Long-Horizon Robotic Manipulation

• URL: http://arxiv.org/abs/2608.14986v1
• Date: Sat, 15 Aug 2026 02:28:59 GMT
• Title: GaussMemory: Task-Driven 3D Gaussian Scene Memory for Long-Horizon Robotic Manipulation
• Abstract summary: Long-horizon robotic manipulation relies on persistent spatial memory.<n>Existing 3D memory systems store observations using fixed, hand-crafted rules.<n>We propose a paradigm shift from passive storage to active, task-driven spatial memory.
• Score: 10.233658861023612
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-horizon robotic manipulation fundamentally relies on persistent spatial memory. However, existing 3D memory systems function merely as passive recorders: they store observations using fixed, hand-crafted rules, treating every scene element--whether a critical grasp target or an irrelevant background wall--with equal importance. In this paper, we propose a paradigm shift from passive storage to active, task-driven spatial memory. We argue that a robot's memory should not simply record what it sees, but actively learn how to remember--discovering which objects to track precisely, how aggressively to update them, and what to discard, all learned end-to-end without hand-designed rules. Crucially, this active paradigm is realized by unifying memory update and readout as two sides of the same cognitive process, enabling bidirectional flow where task needs shape update strategies and vice versa. To instantiate this vision, we introduce GaussMemory, which leverages 3D Gaussian Splatting as a persistent geometric substrate. On LIBERO, GaussMemory outperforms MemoryVLA on Goal and Long-10; on VLABench, it surpasses $π_0$-FAST by +5.2% (Track 1) and +6.0% (Track 6).

Related papers

• AutoMem: Automated Learning of Memory as a Cognitive Skill [63.07128029793901]
  We show that memory expertise is a learned skill.<n>We introduce AutoMem, a framework that automates the two axes of memory management.<n>Our results show that memory management is an independently learnable skill.
  arXiv  Detail & Related papers  (2026-07-01T17:57:03Z)
• Worth Remembering: Surprise-Gated Robot Episodic Memory [12.101245779148316]
  We present an approach to compute surprise in a semantically rich deployment-agnostic latent space provided by V-JEPA-2.<n>Using our gated episodic memory to augment 4D scene graph-based spatial memory, we show a consistent improvement over state-of-the-art benchmarks in robot question answering.
  arXiv  Detail & Related papers  (2026-06-02T15:39:27Z)
• EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigation [81.54723508469617]
  EvoMemNav is an efficient, self-evolving, fine-grained memory framework for zero-shot embodied navigation.<n>VSMGraph organizes views with semantic cues and topological relations into a room-view-object hierarchy.<n>Experiments on GOAT-Bench and HM3D across object, text-description, and image-goal modalities show consistent gains in SR/SPL.
  arXiv  Detail & Related papers  (2026-06-02T11:27:44Z)
• Robust Dreamer: Deviation-Aware Latent Gaussian Memory for Action-Controlled AR Video Generation [89.70897512515477]
  Action-controlled image-to-video generation is a promising paradigm for interactive world simulation, where each control signal should elicit an immediate visual response.<n>Maintaining visual fidelity and 3D consistency over long autoregressive rollouts remains challenging.<n>Existing 3D-aware methods often suffer from catastrophic drift due to two impediments: information loss from textitLatent--RGB Cycling, and the training--inference gap induced by the textiterror-free hypothesis.<n>We present textbfRobust Dreamer, a memory-augmented framework built around how to
  arXiv  Detail & Related papers  (2026-05-29T05:21:33Z)
• RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies [54.23445842621374]
  Memory is critical for long-horizon and history-dependent robotic manipulation.<n>Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms.<n>We introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models.
  arXiv  Detail & Related papers  (2026-03-04T21:59:32Z)
• MEM: Multi-Scale Embodied Memory for Vision Language Action Models [73.3883864595845]
  We introduce Multi-Scale Embodied Memory (MEM), an approach for mixed-modal long-horizon memory in robot policies.<n>MEM combines video-based short-horizon memory, compressed via a video encoder, with text-based long-horizon memory.<n>MEM enables robot policies to perform tasks that span up to fifteen minutes, like cleaning up a kitchen, or preparing a grilled cheese sandwich.
  arXiv  Detail & Related papers  (2026-03-04T00:03:02Z)
• MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation [59.31354761628506]
  Temporal context is essential for robotic manipulation because such tasks are inherently non-Markovian, yet mainstream VLA models typically overlook it.<n>We propose MemoryVLA, a Cognition-Memory-Action framework for long-horizon robotic manipulation.<n>We evaluate it on 150+ simulation and real-world tasks across three robots.
  arXiv  Detail & Related papers  (2025-08-26T17:57:16Z)
• Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents [4.371266695484245]
  Memory Gym presents a suite of 2D partially observable environments, namely Mortar Mayhem, Mystery Path, and Searing Spotlights.<n>These environments, originally with finite tasks, are expanded into innovative, endless formats, mirroring the escalating challenges of cumulative memory games such as "I packed my bag"
  arXiv  Detail & Related papers  (2023-09-29T12:59:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.