MEMOBench: A Process Level Memory Benchmark for Robotic Manipulation

• URL: http://arxiv.org/abs/2609.07047v1
• Date: Mon, 07 Sep 2026 05:06:20 GMT
• Title: MEMOBench: A Process Level Memory Benchmark for Robotic Manipulation
• Abstract summary: We present textbfMEMOBench, a benchmark for process level memory evaluation in robotic manipulation.<n>MEMOBench includes 30 history dependent tasks, 1,500 expert demonstrations, and 4,200 executable checkpoint instances from 84 templates.
• Score: 36.81053166644513
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Robotic manipulation often requires acting on information that is no longer visible, yet Vision-Language-Action policies are usually evaluated when the current observation largely determines the next action. Existing robotic memory benchmarks expose this gap, but they still rely mainly on final task success and therefore conflate forgetting with manipulation failure. We present \textbf{MEMOBench}, a benchmark for process level memory evaluation in robotic manipulation. MEMOBench includes 30 history dependent tasks, 1{,}500 expert demonstrations, and 4{,}200 executable checkpoint instances from 84 templates. Each checkpoint pairs coarse to fine language with a simulator predicate and labels one memory operation: Storage, Update, or Compression. These annotations define Memory Storage Rate, Memory Update Rate, and Memory Compression Rate, which measure memory fidelity alongside task success. Across standard and memory augmented VLA policies, the strongest memory module baseline reaches only 31.9\% average success rate, and high storage often coexists with weak update and compression. Checkpoint language also supervises semantic, contrastive, and framewise memory alignment objectives, yielding modest gains across different memory operations. MEMOBench provides a diagnostic evaluation suite and training supervision for memory grounded robotic policies. The project page is available at https://github.com/Collab-Gen/MEMOBench.

Related papers

• Mechanistic Attention Guidance for Agent Memory Refinement [32.57216046158823]
  We show that retrieval-head attention provides a mechanistic signal for revealing segment-level memory utilization.<n>We propose Attention-Guided Memory Refinement (AGMR), a framework that uses utilization patterns revealed by attention to guide targeted segment-level memory updates.
  arXiv  Detail & Related papers  (2026-07-20T07:17:50Z)
• MEMORA: Embodied Action Memory from Egocentric Videos for Reasoning and Planning [27.103249755262212]
  Long-horizon robot planning requires memory of the embodied experience that makes future goals interpretable.<n>We formulate Embodied Action Memory (EAM) as the capability to form, maintain, and use such experience as a persistent memory state for later decisions.<n>MEMORA realizes EAM with a formation-consolidation-retrieval lifecycle and four typed stores.
  arXiv  Detail & Related papers  (2026-07-15T18:12:28Z)
• Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents [72.3752981234916]
  Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.<n>We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process.<n>MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5-20%.
  arXiv  Detail & Related papers  (2026-07-15T08:32:40Z)
• MemGym: a Long-Horizon Memory Environment for LLM Agents [69.79226770543049]
  We present MemGym, a benchmark for agentic memory that unifies existing agent gyms and in-house memory-grounded pipelines.<n>MemGym reports memory-isolated scores that decouple memory performance from reasoning, retrieval, and tool-use ability.<n>Our synthetic pipelines for MEMGYM-CODEQA and MEMGYM-DR are length-controllable, ablation-verified at every stage, and tightly aligned with downstream scenarios.
  arXiv  Detail & Related papers  (2026-05-20T07:25:33Z)
• EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective [21.66000179149483]
  EvoMemBench is a unified benchmark organized along two axes: memory scope and memory content.<n>We compare 15 representative memory methods with strong long-context baselines under a standardized protocol.<n>Results show that current memory systems are still far from a general solution.
  arXiv  Detail & Related papers  (2026-05-18T13:54:38Z)
• RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark [49.16825786282095]
  Memory is a critical component of robotic intelligence, as robots must rely on past observations and actions to accomplish long-horizon tasks.<n>Existing robotic memory benchmarks still lack multimodal annotations for memory formation, provide limited task coverage and structural complexity, and remain restricted to simulation without real-world evaluation.<n>We address this gap with RoboMemArena, a large-scale benchmark of 26 tasks, with average trajectory lengths exceeding 1,000 steps per task and 68.9% of subtasks being memory-dependent.
  arXiv  Detail & Related papers  (2026-05-11T17:54:49Z)
• RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies [54.23445842621374]
  Memory is critical for long-horizon and history-dependent robotic manipulation.<n>Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms.<n>We introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models.
  arXiv  Detail & Related papers  (2026-03-04T21:59:32Z)
• RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design [77.30163153176954]
  RMBench is a simulation benchmark comprising 9 manipulation tasks that span multiple levels of memory complexity.<n>Mem-0 is a modular manipulation policy with explicit memory components designed to support controlled ablation studies.<n>We identify memory-related limitations in existing policies and provide empirical insights into how architectural design choices influence memory performance.
  arXiv  Detail & Related papers  (2026-03-01T18:59:59Z)
• MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks [55.145729491377374]
  Existing evaluations of agents with memory typically assess memorization and action in isolation.<n>We introduce MemoryArena, a unified evaluation gym for benchmarking agent memory in multi-session Memory-Agent-Environment loops.<n> MemoryArena supports evaluation across web navigation, preference-constrained planning, progressive information search, and sequential formal reasoning.
  arXiv  Detail & Related papers  (2026-02-18T09:49:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.