FM-VLA: Force-based Memory for Vision-Language-Action Models in Contact-Rich Manipulation

• URL: http://arxiv.org/abs/2607.18231v1
• Date: Mon, 20 Jul 2026 17:58:31 GMT
• Title: FM-VLA: Force-based Memory for Vision-Language-Action Models in Contact-Rich Manipulation
• Abstract summary: We propose FM-VLA, a vision-language-action model with force-based memory.<n>We encode force histories into compact force memory tokens with a variational autoencoder pretrained with force time series reconstruction.<n>Our lightweight force memory achieves over 80% success rate with minimal inference overhead.
• Score: 30.071471609769755
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language-action (VLA) models have achieved impressive generalization in robotic manipulation, and recent memory-augmented VLAs have relaxed the Markovian assumption by conditioning on past images or language summaries. Vision-based memory approaches address this by conditioning on sampled past image frames, but they are computationally expensive and fundamentally limited when temporal events are visually ambiguous, e.g., pushing a button multiple times with small movements. We propose FM-VLA, a VLA model with force-based memory, enabling temporal context reasoning for non-Markovian, contact-rich manipulation. We encode force histories into compact force memory tokens with a variational autoencoder (VAE) pretrained with force time series reconstruction. By projecting force latent representations and short state history as additional conditioning tokens to the action expert module, we enable VLAs to leverage accumulated contact event history to guide manipulation. We evaluate FM-VLA on three memory-dependent tasks, including finding a hidden block, pressing a button, and wiping a dish for a specific number of times. Our lightweight force memory achieves over 80% success rate with minimal inference overhead, significantly outperforming baseline approaches. Project page: https://qft-333.github.io/FM-VLA-Page/

Related papers

• Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation [96.18832998704329]
  We introduce LaMem-VLA, a latent-memory-native framework that reconstructs historical experience into latent memory tokens.<n>LaMem-VLA enables memory to directly participate in VLA reasoning and guide action generation under a bounded context.
  arXiv  Detail & Related papers  (2026-07-08T16:26:06Z)
• MemoryWAM: Efficient World Action Modeling with Persistent Memory [80.90899269062128]
  We introduce MemoryWAM, a world action model with efficient persistent memory.<n>A tailored attention mechanism enables retrieval of both detailed short-term context and compressed long-term context.<n>MemoryWAM outperforms strong vision-language-action (VLA) and WAM baselines in both simulation and the real world.
  arXiv  Detail & Related papers  (2026-06-18T17:59:51Z)
• EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies [68.812675280427]
  EventVLA is an end-to-end framework founded on the concept of sparse visual evidence memory.<n>KEM directly predicts future probabilities from the VLA's latent embeddings to autonomously capture and store sparse, task-critical visual events.<n>We propose RoboTwin-MeM, a diagnostic benchmark designed to evaluate non-Markovian manipulation tasks with interactive visual evidence.
  arXiv  Detail & Related papers  (2026-06-18T11:11:37Z)
• MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Models [80.70528162709276]
  Temporal modeling is essential for robotic manipulation, as effective control requires memory of past interactions and imagination of future states.<n>We propose MemoryVLA++, a full temporal modeling framework that equips VLA models with memory and imagination for robotic manipulation.<n>Our method achieves strong performance across Libero, SimplerEnv, Mikasa-Robo, Calvin, Libero-Plus, and diverse real-robot tasks.
  arXiv  Detail & Related papers  (2026-06-08T17:59:53Z)
• ReMem-VLA: Empowering Vision-Language-Action Model with Memory via Dual-Level Recurrent Queries [45.23935281952228]
  We introduce ReMem-VLA, a vision-language-action (VLA) model equipped with two sets of learnable queries.<n>These queries are trained end-to-end to aggregate and maintain relevant context over time.<n>We demonstrate that ReMem-VLA exhibits strong memory capabilities across multiple dimensions.
  arXiv  Detail & Related papers  (2026-03-13T12:38:42Z)
• VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memory [31.464584758455356]
  VPWEM is a non-Markovian visuomotor policy equipped with working and episodic memories.<n>It exploits both short-term and episode-wide information for action generation with nearly constant memory and computation per step.
  arXiv  Detail & Related papers  (2026-03-05T07:52:50Z)
• RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies [54.23445842621374]
  Memory is critical for long-horizon and history-dependent robotic manipulation.<n>Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms.<n>We introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models.
  arXiv  Detail & Related papers  (2026-03-04T21:59:32Z)
• MAP-VLA: Memory-Augmented Prompting for Vision-Language-Action Model in Robotic Manipulation [22.063033063941788]
  Memory-Augmented Prompting for Vision-Language-Action model (MAP-VLA) is a novel framework that empowers pre-trained VLA models with demonstration-derived memory prompts to augment action generation for long-horizon robotic manipulation tasks.<n>Results show that MAP-VLA delivers up to 7.0% absolute performance gains in the simulation benchmark and 25.0% on real robot evaluations for long-horizon tasks, surpassing the current state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-11-12T17:56:13Z)
• MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation [59.31354761628506]
  Temporal context is essential for robotic manipulation because such tasks are inherently non-Markovian, yet mainstream VLA models typically overlook it.<n>We propose MemoryVLA, a Cognition-Memory-Action framework for long-horizon robotic manipulation.<n>We evaluate it on 150+ simulation and real-world tasks across three robots.
  arXiv  Detail & Related papers  (2025-08-26T17:57:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.