MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation

• URL: http://arxiv.org/abs/2607.23504v1
• Date: Sun, 26 Jul 2026 07:08:30 GMT
• Title: MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation
• Abstract summary: We propose a novel VLN framework that achieves state-of-the-art performance with real-time inference efficiency (14 FPS)<n>MemVLN utilizes a visual encoder to process continuous observations and a Large Language Model (LLM) to interpret instructions and generate actions.<n> Experiments on VLN-CE show that MemVLN-4B surpasses the baseline Qwen3-VL-4B architecture by 5.8% SR in R2R and 9.7% SR in RxR, while achieving a 7$times$ speedup in inference latency.
• Score: 68.89913903899101
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to maintain long-horizon visual history for trajectory consistency while executing actions with low latency. Existing video-based VLN approaches typically struggle to satisfy both demands simultaneously. To address these challenges, we propose MemVLN, a novel VLN framework that achieves state-of-the-art performance with real-time inference efficiency (14 FPS). MemVLN utilizes a visual encoder to process continuous observations and a Large Language Model (LLM) to interpret instructions and generate actions. Central to our approach is an Episodic Memory management that applies pyramidal resolutions. This mechanism concentrates computation on immediate percepts while retaining compressed long-term history. Complementing to this design, we introduce Procedural Memory for fast action with a compact vocabulary of atomic mid-level actions to bypass auto-regressive decoding latency. Experiments on VLN-CE show that MemVLN-4B surpasses the baseline Qwen3-VL-4B architecture by 5.8\% SR in R2R and 9.7\% SR in RxR, while achieving a 7$\times$ speedup in inference latency.

Related papers

• EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies [68.812675280427]
  EventVLA is an end-to-end framework founded on the concept of sparse visual evidence memory.<n>KEM directly predicts future probabilities from the VLA's latent embeddings to autonomously capture and store sparse, task-critical visual events.<n>We propose RoboTwin-MeM, a diagnostic benchmark designed to evaluate non-Markovian manipulation tasks with interactive visual evidence.
  arXiv  Detail & Related papers  (2026-06-18T11:11:37Z)
• Fast SAM2 with Text-Driven Token Pruning [52.8350457627401]
  Segment Anything Model 2 (SAM2), a vision computation model has significantly advanced in prompt-driven video object segmentation.<n>SAM2 pipelines propagate all visual tokens produced by the image encoder through downstream temporal reasoning modules, regardless of their relevance to the target object.<n>We introduce a text-guided token pruning framework that improves inference efficiency by selectively reducing token density prior to temporal propagation.
  arXiv  Detail & Related papers  (2025-12-24T18:59:05Z)
• InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Models [49.08289742711585]
  We propose InfiniteVL, a linear-complexity VLM architecture that synergizes sliding window attention (SWA) with Gated DeltaNet.<n>We show that InfiniteVL achieves over 3.6times inference speedup while maintaining constant latency and memory footprint.<n>In streaming video understanding scenarios, it sustains a stable 24 FPS real-time prefill speed while preserving long-term memory cache.
  arXiv  Detail & Related papers  (2025-12-09T17:18:32Z)
• SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inference [49.84148668264725]
  We present SparseVILA, a new paradigm for efficient VLM inference that decouples visual sparsity across the prefilling and decoding stages.<n>Built on an AWQ-optimized inference pipeline, SparseVILA achieves up to 4.0 times faster prefilling, 2.5 times faster decoding, and an overall 2.6 times end-to-end speedup on long-context video tasks.
  arXiv  Detail & Related papers  (2025-10-20T17:35:47Z)
• StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling [27.468345201477504]
  Vision-and-Language Navigation (VLN) in real-world settings requires agents to process continuous visual streams and generate actions with low latency grounded in language instructions.<n>We introduce StreamVLN, a streaming VLN framework that employs a hybrid slow-fast context modeling strategy to support multi-modal reasoning over interleaved vision, language and action inputs.<n> Experiments on VLN-CE benchmarks demonstrate state-of-the-art performance with stable low latency, ensuring robustness and efficiency in real-world deployment.
  arXiv  Detail & Related papers  (2025-07-07T17:49:41Z)
• Mem4Nav: Boosting Vision-and-Language Navigation in Urban Environments with a Hierarchical Spatial-Cognition Long-Short Memory System [9.687946545604492]
  Vision-and-Language Navigation (VLN) in large-scale urban environments requires embodied agents to ground linguistic instructions in complex scenes.<n>We introduce textbfMem4Nav, a hierarchical spatial-cognition long-short memory system that can augment any VLN backbone.
  arXiv  Detail & Related papers  (2025-06-24T09:00:43Z)
• Multimodal Transformer with Variable-length Memory for Vision-and-Language Navigation [79.1669476932147]
  Vision-and-Language Navigation (VLN) is a task that an agent is required to follow a language instruction to navigate to the goal position. Recent Transformer-based VLN methods have made great progress benefiting from the direct connections between visual observations and the language instruction. We introduce Multimodal Transformer with Variable-length Memory (MTVM) for visually-grounded natural language navigation.
  arXiv  Detail & Related papers  (2021-11-10T16:04:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.