LongSpace: Exploring Long-Horizon Spatial Memory from Perception to Recall in Video

• URL: http://arxiv.org/abs/2606.05677v1
• Date: Thu, 04 Jun 2026 04:00:12 GMT
• Title: LongSpace: Exploring Long-Horizon Spatial Memory from Perception to Recall in Video
• Abstract summary: Long-horizon tasks such as autonomous driving and robotic navigation require more than recognizing the current view.<n>We introduce LongSpace, a room-tour video benchmark for long-horizon spatial memory.<n>LongSpace models long videos as sequential chunks, incorporates 3D structural cues into early decoder layers, and constructs layer-aware memory for question-guided retrieval.
• Score: 20.1389583507481
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Large Language Models (MLLMs) have advanced image and video understanding and can increasingly handle longer visual inputs. Long-horizon tasks such as autonomous driving and robotic navigation require more than recognizing the current view, as models must remember and retrieve previously observed spatial layouts, routes, viewpoint changes, and object states. To evaluate this capability, we introduce LongSpace-Bench, a room-tour video benchmark for long-horizon spatial memory, covering scene perception, spatial relations, and spatial memory. In this work, we further propose LongSpace, a memory framework for long-video spatial reasoning. LongSpace models long videos as sequential chunks, incorporates 3D structural cues into early decoder layers, and constructs layer-aware memory for question-guided retrieval. Experiments on multiple spatial reasoning benchmarks show that LongSpace improves long-video spatial understanding, further demonstrating explicit spatial memory as a key capability for long-horizon video MLLMs.

Related papers

• Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoning [77.7531245219403]
  textscHomer is a Hierarchical Online Memory Exploration and Reasoning framework.<n>textscHomer's memory mirrors the multi-scale structure of long videos, ranging from raw perception, to recurring entities, to events connected by explicit temporal and causal relations.<n>textscHomer outperforms the previous best agent method by $+5.5$, $+10.8$, and $+4.4$ points on M3-Bench-robot, M3-Bench-web, and Video-MME-Long.
  arXiv  Detail & Related papers  (2026-07-01T05:53:09Z)
• RAVEN: Long-Horizon Reasoning & Navigation with a Visuo-Spatio-Temporal Memory [10.92726806758712]
  RAVEN is an agentic memory system for long-horizon robotic question answering and navigation.<n>RAVEN stores visual embeddings with pose and time in a vector database, and grounds retrieval in a spatial map to answer queries.<n>We instantiate RAVEN on a Unitree Go1 robot for the task of long-horizon navigation for natural language goal-reaching.
  arXiv  Detail & Related papers  (2026-06-23T21:56:41Z)
• Keep It in Mind: User Centric Continual Spatial Intelligence Reasoning in Egocentric Video Streams [58.77207336324662]
  UCS-Bench is a dataset spanning 170+ hours of egocentric visual observations with 8.1K+ timestamped questions.<n>We propose DirectMe, a framework that incrementally constructs and maintains a structured spatial memory from streaming egocentric observations.
  arXiv  Detail & Related papers  (2026-06-13T08:50:49Z)
• Spatia: Video Generation with Updatable Spatial Memory [60.21619361473996]
  Spatia is a spatial memory-aware video generation framework that preserves a 3D scene point cloud as persistent spatial memory.<n>Spatia iteratively generates video clips conditioned on this spatial memory and continuously updates it through visual SLAM.<n>Spatia enables applications such as explicit camera control and 3D-aware interactive editing, providing a geometrically grounded framework for scalable, memory-driven video generation.
  arXiv  Detail & Related papers  (2025-12-17T18:59:59Z)
• Vision-Language Memory for Spatial Reasoning [4.486751990718678]
  We present VLM$2$, a vision-language model with persistent memory for spatial reasoning.<n>We show that VLM$2$ achieves state-of-the-art performance among video-only models.
  arXiv  Detail & Related papers  (2025-11-25T18:59:02Z)
• Video World Models with Long-term Spatial Memory [110.530715838396]
  We introduce a novel framework to enhance long-term consistency of video world models.<n>Our framework includes mechanisms to store and retrieve information from the long-term spatial memory.<n>Our evaluations show improved quality, consistency, and context length compared to relevant baselines.
  arXiv  Detail & Related papers  (2025-06-05T17:42:34Z)
• 3DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Model [83.70640091897947]
  Humans excel at performing complex tasks by leveraging long-term memory across temporal and spatial experiences.<n>Current Large Language Models (LLMs) struggle to effectively plan and act in dynamic, multi-room 3D environments.<n>We propose 3DLLM-Mem, a novel dynamic memory management and fusion model for embodied spatial-temporal reasoning and actions.
  arXiv  Detail & Related papers  (2025-05-28T17:59:13Z)
• SpaceVLLM: Endowing Multimodal Large Language Model with Spatio-Temporal Video Grounding Capability [58.46310813774538]
  Large language models (LMLMs) have made remarkable progress in either temporal or spatial localization.<n>However they struggle to perform-temporal video grounding.<n>This limitation stems from two major challenges.<n>We introduce SpaceLM, a MLLMVL endowed with temporal-temporal video grounding.
  arXiv  Detail & Related papers  (2025-03-18T07:40:36Z)
• Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces [34.809309396448654]
  We present a novel video-based visual-spatial intelligence benchmark (VSI-Bench) of over 5,000 question-answer pairs.<n>We find that Multimodal Large Language Models (MLLMs) exhibit competitive - though subhuman - visual-spatial intelligence.
  arXiv  Detail & Related papers  (2024-12-18T18:59:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.