CoCoSI: Collaborative Cognitive Map Construction for Spatial Intelligence

• URL: http://arxiv.org/abs/2606.10401v2
• Date: Wed, 10 Jun 2026 09:54:52 GMT
• Title: CoCoSI: Collaborative Cognitive Map Construction for Spatial Intelligence
• Abstract summary: We propose a plug-and-play framework that collaboratively constructs cognitive maps as structured spatial memory.<n>Our framework features local-global agent coordination, cognitive map construction with atomic commits, and cross-agent verification.
• Score: 13.747177758222364
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Spatial intelligence is a key frontier for multimodal large language models (MLLMs), enabling them to reason about the physical world from visual experience. Inspired by human spatial cognition, recent approaches construct grid-based cognitive maps from multi-frame visual inputs to maintain coherent spatial representations over time. However, limited context lengths still challenge spatial understanding, while existing methods, such as long-context modeling and external memory, often require architectural changes, memory modules, or finetuning, limiting their applicability to off-the-shelf pretrained MLLMs. This motivates a lightweight, model-agnostic method for preserving spatial information beyond the native context window. To this end, we propose a plug-and-play multi-agent framework that collaboratively constructs cognitive maps as structured spatial memory, enhancing the spatial understanding of arbitrary pretrained MLLMs without architectural modification or additional training. Our framework features local-global agent coordination, cognitive map construction with atomic commits, and cross-agent verification. Extensive experiments demonstrate that our method achieves superior performance on spatial understanding tasks while remaining fully training-free. Code will be released.

Related papers

• SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem [40.48570356491105]
  Large Vision-Language Models (LVLMs) have achieved strong performance on diverse visual tasks.<n>But their ability to reconstruct and reason about the 3D structure of the scene depicted in 2D images remains limited.<n>We propose a novel paradigm inspired by human cognitive development: learning foundational spatial skills through structured block-manipulation tasks.
  arXiv  Detail & Related papers  (2026-09-07T05:41:13Z)
• OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping [69.48762031013514]
  We propose OmniView-Space, a framework designed to maintain spatial consistency through multimodal egocentric evidence.<n>Our approach consists of three core components: (1) Multi-Perspective Spatial Mapping (MPSM), which re-anchors reconstructed geometry into a query-aligned visual cognitive map and a textual spatial graph; (2) Tool-Guided Egocentric Reasoning, an interleaved policy trained to actively select the ego anchor required by the query and request the corresponding MPSM evidence; and (3) Cognitive-Map Distillation, which uses MPSM-generated trajectories and ego-frame rewards to train the model to reason with self-generated cognitive maps
  arXiv  Detail & Related papers  (2026-07-01T12:45:12Z)
• SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning [67.67774742200626]
  spatial intelligence, which refers to the ability to reason about geometric and physical structure from visual observations, is a core challenge for large language models.<n>We propose a unified multimodal generation framework that integrates textual reasoning with visual imagination.<n>Our framework adopts a divide-and-conquer strategy, using text chain-of-thought for high-level semantic planning and the visual imagination for geometry-sensitive state transformation and consistency preservation.
  arXiv  Detail & Related papers  (2026-04-19T11:21:59Z)
• OnlineSI: Taming Large Language Model for Online 3D Understanding and Grounding [53.33067495235966]
  OnlineSI is a framework that can improve its spatial understanding of its surroundings given a video stream.<n>Our core idea is to maintain a finite spatial memory to retain past observations.<n>We further integrate 3D point cloud information with semantic information, helping MLLM to better locate and identify objects in the scene.
  arXiv  Detail & Related papers  (2026-01-23T08:17:57Z)
• The AI Hippocampus: How Far are We From Human Memory? [77.04745635827278]
  Implicit memory refers to the knowledge embedded within the internal parameters of pre-trained transformers.<n>Explicit memory involves external storage and retrieval components designed to augment model outputs with dynamic, queryable knowledge representations.<n>Agentic memory introduces persistent, temporally extended memory structures within autonomous agents.
  arXiv  Detail & Related papers  (2026-01-14T03:24:08Z)
• COOPER: A Unified Model for Cooperative Perception and Reasoning in Spatial Intelligence [57.63155257058967]
  We investigate whether a unified MLLM can develop an intrinsic ability to enhance spatial perception and, through adaptive interleaved reasoning, achieve stronger spatial intelligence.<n>We propose textbfCOOPER, a unified MLLM that leverages depth and segmentation as auxiliary modalities and is trained in two stages to acquire auxiliary modality generation and adaptive, interleaved reasoning capabilities.
  arXiv  Detail & Related papers  (2025-12-04T08:26:04Z)
• Words into World: A Task-Adaptive Agent for Language-Guided Spatial Retrieval in AR [8.295391485284298]
  We present a modular augmented reality (AR) agent system that integrates multimodal large language models (MLLMs) with grounded vision models.<n>Our adaptive task agent coordinates MLLMs and coordinate-aware perception tools to address varying query complexities.<n>The system guides human attention to information-dense areas while supporting human-in-the-loop refinement.
  arXiv  Detail & Related papers  (2025-11-29T03:29:15Z)
• SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition [19.526371771173064]
  spatial cognition is fundamental to real-world multimodal intelligence, allowing models to interact with the physical environment.<n>Existing benchmarks often oversimplify spatial cognition, reducing it to a single-dimensional metric.<n>We propose a hierarchical spatial cognition framework that decomposes spatial intelligence into five progressively complex levels.
  arXiv  Detail & Related papers  (2025-11-26T15:04:18Z)
• Why Do MLLMs Struggle with Spatial Understanding? A Systematic Analysis from Data to Architecture [16.15618237704827]
  We present a systematic analysis of spatial understanding from both data and architectural perspectives.<n>From the data perspective, the performance of spatial understanding converges quickly as the training data increases.<n>From the architectural perspective, we find that spatial understanding relies more heavily on the positional encoding within the visual encoder than within the language model.
  arXiv  Detail & Related papers  (2025-09-02T14:22:43Z)
• From reactive to cognitive: brain-inspired spatial intelligence for embodied agents [50.99942960312313]
  Brain-inspired Spatial Cognition for Navigation (BSC-Nav) is a unified framework for constructing and leveraging structured spatial memory in embodied agents.<n> BSC-Nav builds allocentric cognitive maps from egocentric trajectories and contextual cues, and dynamically retrieves spatial knowledge aligned with semantic goals.
  arXiv  Detail & Related papers  (2025-08-24T03:20:48Z)
• Spatial Knowledge Graph-Guided Multimodal Synthesis [78.11669780958657]
  We introduce a novel multimodal synthesis approach guided by spatial knowledge graphs, grounded in the concept of knowledge-to-data generation.<n>In experiments, data synthesized from diverse types of spatial knowledge, including direction and distance, enhance the spatial perception and reasoning abilities of MLLMs markedly.<n>We hope that the idea of knowledge-based data synthesis can advance the development of spatial intelligence.
  arXiv  Detail & Related papers  (2025-05-28T17:50:21Z)
• EmbodiedVSR: Dynamic Scene Graph-Guided Chain-of-Thought Reasoning for Visual Spatial Tasks [24.41705039390567]
  EmbodiedVSR (Embodied Visual Spatial Reasoning) is a novel framework that integrates dynamic scene graph-guided Chain-of-Thought (CoT) reasoning.<n>Our method enables zero-shot spatial reasoning without task-specific fine-tuning.<n>Experiments demonstrate that our framework significantly outperforms existing MLLM-based methods in accuracy and reasoning coherence.
  arXiv  Detail & Related papers  (2025-03-14T05:06:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.