Cambrian-S: Towards Spatial Supersensing in Video

• URL: http://arxiv.org/abs/2511.04670v1
• Date: Thu, 06 Nov 2025 18:55:17 GMT
• Title: Cambrian-S: Towards Spatial Supersensing in Video
• Authors: Shusheng Yang, Jihan Yang, Pinzhi Huang, Ellis Brown, Zihao Yang, Yue Yu, Shengbang Tong, Zihan Zheng, Yifan Xu, Muhan Wang, Daohan Lu, Rob Fergus, Yann LeCun, Li Fei-Fei, Saining Xie,
• Abstract summary: We frame spatial supersensing as four stages beyond linguistic-only understanding: semantic perception, streaming event cognition, implicit 3D spatial cognition, and predictive world modeling.<n>To drive progress in spatial supersensing, we present-SUPER, a two-part benchmark: VSR (long-horizon visual spatial recall) and VSC (continual visual spatial counting)<n>We then test data scaling limits by curating-590K and training Cambrian-S, achieving +30% absolute improvement on arbitrarily general capabilities.<n>We propose predictive sensing as a path forward, presenting a proof-of-concept in which a self-supervised
• Score: 78.46305169769884
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We argue that progress in true multimodal intelligence calls for a shift from reactive, task-driven systems and brute-force long context towards a broader paradigm of supersensing. We frame spatial supersensing as four stages beyond linguistic-only understanding: semantic perception (naming what is seen), streaming event cognition (maintaining memory across continuous experiences), implicit 3D spatial cognition (inferring the world behind pixels), and predictive world modeling (creating internal models that filter and organize information). Current benchmarks largely test only the early stages, offering narrow coverage of spatial cognition and rarely challenging models in ways that require true world modeling. To drive progress in spatial supersensing, we present VSI-SUPER, a two-part benchmark: VSR (long-horizon visual spatial recall) and VSC (continual visual spatial counting). These tasks require arbitrarily long video inputs yet are resistant to brute-force context expansion. We then test data scaling limits by curating VSI-590K and training Cambrian-S, achieving +30% absolute improvement on VSI-Bench without sacrificing general capabilities. Yet performance on VSI-SUPER remains limited, indicating that scale alone is insufficient for spatial supersensing. We propose predictive sensing as a path forward, presenting a proof-of-concept in which a self-supervised next-latent-frame predictor leverages surprise (prediction error) to drive memory and event segmentation. On VSI-SUPER, this approach substantially outperforms leading proprietary baselines, showing that spatial supersensing requires models that not only see but also anticipate, select, and organize experience.

Related papers

• Solving Spatial Supersensing Without Spatial Supersensing [31.7966908405844]
  Cambrian-S aims to take the first steps towards improving video world models with spatial supersensing.<n>In this work, we conduct a critical analysis of Cambrian-S across two benchmarks.<n>We show that benchmarks like VSR can be nearly solved without spatial cognition, world modeling or spatial supersensing.
  arXiv  Detail & Related papers  (2025-11-20T18:57:05Z)
• Visual Spatial Tuning [98.96397681943255]
  We introduce Visual Spatial Tuning (VST), a comprehensive framework to cultivate Vision-Language Models (VLMs) with human-like visuospatial abilities.<n>In particular, we adopt a progressive training pipeline: supervised fine-tuning to build foundational spatial knowledge, followed by reinforcement learning to further improve spatial reasoning abilities.<n>It turns out that the Vision-Language-Action models can be significantly enhanced with proposed spatial tuning paradigm, paving the way for more physically grounded AI.
  arXiv  Detail & Related papers  (2025-11-07T18:59:16Z)
• SIMS-V: Simulated Instruction-Tuning for Spatial Video Understanding [64.86119288520419]
  multimodal language models struggle with spatial reasoning across time and space.<n>We present SIMS-V -- a systematic data-generation framework that leverages the privileged information of 3D simulators.<n>Our approach demonstrates robust generalization, maintaining performance on general video understanding while showing substantial improvements on embodied and real-world spatial tasks.
  arXiv  Detail & Related papers  (2025-11-06T18:53:31Z)
• Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks [108.15756345836901]
  We provide a comprehensive review of multimodal spatial reasoning tasks with large models.<n>We review advances in embodied AI, including vision-language navigation and action models.<n>We consider emerging modalities such as audio and egocentric video, which contribute to novel spatial understanding through new sensors.
  arXiv  Detail & Related papers  (2025-10-29T17:55:43Z)
• How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective [103.44502230776352]
  We present a systematic investigation of Visual Spatial Reasoning (VSR) in Vision-Language Models (VLMs)<n>We categorize spatial intelligence into three levels of capability, ie, basic perception, spatial understanding, spatial planning, and curate SIBench, a spatial intelligence benchmark encompassing nearly 20 open-source datasets across 23 task settings.
  arXiv  Detail & Related papers  (2025-09-23T12:00:14Z)
• See&Trek: Training-Free Spatial Prompting for Multimodal Large Language Model [33.18304419115947]
  We introduce SEE&TREK, the first training-free prompting framework to enhance the spatial understanding of Multimodal Large Language Models (MLLM) under vision-only constraints.<n>We focus on increasing visual diversity and motion reconstruction.<n>Our method is training&GPU-free, requiring only a single forward pass, and can be seamlessly integrated into existing MLLMS.
  arXiv  Detail & Related papers  (2025-09-19T15:30:26Z)
• LRR-Bench: Left, Right or Rotate? Vision-Language models Still Struggle With Spatial Understanding Tasks [22.011855291417856]
  It remains underexplored how Vision-Language Models (VLMs) recognize spatial relationships and perceive spatial movement.<n>In this work, we introduce a spatial evaluation pipeline and construct a corresponding benchmark.<n>We conduct experiments on multiple state-of-the-art VLMs and observe that there is significant room for improvement.
  arXiv  Detail & Related papers  (2025-07-27T08:31:24Z)
• ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models [68.46716645478661]
  Vision-language models (VLMs) have demonstrated remarkable capabilities in understanding and reasoning about visual content.<n>Current VLMs excel primarily at egocentric spatial reasoning (from the camera's perspective) but fail to generalize to allocentric viewpoints.<n>We introduce ViewSpatial-Bench, the first comprehensive benchmark designed specifically for multi-viewpoint spatial localization recognition evaluation.
  arXiv  Detail & Related papers  (2025-05-27T17:59:26Z)
• EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation [55.26713167507132]
  We introduce a generative robotics foundation model that constructs and interprets embodied spaces.<n>EnerVerse employs an autoregressive video diffusion framework to predict future embodied spaces from instructions, enhanced by a sparse context memory for long-term reasoning.<n>We present EnerVerse-D, a data engine pipeline combining the generative model with 4D Gaussian Splatting, forming a self-reinforcing data loop to reduce the sim-to-real gap.
  arXiv  Detail & Related papers  (2025-01-03T17:00:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.