Cambrian-P: Pose-Grounded Video Understanding

• URL: http://arxiv.org/abs/2605.22819v1
• Date: Thu, 21 May 2026 17:59:45 GMT
• Title: Cambrian-P: Pose-Grounded Video Understanding
• Abstract summary: We introduce CambrianP, a video MLLM augmented with per-frame learnable camera tokens and a pose regression head.<n>With a carefully designed sampling scheme, the model achieves substantial gains on spatial reasoning benchmarks.<n>Surprisingly, training on pseudo-annotated poses in the wild video further improves general video QA benchmarks.
• Score: 51.130482325090064
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Camera pose matters. The position and orientation of each viewpoint define a shared spatial coordinate frame that relates observations across video frames. Yet this signal is largely absent from multimodal LLMs (MLLMs) for video understanding, which process frames as isolated 2D snapshots, instead of the persistent scene humans perceive. We revisit pose as a lightweight supervisory signal and introduce Cambrian-P, a video MLLM augmented with per-frame learnable camera tokens and a pose regression head. With a carefully designed sampling scheme, the model achieves substantial gains of 4.5-6.5% on spatial reasoning benchmarks such as VSI-Bench, generalizes across eight additional spatial and general video QA benchmarks, and, as a byproduct, achieves state of the art streaming pose estimation on ScanNet. Surprisingly, training on pseudo-annotated poses from in-the-wild video further improves general video QA benchmarks, showing pose helps beyond spatial reasoning. Together, these results position camera pose as a fundamental signal for video models that reason about the physical world.

Related papers

• Geometry-Guided Camera Motion Understanding in VideoLLMs [8.928448861727274]
  Camera motion is a fundamental geometric signal that shapes visual perception and cinematic style.<n>Current video-capable vision-injection models (VideoLLMs) rarely represent it explicitly and often fail on fine-grained motion primitives.<n>We formulate camera motion as constraint-aware multi-label recognition.<n> Experiments demonstrate improved motion recognition and more camera-aware model responses.
  arXiv  Detail & Related papers  (2026-03-13T16:13:09Z)
• On the Generalization Capacities of MLLMs for Spatial Intelligence [72.21075026598761]
  We argue that RGB-only approaches are fundamentally flawed in their ability to generalize across cameras.<n>We show this leads MLLMs to overfit to the training camera distribution, rather than learning true and generalizable 3D geometric principles.<n>We propose Camera-Aware MLLM framework for spatial MLLMs.
  arXiv  Detail & Related papers  (2026-03-05T14:46:11Z)
• PoseAnything: Universal Pose-guided Video Generation with Part-aware Temporal Coherence [67.78835640962167]
  Pose-guided video generation refers to controlling the motion of subjects in generated video through a sequence of poses.<n>We propose PoseAnything, the first universal pose-guided video generation framework capable of handling both human and non-human characters.<n>We present XPose, a high-quality public dataset containing 50,000 non-human pose-video pairs, along with an automated pipeline for annotation and filtering.
  arXiv  Detail & Related papers  (2025-12-15T16:03:26Z)
• SEE4D: Pose-Free 4D Generation via Auto-Regressive Video Inpainting [83.5106058182799]
  We introduce SEE4D, a pose-free, trajectory-to-camera framework for 4D world modeling from casual videos.<n>A view-conditional video in model is trained to learn a robust geometry prior to denoising realistically synthesized images.<n>We validate See4D on cross-view video generation and sparse reconstruction benchmarks.
  arXiv  Detail & Related papers  (2025-10-30T17:59:39Z)
• SmallGS: Gaussian Splatting-based Camera Pose Estimation for Small-Baseline Videos [9.416105948337291]
  SmallGS is a camera pose estimation framework specifically designed for small-baseline videos.<n>It learns camera poses without explicit feature correspondences or strong parallax motion.<n>It achieves impressive accuracy in camera pose estimation compared to MonST3R and DORID-SLAM for small-baseline videos in dynamic scenes.
  arXiv  Detail & Related papers  (2025-04-22T18:00:00Z)
• Towards Understanding Camera Motions in Any Video [89.97247162415158]
  We introduce CameraBench, a large-scale dataset and benchmark designed to assess and improve camera motion understanding.<n>CameraBench consists of 3,000 diverse internet videos annotated by experts through a rigorous quality control process.<n>One of our contributions is a taxonomy of camera motion primitives, designed in collaboration with cinematographers.
  arXiv  Detail & Related papers  (2025-04-21T18:34:57Z)
• Self-learning Canonical Space for Multi-view 3D Human Pose Estimation [57.969696744428475]
  Multi-view 3D human pose estimation is naturally superior to single view one. The accurate annotation of these information is hard to obtain. We propose a fully self-supervised framework, named cascaded multi-view aggregating network (CMANet) CMANet is superior to state-of-the-art methods in extensive quantitative and qualitative analysis.
  arXiv  Detail & Related papers  (2024-03-19T04:54:59Z)
• Coarse-to-Fine Multi-Scene Pose Regression with Transformers [19.927662512903915]
  A convolutional backbone with a multi-layer perceptron (MLP) head is trained using images and pose labels to embed a single reference at a time. We propose to learn multi-scene absolute camera pose regression with Transformers, where encoders are used to aggregate activation maps with self-attention. Our method is evaluated on commonly benchmark indoor and outdoor datasets and has been shown to exceed both multi-scene and state-of-the-art single-scene absolute pose regressors.
  arXiv  Detail & Related papers  (2023-08-22T20:43:31Z)
• FlowCam: Training Generalizable 3D Radiance Fields without Camera Poses via Pixel-Aligned Scene Flow [26.528667940013598]
  Reconstruction of 3D neural fields from posed images has emerged as a promising method for self-supervised representation learning. Key challenge preventing the deployment of these 3D scene learners on large-scale video data is their dependence on precise camera poses from structure-from-motion. We propose a method that jointly reconstructs camera poses and 3D neural scene representations online and in a single forward pass.
  arXiv  Detail & Related papers  (2023-05-31T20:58:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.