Stereo World Model: Camera-Guided Stereo Video Generation

• URL: http://arxiv.org/abs/2603.17375v1
• Date: Wed, 18 Mar 2026 05:42:22 GMT
• Title: Stereo World Model: Camera-Guided Stereo Video Generation
• Abstract summary: We present StereoWorld, a camera-conditioned stereo world model that jointly learns appearance and binocular geometry for end-to-end stereo video generation.<n>Unlike monocular RGB or RGBD approaches, StereoWorld operates exclusively within the RGB modality, while simultaneously grounding geometry directly from disparity.
• Score: 52.3922115596956
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present StereoWorld, a camera-conditioned stereo world model that jointly learns appearance and binocular geometry for end-to-end stereo video generation.Unlike monocular RGB or RGBD approaches, StereoWorld operates exclusively within the RGB modality, while simultaneously grounding geometry directly from disparity. To efficiently achieve consistent stereo generation, our approach introduces two key designs: (1) a unified camera-frame RoPE that augments latent tokens with camera-aware rotary positional encoding, enabling relative, view- and time-consistent conditioning while preserving pretrained video priors via a stable attention initialization; and (2) a stereo-aware attention decomposition that factors full 4D attention into 3D intra-view attention plus horizontal row attention, leveraging the epipolar prior to capture disparity-aligned correspondences with substantially lower compute. Across benchmarks, StereoWorld improves stereo consistency, disparity accuracy, and camera-motion fidelity over strong monocular-then-convert pipelines, achieving more than 3x faster generation with an additional 5% gain in viewpoint consistency. Beyond benchmarks, StereoWorld enables end-to-end binocular VR rendering without depth estimation or inpainting, enhances embodied policy learning through metric-scale depth grounding, and is compatible with long-video distillation for extended interactive stereo synthesis.

Related papers

• TSR-Ego: Temporally Guided Stereo Refinement Framework for Egocentric 3D Human Pose Estimation [11.399976049695212]
  Egocentric 3D human pose estimation from head-mounted stereo cameras is challenging due to fisheye distortion, severe self-occlusion, and frequent truncation of body joints.<n>We propose TSR-Ego, a temporally guided stereo framework that couples short-term motion evidence with projection-guided feature sampling.<n>Experiments on UnrealEgo2 and UnrealEgo-RW show state-of-the-art performance, with especially strong gains on real-world sequences.
  arXiv  Detail & Related papers  (2026-07-10T07:56:30Z)
• H-OmniStereo: Zero-Shot Omnidirectional Stereo Matching with Heading-Aligned Normal Priors [39.6437115448559]
  H- OmniStereo is a zero-shot omnidirectional stereo matching framework.<n>We construct high-quality synthetic dataset comprising over 2.8 million top-bottom equirectangular stereo pairs.<n>Our approach achieves higher accuracy than existing methods on out-of-domain datasets.
  arXiv  Detail & Related papers  (2026-05-14T15:30:22Z)
• StereoPilot: Learning Unified and Efficient Stereo Conversion via Generative Priors [41.34827274890319]
  We introduce UniStereo, the first large-scale unified dataset for stereo video conversion.<n>We propose StereoPilot, an efficient feed-forward model that directly synthesizes the target view without relying on explicit depth maps.<n>Experiments demonstrate that StereoPilot significantly outperforms state-of-the-art methods in both visual fidelity and computational efficiency.
  arXiv  Detail & Related papers  (2025-12-18T18:59:50Z)
• StereoWorld: Geometry-Aware Monocular-to-Stereo Video Generation [108.97993219426509]
  StereoWorld is an end-to-end framework for high-fidelity monocular-to-stereo video generation.<n>Our framework conditions the model on the monocular video input while explicitly supervising the generation with a geometry-aware regularization.<n>To enable large-scale training and evaluation, we curate a high-definition stereo video dataset.
  arXiv  Detail & Related papers  (2025-12-10T06:50:16Z)
• Boosting Omnidirectional Stereo Matching with a Pre-trained Depth Foundation Model [70.67610495024459]
  Camera-based setups offer a cost-effective option by using stereo depth estimation to generate dense, high-resolution depth maps.<n>Existing omnidirectional stereo matching approaches achieve only limited depth accuracy across diverse environments.<n>We present DFI-OmniStereo, a novel omnidirectional stereo matching method that leverages a large-scale pre-trained foundation model for relative monocular depth estimation.
  arXiv  Detail & Related papers  (2025-03-30T16:24:22Z)
• Mono2Stereo: A Benchmark and Empirical Study for Stereo Conversion [88.67015254278859]
  We introduce the Mono2Stereo dataset, providing high-quality training data and benchmark to support in-depth exploration of stereo conversion.<n>We conduct an empirical study that yields two primary findings. 1) The differences between the left and right views are subtle, yet existing metrics consider overall pixels, failing to concentrate on regions critical to stereo effects.<n>We introduce a new evaluation metric, Stereo Intersection-over-Union, which harmonizes disparity and achieves a high correlation with human judgments on stereo effect.
  arXiv  Detail & Related papers  (2025-03-28T09:25:58Z)
• Stereo Any Video: Temporally Consistent Stereo Matching [15.876953256378224]
  This paper introduces Stereo Any Video, a powerful framework for video stereo matching.<n>It can estimate spatially accurate and temporally consistent disparities without relying on auxiliary information such as camera poses or optical flow.<n>Key architectural innovations are introduced: all-to-all-pairs correlation, which constructs smooth and robust matching cost volumes, and temporal convex upsampling, which improves temporal coherence.
  arXiv  Detail & Related papers  (2025-03-07T16:20:36Z)
• Stereo Anything: Unifying Zero-shot Stereo Matching with Large-Scale Mixed Data [77.27700893908012]
  Stereo matching serves as a cornerstone in 3D vision, aiming to establish pixel-wise correspondences between stereo image pairs for depth recovery.<n>Current models often exhibit severe performance degradation when deployed in unseen domains.<n>We introduce StereoAnything, a data-centric framework that substantially enhances the zero-shot generalization capability of existing stereo models.
  arXiv  Detail & Related papers  (2024-11-21T11:59:04Z)
• Stereo Matching in Time: 100+ FPS Video Stereo Matching for Extended Reality [65.70936336240554]
  Real-time Stereo Matching is a cornerstone algorithm for many Extended Reality (XR) applications, such as indoor 3D understanding, video pass-through, and mixed-reality games. One of the major difficulties is the lack of high-quality indoor video stereo training datasets captured by head-mounted VR/AR glasses. We introduce a novel video stereo synthetic dataset that comprises renderings of various indoor scenes and realistic camera motion captured by a 6-DoF moving VR/AR head-mounted display (HMD). This facilitates the evaluation of existing approaches and promotes further research on indoor augmented reality scenarios.
  arXiv  Detail & Related papers  (2023-09-08T07:53:58Z)
• Video Frame Interpolation with Stereo Event and Intensity Camera [40.07341828127157]
  We propose a novel Stereo Event-based VFI network (SE-VFI-Net) to generate high-quality intermediate frames. We exploit the fused features accomplishing accurate optical flow and disparity estimation. Our proposed SEVFI-Net outperforms state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2023-07-17T04:02:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.