StereoVGGT: A Training-Free Visual Geometry Transformer for Stereo Vision

• URL: http://arxiv.org/abs/2603.29368v1
• Date: Tue, 31 Mar 2026 07:41:07 GMT
• Title: StereoVGGT: A Training-Free Visual Geometry Transformer for Stereo Vision
• Abstract summary: Visual Geometry Grounded Transformer (VGGT) operates as a foundation model pretrained on extensive 3D priors, including camera poses.<n>We propose StereoVGGT, a feature backbone specifically tailored for stereo vision.
• Score: 37.95373405958899
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Driven by the advancement of 3D devices, stereo vision tasks including stereo matching and stereo conversion have emerged as a critical research frontier. Contemporary stereo vision backbones typically rely on either monocular depth estimation (MDE) models or visual foundation models (VFMs). Crucially, these models are predominantly pretrained without explicit supervision of camera poses. Given that such geometric knowledge is indispensable for stereo vision, the absence of explicit spatial constraints constitutes a significant performance bottleneck for existing architectures. Recognizing that the Visual Geometry Grounded Transformer (VGGT) operates as a foundation model pretrained on extensive 3D priors, including camera poses, we investigate its potential as a robust backbone for stereo vision tasks. Nevertheless, empirical results indicate that its direct application to stereo vision yields suboptimal performance. We observe that VGGT suffers from a more significant degradation of geometric details during feature extraction. Such characteristics conflict with the requirements of binocular stereo vision, thereby constraining its efficacy for relative tasks. To bridge this gap, we propose StereoVGGT, a feature backbone specifically tailored for stereo vision. By leveraging the frozen VGGT and introducing a training-free feature adjustment pipeline, we mitigate geometric degradation and harness the latent camera calibration knowledge embedded within the model. StereoVGGT-based stereo matching network achieved the $1^{st}$ rank among all published methods on the KITTI benchmark, validating that StereoVGGT serves as a highly effective backbone for stereo vision.

Related papers

• Stereo World Model: Camera-Guided Stereo Video Generation [52.3922115596956]
  We present StereoWorld, a camera-conditioned stereo world model that jointly learns appearance and binocular geometry for end-to-end stereo video generation.<n>Unlike monocular RGB or RGBD approaches, StereoWorld operates exclusively within the RGB modality, while simultaneously grounding geometry directly from disparity.
  arXiv  Detail & Related papers  (2026-03-18T05:42:22Z)
• StereoVLA: Enhancing Vision-Language-Action Models with Stereo Vision [20.470652428992338]
  Stereo cameras closely mimic human binocular vision, providing rich spatial cues critical for precise robotic manipulation.<n>Despite their advantage, the adoption of stereo vision in vision-language-action models (VLAs) remains underexplored.<n>We present StereoVLA, a VLA model that leverages rich geometric cues from stereo vision.
  arXiv  Detail & Related papers  (2025-12-26T10:34:20Z)
• Empowering Dynamic Urban Navigation with Stereo and Mid-Level Vision [13.586199223564273]
  We show that relying on monocular vision and ignoring mid-level vision priors is inefficient.<n>We present StereoWalker, which augments NFMs with stereo inputs and explicit mid-level vision such as depth estimation and dense pixel tracking.<n>We find that mid-level vision enables StereoWalker to achieve a comparable performance as the state-of-the-art using only 1.5% of the training data, and surpasses the state-of-the-art using the full data.
  arXiv  Detail & Related papers  (2025-12-11T18:59:56Z)
• OmniVGGT: Omni-Modality Driven Visual Geometry Grounded Transformer [48.16706802780516]
  General 3D foundation models have started to lead the trend of unifying diverse vision tasks.<n>We introduce OmniVGGT, a novel framework that can effectively benefit from an arbitrary number of auxiliary modalities during both training and inference.
  arXiv  Detail & Related papers  (2025-11-13T17:59:01Z)
• GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision [49.839374549646884]
  This paper presents GEOcc, a Geometric-Enhanced Occupancy network tailored for vision-only surround-view perception.<n>Our approach achieves State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image resolution needed and the most weightless image backbone.
  arXiv  Detail & Related papers  (2024-05-17T07:31:20Z)
• Playing to Vision Foundation Model's Strengths in Stereo Matching [13.887661472501618]
  This study serves as the first exploration of a viable approach for adapting vision foundation models (VFMs) to stereo matching. Our ViT adapter, referred to as ViTAS, is constructed upon three types of modules: spatial differentiation, patch attention fusion, and cross-attention. ViTAStereo outperforms the second-best network StereoBase by approximately 7.9% in terms of the percentage of error pixels, with a tolerance of 3 pixels.
  arXiv  Detail & Related papers  (2024-04-09T12:34:28Z)
• Geometric-aware Pretraining for Vision-centric 3D Object Detection [77.7979088689944]
  We propose a novel geometric-aware pretraining framework called GAPretrain. GAPretrain serves as a plug-and-play solution that can be flexibly applied to multiple state-of-the-art detectors. We achieve 46.2 mAP and 55.5 NDS on the nuScenes val set using the BEVFormer method, with a gain of 2.7 and 2.1 points, respectively.
  arXiv  Detail & Related papers  (2023-04-06T14:33:05Z)
• Bridging Stereo Geometry and BEV Representation with Reliable Mutual Interaction for Semantic Scene Completion [45.171150395915056]
  3D semantic scene completion (SSC) is an ill-posed perception task that requires inferring a dense 3D scene from limited observations. Previous camera-based methods struggle to predict accurate semantic scenes due to inherent geometric ambiguity and incomplete observations. We resort to stereo matching technique and bird's-eye-view (BEV) representation learning to address such issues in SSC.
  arXiv  Detail & Related papers  (2023-03-24T12:33:44Z)
• SGM3D: Stereo Guided Monocular 3D Object Detection [62.11858392862551]
  We propose a stereo-guided monocular 3D object detection network, termed SGM3D. We exploit robust 3D features extracted from stereo images to enhance the features learned from the monocular image. Our method can be integrated into many other monocular approaches to boost performance without introducing any extra computational cost.
  arXiv  Detail & Related papers  (2021-12-03T13:57:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.