PanoVGGT: Feed-Forward 3D Reconstruction from Panoramic Imagery

• URL: http://arxiv.org/abs/2603.17571v1
• Date: Wed, 18 Mar 2026 10:24:06 GMT
• Title: PanoVGGT: Feed-Forward 3D Reconstruction from Panoramic Imagery
• Abstract summary: PanoVGGT jointly predicts camera poses, depth maps, and 3D point clouds from one or multiple panoramas in a single forward pass.<n>In addition, we contribute PanoCity, a large-scale outdoor panoramic dataset with dense depth and 6-DoF pose annotations.
• Score: 41.23812453847979
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Panoramic imagery offers a full 360° field of view and is increasingly common in consumer devices. However, it introduces non-pinhole distortions that challenge joint pose estimation and 3D reconstruction. Existing feed-forward models, built for perspective cameras, generalize poorly to this setting. We propose PanoVGGT, a permutation-equivariant Transformer framework that jointly predicts camera poses, depth maps, and 3D point clouds from one or multiple panoramas in a single forward pass. The model incorporates spherical-aware positional embeddings and a panorama-specific three-axis SO(3) rotation augmentation, enabling effective geometric reasoning in the spherical domain. To resolve inherent global-frame ambiguity, we further introduce a stochastic anchoring strategy during training. In addition, we contribute PanoCity, a large-scale outdoor panoramic dataset with dense depth and 6-DoF pose annotations. Extensive experiments on PanoCity and standard benchmarks demonstrate that PanoVGGT achieves competitive accuracy, strong robustness, and improved cross-domain generalization. Code and dataset will be released.

Related papers

• Geometry and Gradient-based Partitioning for Panoramic Outdoor Reconstruction [32.80582033500264]
  PanoLOG is a two-stage coarse-to-fine framework equipped with a Geometry and Gradient-based Partitioning Strategy.<n>We construct Pano360, the first benchmark on large-scale panoramic dataset for outdoor scene reconstruction.
  arXiv  Detail & Related papers  (2026-07-09T17:59:43Z)
• Pano2World: End-to-End 3D Generation via Unified Multi-View Sequences [9.89397063309918]
  We present Pano2World, which takes a single indoor panorama as input and directly outputs a persistent, explorable 3D Gaussian scene.<n> Experiments demonstrate that Pano2World significantly outperforms existing methods on the multi-position panoramic novel-view synthesis benchmark.
  arXiv  Detail & Related papers  (2026-07-01T11:54:02Z)
• PanoImager: Geometry-Guided Novel View Synthesis and Reconstruction from Sparse Panoramic Views [3.5844726281072448]
  PanoImager is an SfM-free framework that combines feed-forward pose/depth priors, geometry-conditioned diffusion view completion, and depth-guided 3DGS optimization.<n>We show improved stability under extreme sparsity, suggesting PanoImager as an offline/background component for map refinement when SfM/SLAM fails to initialize.
  arXiv  Detail & Related papers  (2026-06-25T14:15:09Z)
• Unified Panoramic Geometry Estimation via Multi-View Foundation Models [22.160964470897003]
  PaGeR (Panoramic Geometry Reconstruction) is a framework to lift powerful 3D foundation models to the panorama domain.<n>Our strategy is to start from a pre-trained transformer for 3D reconstruction and turn it into a unified high-performance model.<n>We extensively test our method in both indoor and outdoor environments and find that it delivers state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-05-25T22:32:42Z)
• Pano360: Perspective to Panoramic Vision with Geometric Consistency [7.713672589538202]
  We adopt a novel transformer-based architecture to achieve 3D awareness and aggregate global information across all views.<n>To establish an evaluation benchmark and train our network, we constructed a large-scale dataset of real-world scenes.
  arXiv  Detail & Related papers  (2026-03-12T14:56:14Z)
• Spherical-GOF: Geometry-Aware Panoramic Gaussian Opacity Fields for 3D Scene Reconstruction [17.20174890101058]
  We present Spherical-GOF, an omnidirectional rendering framework built upon Gaussian Opacity Fields (GOF)<n>GOF performs GOF ray sampling directly on the unit sphere in spherical space, enabling consistent ray-Gaussian interactions for panoramic rendering.<n>Experiments on standard panoramic benchmarks demonstrate competitive photometric quality and substantially improved consistency.
  arXiv  Detail & Related papers  (2026-03-09T15:35:56Z)
• CylinderSplat: 3D Gaussian Splatting with Cylindrical Triplanes for Panoramic Novel View Synthesis [53.231427733457224]
  CylinderSplat is a feed-forward framework for panoramic 3DGS.<n>It achieves state-of-the-art results in both single-view and multi-view panoramic novel view synthesis.
  arXiv  Detail & Related papers  (2026-03-06T04:04:20Z)
• You Need a Transition Plane: Bridging Continuous Panoramic 3D Reconstruction with Perspective Gaussian Splatting [57.44295803750027]
  We present a novel framework, named TPGS, to bridge continuous panoramic 3D scene reconstruction with perspective Gaussian splatting.<n>Specifically, we optimize 3D Gaussians within individual cube faces and then fine-tune them in the stitched panoramic space.<n>Experiments on indoor and outdoor, egocentric, and roaming benchmark datasets demonstrate that our approach outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-04-12T03:42:50Z)
• Splatter-360: Generalizable 360$^{\circ}$ Gaussian Splatting for Wide-baseline Panoramic Images [52.48351378615057]
  textitSplatter-360 is a novel end-to-end generalizable 3DGS framework to handle wide-baseline panoramic images.<n>We introduce a 3D-aware bi-projection encoder to mitigate the distortions inherent in panoramic images.<n>This enables robust 3D-aware feature representations and real-time rendering capabilities.
  arXiv  Detail & Related papers  (2024-12-09T06:58:31Z)
• DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion [60.45000652592418]
  We propose a novel text-driven panoramic generation framework, DiffPano, to achieve scalable, consistent, and diverse panoramic scene generation. We show that DiffPano can generate consistent, diverse panoramic images with given unseen text descriptions and camera poses.
  arXiv  Detail & Related papers  (2024-10-31T17:57:02Z)
• PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramas [54.4948540627471]
  We propose PanoGRF, Generalizable Spherical Radiance Fields for Wide-baseline Panoramas. Unlike generalizable radiance fields trained on perspective images, PanoGRF avoids the information loss from panorama-to-perspective conversion. Results on multiple panoramic datasets demonstrate that PanoGRF significantly outperforms state-of-the-art generalizable view synthesis methods.
  arXiv  Detail & Related papers  (2023-06-02T13:35:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.