Unified Panoramic Geometry Estimation via Multi-View Foundation Models

• URL: http://arxiv.org/abs/2605.26368v2
• Date: Wed, 27 May 2026 12:32:56 GMT
• Title: Unified Panoramic Geometry Estimation via Multi-View Foundation Models
• Abstract summary: PaGeR (Panoramic Geometry Reconstruction) is a framework to lift powerful 3D foundation models to the panorama domain.<n>Our strategy is to start from a pre-trained transformer for 3D reconstruction and turn it into a unified high-performance model.<n>We extensively test our method in both indoor and outdoor environments and find that it delivers state-of-the-art performance.
• Score: 22.160964470897003
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Geometry estimation from perspective images has greatly advanced, maturing to the point where off-the-shelf foundation models are able to reconstruct 3D scene structure not only from multi-view imagery, but even from a single view. A natural extension is 3D reconstruction from panoramas, with the exciting prospect of recovering a full 360-degree scene from a single panoramic image. In this work, we introduce PaGeR (Panoramic Geometry Reconstruction), a framework to lift powerful 3D foundation models designed for perspective imagery to the panorama domain. Our strategy is to start from a pre-trained transformer for 3D reconstruction and turn it into a unified high-performance model that predicts scale-invariant depth, metric depth, surface normals, and sky masks from both perspective and omnidirectional images, in a single forward pass. By keeping architectural changes to a minimum and mixing perspective and panoramic images during training, PaGeR retains the rich 3D prior of the underlying foundation model while learning to also estimate geometrically consistent 360-degree scenes from single panoramas. We extensively test our method in both indoor and outdoor environments and find that it delivers state-of-the-art performance and excellent zero-shot performance across a wide range of scenes. Code, data and models are available $\href{https://github.com/prs-eth/PaGeR}{\text{here}}$.

Related papers

• Pano2World: End-to-End 3D Generation via Unified Multi-View Sequences [9.89397063309918]
  We present Pano2World, which takes a single indoor panorama as input and directly outputs a persistent, explorable 3D Gaussian scene.<n> Experiments demonstrate that Pano2World significantly outperforms existing methods on the multi-position panoramic novel-view synthesis benchmark.
  arXiv  Detail & Related papers  (2026-07-01T11:54:02Z)
• Stepper: Stepwise Immersive Scene Generation with Multiview Panoramas [101.58587238272462]
  We present Stepper, a unified framework for text-driven immersive 3D scene synthesis.<n>Stepper is trained on a new large-scale, multi-view panorama dataset.<n>It achieves state-of-the-art fidelity and structural consistency, outperforming prior approaches.
  arXiv  Detail & Related papers  (2026-03-30T20:26:28Z)
• PanoVGGT: Feed-Forward 3D Reconstruction from Panoramic Imagery [41.23812453847979]
  PanoVGGT jointly predicts camera poses, depth maps, and 3D point clouds from one or multiple panoramas in a single forward pass.<n>In addition, we contribute PanoCity, a large-scale outdoor panoramic dataset with dense depth and 6-DoF pose annotations.
  arXiv  Detail & Related papers  (2026-03-18T10:24:06Z)
• OmniX: From Unified Panoramic Generation and Perception to Graphics-Ready 3D Scenes [57.790894531046796]
  Panorama-based 2D lifting has emerged as a promising technique to produce immersive, realistic, and diverse 3D environments.<n>In this work, we advance this technique to generate graphics-ready 3D scenes suitable for physically based rendering (PBR), relighting, and simulation.<n>Our key insight is to repurpose 2D generative models for panoramic perception of geometry, textures, and PBR materials.<n>Based on a lightweight and efficient cross-modal adapter structure, OmniX reuses 2D generative priors for a broad range of panoramic vision tasks.
  arXiv  Detail & Related papers  (2025-10-30T17:59:51Z)
• WorldMirror: Universal 3D World Reconstruction with Any-Prior Prompting [51.69408870574092]
  We present WorldMirror, an all-in-one, feed-forward model for versatile 3D geometric prediction tasks.<n>Our framework flexibly integrates diverse geometric priors, including camera poses, intrinsics, and depth maps.<n>WorldMirror achieves state-of-the-art performance across diverse benchmarks from camera, point map, depth, and surface normal estimation to novel view synthesis.
  arXiv  Detail & Related papers  (2025-10-12T17:59:09Z)
• TiP4GEN: Text to Immersive Panorama 4D Scene Generation [82.8444414014506]
  TiP4GEN is a text-to-dynamic panorama scene generation framework.<n>It enables fine-grained content control and synthesizes motion-rich, geometry-consistent panoramic 4D scenes.<n> TiP4GEN integrates panorama video generation and dynamic scene reconstruction to create 360-degree immersive virtual environments.
  arXiv  Detail & Related papers  (2025-08-17T16:02:24Z)
• DreamCube: 3D Panorama Generation via Multi-plane Synchronization [17.690754213112108]
  3D panorama synthesis is a promising yet challenging task that demands high-quality and diverse visual appearance and geometry of the generated omnidirectional content.<n>Existing methods leverage rich image priors from pre-trained 2D foundation models to circumvent the scarcity of 3D panoramic data.<n>In this work, we demonstrate that by applying multi-plane synchronization to the operators from 2D foundation models, their capabilities can be seamlessly extended to the omnidirectional domain.
  arXiv  Detail & Related papers  (2025-06-20T17:55:06Z)
• Splatter-360: Generalizable 360$^{\circ}$ Gaussian Splatting for Wide-baseline Panoramic Images [52.48351378615057]
  textitSplatter-360 is a novel end-to-end generalizable 3DGS framework to handle wide-baseline panoramic images.<n>We introduce a 3D-aware bi-projection encoder to mitigate the distortions inherent in panoramic images.<n>This enables robust 3D-aware feature representations and real-time rendering capabilities.
  arXiv  Detail & Related papers  (2024-12-09T06:58:31Z)
• Pano2Room: Novel View Synthesis from a Single Indoor Panorama [20.262621556667852]
  Pano2Room is designed to automatically reconstruct high-quality 3D indoor scenes from a single panoramic image. The key idea is to initially construct a preliminary mesh from the input panorama, and iteratively refine this mesh using a panoramic RGBD inpainter. The refined mesh is converted into a 3D Gaussian Splatting field and trained with the collected pseudo novel views.
  arXiv  Detail & Related papers  (2024-08-21T08:19:12Z)
• Scaled 360 layouts: Revisiting non-central panoramas [5.2178708158547025]
  We present a novel approach for 3D layout recovery of indoor environments using single non-central panoramas. We exploit the properties of non-central projection systems in a new geometrical processing to recover the scaled layout.
  arXiv  Detail & Related papers  (2024-02-02T14:55:36Z)
• PERF: Panoramic Neural Radiance Field from a Single Panorama [109.31072618058043]
  PERF is a novel view synthesis framework that trains a panoramic neural radiance field from a single panorama. We propose a novel collaborative RGBD inpainting method and a progressive inpainting-and-erasing method to lift up a 360-degree 2D scene to a 3D scene. Our PERF can be widely used for real-world applications, such as panorama-to-3D, text-to-3D, and 3D scene stylization applications.
  arXiv  Detail & Related papers  (2023-10-25T17:59:01Z)
• Learning 3D Face Reconstruction with a Pose Guidance Network [49.13404714366933]
  We present a self-supervised learning approach to learning monocular 3D face reconstruction with a pose guidance network (PGN) First, we unveil the bottleneck of pose estimation in prior parametric 3D face learning methods, and propose to utilize 3D face landmarks for estimating pose parameters. With our specially designed PGN, our model can learn from both faces with fully labeled 3D landmarks and unlimited unlabeled in-the-wild face images.
  arXiv  Detail & Related papers  (2020-10-09T06:11:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.