DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images

• URL: http://arxiv.org/abs/2606.12368v2
• Date: Thu, 11 Jun 2026 05:24:02 GMT
• Title: DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images
• Abstract summary: We introduce DepthMaster, a unified metric depth estimation framework.<n>We reformulate the problem by decomposing panoramic images into overlapping perspective patches.<n> DepthMaster achieves state-of-the-art zero-shot performance on 13 diverse datasets.
• Score: 20.087102336395173
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While monocular depth estimation has achieved significant progress, achieving generalized metric depth estimation for both narrow field-of-view (FoV) perspectives and $360^\circ$ panoramas remains an unsolved challenge. Existing methods are often tailored to specific camera types and struggle to produce accurate metric depth that generalizes across diverse settings. This limitation stems from two key challenges: the inherent geometric discrepancy between perspective and panoramic cameras, and the scarcity of panoramic training data with metric annotations. In this work, we introduce DepthMaster, a unified metric depth estimation framework. Rather than employing specialized networks to learn spherical distortions, we reformulate the problem by decomposing panoramic images into overlapping perspective patches. Crucially, distinct from prior projection-based methods that rely on ad-hoc architectural modifications to handle boundaries, we introduce a novel Correspondence Consistency Loss (CCL) and inject virtual projection cameras as geometric priors, allowing us to seamlessly stitch the patches while avoiding specialized operators and keeping the backbone largely compatible with standard Transformer designs. This strategy also resolves the geometric differences by unifying all inputs into a canonical perspective representation, and effectively circumvents data scarcity by directly unlocking powerful metric priors from vast perspective datasets. Trained on a mixed dataset that contains only one panorama dataset, DepthMaster achieves state-of-the-art zero-shot performance on 13 diverse datasets, outperforming not only universal methods but also leading specialist models in both perspective and panoramic domains.

Related papers

• OmniPoint: Universal Monocular Metric Pointcloud from Any Camera [79.24417597712802]
  We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors.<n>To overcome projection rigidity, OmniPoint abandons conventional planar depth regression.<n>It instead adopts a decoupled ray and distance representation alongside a decoupled training objective.
  arXiv  Detail & Related papers  (2026-09-08T19:45:58Z)
• Geometric Distillation from Rectified Stereo: Leveraging Epipolar Cues for Monocular Depth [11.871264497562008]
  We propose a novel framework that transfers the scale-aware geometric priors of multi-view models into monocular depth foundation models.<n>Specifically, we introduce an Epipolar Distillation (EpiDistill), which enables the single-view prediction model to retain epipolar attention patterns and maintain geometric consistency without requiring multi-view inputs at inference.
  arXiv  Detail & Related papers  (2026-07-17T03:49:14Z)
• Pano360: Perspective to Panoramic Vision with Geometric Consistency [7.713672589538202]
  We adopt a novel transformer-based architecture to achieve 3D awareness and aggregate global information across all views.<n>To establish an evaluation benchmark and train our network, we constructed a large-scale dataset of real-world scenes.
  arXiv  Detail & Related papers  (2026-03-12T14:56:14Z)
• DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training [76.82789568988557]
  DiT360 is a DiT-based framework that performs hybrid training on perspective and panoramic data for panoramic image generation.<n>Our method achieves better boundary consistency and image fidelity across eleven quantitative metrics.
  arXiv  Detail & Related papers  (2025-10-13T17:59:15Z)
• MCPDepth: Omnidirectional Depth Estimation via Stereo Matching from Multi-Cylindrical Panoramas [49.891712558113845]
  We introduce Multi-Cylindrical Panoramic Depth Estimation (MCPDepth)<n>MCPDepth is a two-stage framework designed to enhance omnidirectional depth estimation.<n>Our method improves the mean absolute error (MAE) by 18.8% on the outdoor dataset Deep360 and by 19.9% on the real dataset 3D60.
  arXiv  Detail & Related papers  (2024-08-03T03:35:37Z)
• Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation [74.28509379811084]
  Metric3D v2 is a geometric foundation model for zero-shot metric depth and surface normal estimation from a single image.<n>We propose solutions for both metric depth estimation and surface normal estimation.<n>Our method enables the accurate recovery of metric 3D structures on randomly collected internet images.
  arXiv  Detail & Related papers  (2024-03-22T02:30:46Z)
• 360 Layout Estimation via Orthogonal Planes Disentanglement and Multi-view Geometric Consistency Perception [56.84921040837699]
  Existing panoramic layout estimation solutions tend to recover room boundaries from a vertically compressed sequence, yielding imprecise results. We propose an orthogonal plane disentanglement network (termed DOPNet) to distinguish ambiguous semantics. We also present an unsupervised adaptation technique tailored for horizon-depth and ratio representations. Our solution outperforms other SoTA models on both monocular layout estimation and multi-view layout estimation tasks.
  arXiv  Detail & Related papers  (2023-12-26T12:16:03Z)
• FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models [67.96827539201071]
  We propose a novel test-time optimization approach for 3D scene reconstruction. Our method achieves state-of-the-art cross-dataset reconstruction on five zero-shot testing datasets.
  arXiv  Detail & Related papers  (2023-08-10T17:55:02Z)
• Multi-Frame Self-Supervised Depth with Transformers [33.00363651105475]
  We propose a novel transformer architecture for cost volume generation. We use depth-discretized epipolar sampling to select matching candidates. We refine predictions through a series of self- and cross-attention layers.
  arXiv  Detail & Related papers  (2022-04-15T19:04:57Z)
• Practical Wide-Angle Portraits Correction with Deep Structured Models [17.62752136436382]
  This paper introduces the first deep learning based approach to remove perspective distortions from photos. Given a wide-angle portrait as input, we build a cascaded network consisting of a LineNet, a ShapeNet, and a transition module. For the quantitative evaluation, we introduce two novel metrics, line consistency and face congruence.
  arXiv  Detail & Related papers  (2021-04-26T10:47:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.