PanoNormal: Monocular Indoor 360° Surface Normal Estimation

• URL: http://arxiv.org/abs/2405.18745v1
• Date: Wed, 29 May 2024 04:07:14 GMT
• Title: PanoNormal: Monocular Indoor 360° Surface Normal Estimation
• Authors: Kun Huang, Fanglue Zhang, Neil Dodgson,
• Abstract summary: textitPanoNormal is a monocular surface normal estimation architecture designed for 360deg images. We employ a multi-level global self-attention scheme with the consideration of the spherical feature distribution. Our results demonstrate that our approach achieves state-of-the-art performance across multiple popular 360deg monocular datasets.
• Score: 12.992217830651988
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The presence of spherical distortion on the Equirectangular image is an acknowledged challenge in dense regression computer vision tasks, such as surface normal estimation. Recent advances in convolutional neural networks (CNNs) strive to mitigate spherical distortion but often fall short in capturing holistic structures effectively, primarily due to their fixed receptive field. On the other hand, vision transformers (ViTs) excel in establishing long-range dependencies through a global self-attention mechanism, yet they encounter limitations in preserving local details. We introduce \textit{PanoNormal}, a monocular surface normal estimation architecture designed for 360{\deg} images, which combines the strengths of CNNs and ViTs. Specifically, we employ a multi-level global self-attention scheme with the consideration of the spherical feature distribution, enhancing the comprehensive understanding of the scene. Our experimental results demonstrate that our approach achieves state-of-the-art performance across multiple popular 360{\deg} monocular datasets. The code and models will be released.

Related papers

• GRIN: Zero-Shot Metric Depth with Pixel-Level Diffusion [27.35300492569507]
  We present GRIN, an efficient diffusion model designed to ingest sparse unstructured training data. We show that GRIN establishes a new state of the art in zero-shot metric monocular depth estimation even when trained from scratch.
  arXiv  Detail & Related papers  (2024-09-15T23:32:04Z)
• GeoGen: Geometry-Aware Generative Modeling via Signed Distance Functions [22.077366472693395]
  We introduce a new generative approach for synthesizing 3D geometry and images from single-view collections. By employing volumetric rendering using neural radiance fields, they inherit a key limitation: the generated geometry is noisy and unconstrained. We propose GeoGen, a new SDF-based 3D generative model trained in an end-to-end manner.
  arXiv  Detail & Related papers  (2024-06-06T17:00:10Z)
• SGFormer: Spherical Geometry Transformer for 360 Depth Estimation [54.13459226728249]
  Panoramic distortion poses a significant challenge in 360 depth estimation. We propose a spherical geometry transformer, named SGFormer, to address the above issues. We also present a query-based global conditional position embedding to compensate for spatial structure at varying resolutions.
  arXiv  Detail & Related papers  (2024-04-23T12:36:24Z)
• GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image [94.56927147492738]
  We introduce GeoWizard, a new generative foundation model designed for estimating geometric attributes from single images. We show that leveraging diffusion priors can markedly improve generalization, detail preservation, and efficiency in resource usage. We propose a simple yet effective strategy to segregate the complex data distribution of various scenes into distinct sub-distributions.
  arXiv  Detail & Related papers  (2024-03-18T17:50:41Z)
• Learning Robust Generalizable Radiance Field with Visibility and Feature Augmented Point Representation [7.203073346844801]
  This paper introduces a novel paradigm for the generalizable neural radiance field (NeRF) We propose the first paradigm that constructs the generalizable neural field based on point-based rather than image-based rendering. Our approach explicitly models visibilities by geometric priors and augments them with neural features.
  arXiv  Detail & Related papers  (2024-01-25T17:58:51Z)
• MuRF: Multi-Baseline Radiance Fields [117.55811938988256]
  We present Multi-Baseline Radiance Fields (MuRF), a feed-forward approach to solving sparse view synthesis. MuRF achieves state-of-the-art performance across multiple different baseline settings. We also show promising zero-shot generalization abilities on the Mip-NeRF 360 dataset.
  arXiv  Detail & Related papers  (2023-12-07T18:59:56Z)
• Spherical Vision Transformer for 360-degree Video Saliency Prediction [17.948179628551376]
  We propose a vision-transformer-based model for omnidirectional videos named SalViT360. We introduce a spherical geometry-aware self-attention mechanism that is capable of effective omnidirectional video understanding. Our approach is the first to employ tangent images for omnidirectional saliency prediction prediction, and our experimental results on three ODV saliency datasets demonstrate its effectiveness compared to the state-of-the-art.
  arXiv  Detail & Related papers  (2023-08-24T18:07:37Z)
• MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer [52.0699787446221]
  We propose MonoViT, a framework combining the global reasoning enabled by ViT models with the flexibility of self-supervised monocular depth estimation. By combining plain convolutions with Transformer blocks, our model can reason locally and globally, yielding depth prediction at a higher level of detail and accuracy.
  arXiv  Detail & Related papers  (2022-08-06T16:54:45Z)
• MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction [72.05649682685197]
  State-of-the-art neural implicit methods allow for high-quality reconstructions of simple scenes from many input views. This is caused primarily by the inherent ambiguity in the RGB reconstruction loss that does not provide enough constraints. Motivated by recent advances in the area of monocular geometry prediction, we explore the utility these cues provide for improving neural implicit surface reconstruction.
  arXiv  Detail & Related papers  (2022-06-01T17:58:15Z)
• DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimation [50.08080424613603]
  Long-range correlation is essential for accurate monocular depth estimation. We propose to leverage the Transformer to model this global context with an effective attention mechanism. Our proposed model, termed DepthFormer, surpasses state-of-the-art monocular depth estimation methods with prominent margins.
  arXiv  Detail & Related papers  (2022-03-27T05:03:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.