Indoor Panorama Planar 3D Reconstruction via Divide and Conquer

• URL: http://arxiv.org/abs/2106.14166v1
• Date: Sun, 27 Jun 2021 07:58:29 GMT
• Title: Indoor Panorama Planar 3D Reconstruction via Divide and Conquer
• Authors: Cheng Sun, Chi-Wei Hsiao, Ning-Hsu Wang, Min Sun, Hwann-Tzong Chen
• Abstract summary: Indoor panorama typically consists of human-made structures parallel or perpendicular to gravity. We leverage this phenomenon to approximate the scene in a 360-degree image with (H)orizontal-planes and (V)ertical-planes. We create a benchmark for indoor panorama planar reconstruction by extending existing 360 depth datasets with ground truth H&V-planes.
• Score: 36.466149592254965
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Indoor panorama typically consists of human-made structures parallel or perpendicular to gravity. We leverage this phenomenon to approximate the scene in a 360-degree image with (H)orizontal-planes and (V)ertical-planes. To this end, we propose an effective divide-and-conquer strategy that divides pixels based on their plane orientation estimation; then, the succeeding instance segmentation module conquers the task of planes clustering more easily in each plane orientation group. Besides, parameters of V-planes depend on camera yaw rotation, but translation-invariant CNNs are less aware of the yaw change. We thus propose a yaw-invariant V-planar reparameterization for CNNs to learn. We create a benchmark for indoor panorama planar reconstruction by extending existing 360 depth datasets with ground truth H\&V-planes (referred to as PanoH&V dataset) and adopt state-of-the-art planar reconstruction methods to predict H\&V-planes as our baselines. Our method outperforms the baselines by a large margin on the proposed dataset.

Related papers

• MonoPlane: Exploiting Monocular Geometric Cues for Generalizable 3D Plane Reconstruction [37.481945507799594]
  This paper presents a generalizable 3D plane detection and reconstruction framework named MonoPlane. We first leverage large-scale pre-trained neural networks to obtain the depth and surface normals from a single image. These monocular geometric cues are then incorporated into a proximity-guided RANSAC framework to sequentially fit each plane instance.
  arXiv  Detail & Related papers  (2024-11-02T12:15:29Z)
• AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddings [26.845588648999417]
  We tackle the problem of estimating the planar surfaces in a 3D scene from posed images. We propose a method that predicts multi-view consistent plane embeddings that complement geometry when clustering points into planes. We show through extensive evaluation on the ScanNetV2 dataset that our new method outperforms existing approaches.
  arXiv  Detail & Related papers  (2024-06-13T09:49:31Z)
• 360 Layout Estimation via Orthogonal Planes Disentanglement and Multi-view Geometric Consistency Perception [56.84921040837699]
  Existing panoramic layout estimation solutions tend to recover room boundaries from a vertically compressed sequence, yielding imprecise results. We propose an orthogonal plane disentanglement network (termed DOPNet) to distinguish ambiguous semantics. We also present an unsupervised adaptation technique tailored for horizon-depth and ratio representations. Our solution outperforms other SoTA models on both monocular layout estimation and multi-view layout estimation tasks.
  arXiv  Detail & Related papers  (2023-12-26T12:16:03Z)
• Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction [84.94140661523956]
  We propose a tri-perspective view (TPV) representation which accompanies BEV with two additional perpendicular planes. We model each point in the 3D space by summing its projected features on the three planes. Experiments show that our model trained with sparse supervision effectively predicts the semantic occupancy for all voxels.
  arXiv  Detail & Related papers  (2023-02-15T17:58:10Z)
• Ground Plane Matters: Picking Up Ground Plane Prior in Monocular 3D Object Detection [92.75961303269548]
  The ground plane prior is a very informative geometry clue in monocular 3D object detection (M3OD) We propose a Ground Plane Enhanced Network (GPENet) which resolves both issues at one go. Our GPENet can outperform other methods and achieve state-of-the-art performance, well demonstrating the effectiveness and the superiority of the proposed approach.
  arXiv  Detail & Related papers  (2022-11-03T02:21:35Z)
• PlaneDepth: Self-supervised Depth Estimation via Orthogonal Planes [41.517947010531074]
  Multiple near frontal-parallel planes based depth estimation demonstrated impressive results in self-supervised monocular depth estimation (MDE) We propose the PlaneDepth, a novel planes based presentation, including vertical planes and ground planes. Our method can extract the ground plane in an unsupervised manner, which is important for autonomous driving.
  arXiv  Detail & Related papers  (2022-10-04T13:51:59Z)
• PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo [32.81496429134453]
  We present a novel framework named PlaneMVS for 3D plane reconstruction from multiple input views with known camera poses. In contrast, we reconstruct 3D planes with a multi-view-stereo (MVS) pipeline that takes advantage of multi-view geometry. Our method even outperforms a set of SOTA learning-based MVS methods thanks to the learned plane priors.
  arXiv  Detail & Related papers  (2022-03-22T22:35:46Z)
• PlaneTR: Structure-Guided Transformers for 3D Plane Recovery [56.23402171871664]
  PlaneTR simultaneously detects and reconstructs planes from a single image. PlaneTR achieves a state-of-the-art performance on the ScanNet and NYUv2 datasets.
  arXiv  Detail & Related papers  (2021-07-27T23:55:40Z)
• Dynamic Plane Convolutional Occupancy Networks [4.607145155913717]
  We propose Dynamic Plane Convolutional Occupancy Networks to push further the quality of 3D surface reconstruction. A fully-connected network learns to predict plane parameters that best describe the shapes of objects or scenes. Our method shows superior performance in surface reconstruction from unoriented point clouds in ShapeNet as well as an indoor scene dataset.
  arXiv  Detail & Related papers  (2020-11-11T14:24:52Z)
• From Planes to Corners: Multi-Purpose Primitive Detection in Unorganized 3D Point Clouds [59.98665358527686]
  We propose a new method for segmentation-free joint estimation of orthogonal planes. Such unified scene exploration allows for multitudes of applications such as semantic plane detection or local and global scan alignment. Our experiments demonstrate the validity of our approach in numerous scenarios from wall detection to 6D tracking.
  arXiv  Detail & Related papers  (2020-01-21T06:51:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.