Plane2Depth: Hierarchical Adaptive Plane Guidance for Monocular Depth Estimation

• URL: http://arxiv.org/abs/2409.02494v1
• Date: Wed, 4 Sep 2024 07:45:06 GMT
• Title: Plane2Depth: Hierarchical Adaptive Plane Guidance for Monocular Depth Estimation
• Authors: Li Liu, Ruijie Zhu, Jiacheng Deng, Ziyang Song, Wenfei Yang, Tianzhu Zhang,
• Abstract summary: We propose Plane2Depth, which adaptively utilizes plane information to improve depth prediction within a hierarchical framework. In the proposed plane guided depth generator (PGDG), we design a set of plane queries as prototypes to softly model planes in the scene and predict per-pixel plane coefficients. In the proposed adaptive plane query aggregation (APGA) module, we introduce a novel feature interaction approach to improve the aggregation of multi-scale plane features.
• Score: 38.81275292687583
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Monocular depth estimation aims to infer a dense depth map from a single image, which is a fundamental and prevalent task in computer vision. Many previous works have shown impressive depth estimation results through carefully designed network structures, but they usually ignore the planar information and therefore perform poorly in low-texture areas of indoor scenes. In this paper, we propose Plane2Depth, which adaptively utilizes plane information to improve depth prediction within a hierarchical framework. Specifically, in the proposed plane guided depth generator (PGDG), we design a set of plane queries as prototypes to softly model planes in the scene and predict per-pixel plane coefficients. Then the predicted plane coefficients can be converted into metric depth values with the pinhole camera model. In the proposed adaptive plane query aggregation (APGA) module, we introduce a novel feature interaction approach to improve the aggregation of multi-scale plane features in a top-down manner. Extensive experiments show that our method can achieve outstanding performance, especially in low-texture or repetitive areas. Furthermore, under the same backbone network, our method outperforms the state-of-the-art methods on the NYU-Depth-v2 dataset, achieves competitive results with state-of-the-art methods KITTI dataset and can be generalized to unseen scenes effectively.

Related papers

• AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddings [26.845588648999417]
  We tackle the problem of estimating the planar surfaces in a 3D scene from posed images. We propose a method that predicts multi-view consistent plane embeddings that complement geometry when clustering points into planes. We show through extensive evaluation on the ScanNetV2 dataset that our new method outperforms existing approaches.
  arXiv  Detail & Related papers  (2024-06-13T09:49:31Z)
• 360 Layout Estimation via Orthogonal Planes Disentanglement and Multi-view Geometric Consistency Perception [56.84921040837699]
  Existing panoramic layout estimation solutions tend to recover room boundaries from a vertically compressed sequence, yielding imprecise results. We propose an orthogonal plane disentanglement network (termed DOPNet) to distinguish ambiguous semantics. We also present an unsupervised adaptation technique tailored for horizon-depth and ratio representations. Our solution outperforms other SoTA models on both monocular layout estimation and multi-view layout estimation tasks.
  arXiv  Detail & Related papers  (2023-12-26T12:16:03Z)
• Deep Planar Parallax for Monocular Depth Estimation [24.801102342402828]
  In-depth analysis reveals that utilizing flow-pretrain can optimize the network's usage of consecutive frame modeling. We also propose Planar Position Embedding to handle dynamic objects that defy static scene assumptions.
  arXiv  Detail & Related papers  (2023-01-09T06:02:36Z)
• Ground Plane Matters: Picking Up Ground Plane Prior in Monocular 3D Object Detection [92.75961303269548]
  The ground plane prior is a very informative geometry clue in monocular 3D object detection (M3OD) We propose a Ground Plane Enhanced Network (GPENet) which resolves both issues at one go. Our GPENet can outperform other methods and achieve state-of-the-art performance, well demonstrating the effectiveness and the superiority of the proposed approach.
  arXiv  Detail & Related papers  (2022-11-03T02:21:35Z)
• PlaneDepth: Self-supervised Depth Estimation via Orthogonal Planes [41.517947010531074]
  Multiple near frontal-parallel planes based depth estimation demonstrated impressive results in self-supervised monocular depth estimation (MDE) We propose the PlaneDepth, a novel planes based presentation, including vertical planes and ground planes. Our method can extract the ground plane in an unsupervised manner, which is important for autonomous driving.
  arXiv  Detail & Related papers  (2022-10-04T13:51:59Z)
• Joint Prediction of Monocular Depth and Structure using Planar and Parallax Geometry [4.620624344434533]
  Supervised learning depth estimation methods can achieve good performance when trained on high-quality ground-truth, like LiDAR data. We propose a novel approach combining structure information from a promising Plane and Parallax geometry pipeline with depth information into a U-Net supervised learning network. Our model has impressive performance on depth prediction of thin objects and edges, and compared to structure prediction baseline, our model performs more robustly.
  arXiv  Detail & Related papers  (2022-07-13T17:04:05Z)
• P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior [133.76192155312182]
  We propose a method that learns to selectively leverage information from coplanar pixels to improve the predicted depth. An extensive evaluation of our method shows that we set the new state of the art in supervised monocular depth estimation.
  arXiv  Detail & Related papers  (2022-04-05T10:03:52Z)
• 3DVNet: Multi-View Depth Prediction and Volumetric Refinement [68.68537312256144]
  3DVNet is a novel multi-view stereo (MVS) depth-prediction method. Our key idea is the use of a 3D scene-modeling network that iteratively updates a set of coarse depth predictions. We show that our method exceeds state-of-the-art accuracy in both depth prediction and 3D reconstruction metrics.
  arXiv  Detail & Related papers  (2021-12-01T00:52:42Z)
• PlaneRecNet: Multi-Task Learning with Cross-Task Consistency for Piece-Wise Plane Detection and Reconstruction from a Single RGB Image [11.215334675788952]
  Piece-wise 3D planar reconstruction provides holistic scene understanding of man-made environments, especially for indoor scenarios. Most recent approaches focused on improving the segmentation and reconstruction results by introducing advanced network architectures. We start from enforcing cross-task consistency for our multi-task convolutional neural network, PlaneRecNet. We introduce several novel loss functions (geometric constraint) that jointly improve the accuracy of piece-wise planar segmentation and depth estimation.
  arXiv  Detail & Related papers  (2021-10-21T15:54:03Z)
• Depth Completion using Plane-Residual Representation [84.63079529738924]
  We introduce a novel way of interpreting depth information with the closest depth plane label $p$ and a residual value $r$, as we call it, Plane-Residual (PR) representation. By interpreting depth information in PR representation and using our corresponding depth completion network, we were able to acquire improved depth completion performance with faster computation.
  arXiv  Detail & Related papers  (2021-04-15T10:17:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.