GAM-Depth: Self-Supervised Indoor Depth Estimation Leveraging a Gradient-Aware Mask and Semantic Constraints

• URL: http://arxiv.org/abs/2402.14354v1
• Date: Thu, 22 Feb 2024 07:53:34 GMT
• Title: GAM-Depth: Self-Supervised Indoor Depth Estimation Leveraging a Gradient-Aware Mask and Semantic Constraints
• Authors: Anqi Cheng, Zhiyuan Yang, Haiyue Zhu, Kezhi Mao
• Abstract summary: We propose GAM-Depth, developed upon two novel components: gradient-aware mask and semantic constraints. The gradient-aware mask enables adaptive and robust supervision for both key areas and textureless regions. The incorporation of semantic constraints for indoor self-supervised depth estimation improves depth discrepancies at object boundaries.
• Score: 12.426365333096264
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised depth estimation has evolved into an image reconstruction task that minimizes a photometric loss. While recent methods have made strides in indoor depth estimation, they often produce inconsistent depth estimation in textureless areas and unsatisfactory depth discrepancies at object boundaries. To address these issues, in this work, we propose GAM-Depth, developed upon two novel components: gradient-aware mask and semantic constraints. The gradient-aware mask enables adaptive and robust supervision for both key areas and textureless regions by allocating weights based on gradient magnitudes.The incorporation of semantic constraints for indoor self-supervised depth estimation improves depth discrepancies at object boundaries, leveraging a co-optimization network and proxy semantic labels derived from a pretrained segmentation model. Experimental studies on three indoor datasets, including NYUv2, ScanNet, and InteriorNet, show that GAM-Depth outperforms existing methods and achieves state-of-the-art performance, signifying a meaningful step forward in indoor depth estimation. Our code will be available at https://github.com/AnqiCheng1234/GAM-Depth.

Related papers

• D$^3$epth: Self-Supervised Depth Estimation with Dynamic Mask in Dynamic Scenes [23.731667977542454]
  D$3$epth is a novel method for self-supervised depth estimation in dynamic scenes. It tackles the challenge of dynamic objects from two key perspectives. It consistently outperforms existing self-supervised monocular depth estimation baselines.
  arXiv  Detail & Related papers  (2024-11-07T16:07:00Z)
• ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation [62.600382533322325]
  We propose a novel monocular depth estimation method called ScaleDepth. Our method decomposes metric depth into scene scale and relative depth, and predicts them through a semantic-aware scale prediction module. Our method achieves metric depth estimation for both indoor and outdoor scenes in a unified framework.
  arXiv  Detail & Related papers  (2024-07-11T05:11:56Z)
• Uncertainty-guided Optimal Transport in Depth Supervised Sparse-View 3D Gaussian [49.21866794516328]
  3D Gaussian splatting has demonstrated impressive performance in real-time novel view synthesis. Previous approaches have incorporated depth supervision into the training of 3D Gaussians to mitigate overfitting. We introduce a novel method to supervise the depth distribution of 3D Gaussians, utilizing depth priors with integrated uncertainty estimates.
  arXiv  Detail & Related papers  (2024-05-30T03:18:30Z)
• Depth-aware Volume Attention for Texture-less Stereo Matching [67.46404479356896]
  We propose a lightweight volume refinement scheme to tackle the texture deterioration in practical outdoor scenarios. We introduce a depth volume supervised by the ground-truth depth map, capturing the relative hierarchy of image texture. Local fine structure and context are emphasized to mitigate ambiguity and redundancy during volume aggregation.
  arXiv  Detail & Related papers  (2024-02-14T04:07:44Z)
• Learning Occlusion-Aware Coarse-to-Fine Depth Map for Self-supervised Monocular Depth Estimation [11.929584800629673]
  We propose a novel network to learn an Occlusion-aware Coarse-to-Fine Depth map for self-supervised monocular depth estimation. The proposed OCFD-Net does not only employ a discrete depth constraint for learning a coarse-level depth map, but also employ a continuous depth constraint for learning a scene depth residual.
  arXiv  Detail & Related papers  (2022-03-21T12:43:42Z)
• Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation [84.34227665232281]
  Domain adaptation for semantic segmentation aims to improve the model performance in the presence of a distribution shift between source and target domain. We leverage the guidance from self-supervised depth estimation, which is available on both domains, to bridge the domain gap. We demonstrate the effectiveness of our proposed approach on the benchmark tasks SYNTHIA-to-Cityscapes and GTA-to-Cityscapes.
  arXiv  Detail & Related papers  (2021-04-28T07:47:36Z)
• Boundary-induced and scene-aggregated network for monocular depth prediction [20.358133522462513]
  We propose the Boundary-induced and Scene-aggregated network (BS-Net) to predict the dense depth of a single RGB image. Several experimental results on the NYUD v2 dataset and xffthe iBims-1 dataset illustrate the state-of-the-art performance of the proposed approach.
  arXiv  Detail & Related papers  (2021-02-26T01:43:17Z)
• SelfDeco: Self-Supervised Monocular Depth Completion in Challenging Indoor Environments [50.761917113239996]
  We present a novel algorithm for self-supervised monocular depth completion. Our approach is based on training a neural network that requires only sparse depth measurements and corresponding monocular video sequences without dense depth labels. Our self-supervised algorithm is designed for challenging indoor environments with textureless regions, glossy and transparent surface, non-Lambertian surfaces, moving people, longer and diverse depth ranges and scenes captured by complex ego-motions.
  arXiv  Detail & Related papers  (2020-11-10T08:55:07Z)
• Occlusion-Aware Depth Estimation with Adaptive Normal Constraints [85.44842683936471]
  We present a new learning-based method for multi-frame depth estimation from a color video. Our method outperforms the state-of-the-art in terms of depth estimation accuracy.
  arXiv  Detail & Related papers  (2020-04-02T07:10:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.