Monocular Occupancy Prediction for Scalable Indoor Scenes

• URL: http://arxiv.org/abs/2407.11730v2
• Date: Wed, 17 Jul 2024 02:37:07 GMT
• Title: Monocular Occupancy Prediction for Scalable Indoor Scenes
• Authors: Hongxiao Yu, Yuqi Wang, Yuntao Chen, Zhaoxiang Zhang,
• Abstract summary: We propose a novel method, named ISO, for predicting indoor scene occupancy using monocular images. ISO harnesses the advantages of a pretrained depth model to achieve accurate depth predictions. We also introduce Occ-ScanNet, a large-scale occupancy benchmark for indoor scenes.
• Score: 56.686307396496545
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Camera-based 3D occupancy prediction has recently garnered increasing attention in outdoor driving scenes. However, research in indoor scenes remains relatively unexplored. The core differences in indoor scenes lie in the complexity of scene scale and the variance in object size. In this paper, we propose a novel method, named ISO, for predicting indoor scene occupancy using monocular images. ISO harnesses the advantages of a pretrained depth model to achieve accurate depth predictions. Furthermore, we introduce the Dual Feature Line of Sight Projection (D-FLoSP) module within ISO, which enhances the learning of 3D voxel features. To foster further research in this domain, we introduce Occ-ScanNet, a large-scale occupancy benchmark for indoor scenes. With a dataset size 40 times larger than the NYUv2 dataset, it facilitates future scalable research in indoor scene analysis. Experimental results on both NYUv2 and Occ-ScanNet demonstrate that our method achieves state-of-the-art performance. The dataset and code are made publicly at https://github.com/hongxiaoy/ISO.git.

Related papers

• ROOT: VLM based System for Indoor Scene Understanding and Beyond [83.71252153660078]
  ROOT is a VLM-based system designed to enhance the analysis of indoor scenes. rootname facilitates indoor scene understanding and proves effective in diverse downstream applications, such as 3D scene generation and embodied AI.
  arXiv  Detail & Related papers  (2024-11-24T04:51:24Z)
• Deeper into Self-Supervised Monocular Indoor Depth Estimation [7.30562653023176]
  Self-supervised learning of indoor depth from monocular sequences is quite challenging for researchers. In this work, our proposed method, named IndoorDepth, consists of two innovations. Experiments on the NYUv2 benchmark demonstrate that our IndoorDepth outperforms the previous state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2023-12-03T04:55:32Z)
• InSpaceType: Reconsider Space Type in Indoor Monocular Depth Estimation [22.287982980942235]
  We benchmark 12 methods on InSpaceType and find they severely suffer from performance imbalance concerning space types. We extend our analysis to 4 other datasets, 3 mitigation approaches, and the ability to generalize to unseen space types.
  arXiv  Detail & Related papers  (2023-09-24T00:39:41Z)
• SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving [98.74706005223685]
  3D scene understanding plays a vital role in vision-based autonomous driving. We propose a SurroundOcc method to predict the 3D occupancy with multi-camera images.
  arXiv  Detail & Related papers  (2023-03-16T17:59:08Z)
• SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes [58.89295356901823]
  Self-supervised monocular depth estimation has shown impressive results in static scenes. It relies on the multi-view consistency assumption for training networks, however, that is violated in dynamic object regions. We introduce an external pretrained monocular depth estimation model for generating single-image depth prior. Our model can predict sharp and accurate depth maps, even when training from monocular videos of highly-dynamic scenes.
  arXiv  Detail & Related papers  (2022-11-07T16:17:47Z)
• DevNet: Self-supervised Monocular Depth Learning via Density Volume Construction [51.96971077984869]
  Self-supervised depth learning from monocular images normally relies on the 2D pixel-wise photometric relation between temporally adjacent image frames. This work proposes Density Volume Construction Network (DevNet), a novel self-supervised monocular depth learning framework.
  arXiv  Detail & Related papers  (2022-09-14T00:08:44Z)
• Guiding Monocular Depth Estimation Using Depth-Attention Volume [38.92495189498365]
  We propose guiding depth estimation to favor planar structures that are ubiquitous especially in indoor environments. Experiments on two popular indoor datasets, NYU-Depth-v2 and ScanNet, show that our method achieves state-of-the-art depth estimation results.
  arXiv  Detail & Related papers  (2020-04-06T15:45:52Z)
• Depth Based Semantic Scene Completion with Position Importance Aware Loss [52.06051681324545]
  PALNet is a novel hybrid network for semantic scene completion. It extracts both 2D and 3D features from multi-stages using fine-grained depth information. It is beneficial for recovering key details like the boundaries of objects and the corners of the scene.
  arXiv  Detail & Related papers  (2020-01-29T07:05:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.