Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark Dataset and Baseline

• URL: http://arxiv.org/abs/2104.06174v1
• Date: Tue, 13 Apr 2021 13:27:26 GMT
• Title: Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark Dataset and Baseline
• Authors: Lingzhi He, Hongguang Zhu, Feng Li, Huihui Bai, Runmin Cong, Chunjie Zhang, Chunyu Lin, Meiqin Liu, Yao Zhao
• Abstract summary: We build a large-scale dataset named "RGB-D-D" to promote the study of depth map super-resolution (SR) We provide a fast depth map super-resolution (FDSR) baseline, in which the high-frequency component adaptively decomposed from RGB image to guide the depth map SR. For the real-world LR depth maps, our algorithm can produce more accurate HR depth maps with clearer boundaries and to some extent correct the depth value errors.
• Score: 48.69396457721544
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Depth maps obtained by commercial depth sensors are always in low-resolution, making it difficult to be used in various computer vision tasks. Thus, depth map super-resolution (SR) is a practical and valuable task, which upscales the depth map into high-resolution (HR) space. However, limited by the lack of real-world paired low-resolution (LR) and HR depth maps, most existing methods use downsampling to obtain paired training samples. To this end, we first construct a large-scale dataset named "RGB-D-D", which can greatly promote the study of depth map SR and even more depth-related real-world tasks. The "D-D" in our dataset represents the paired LR and HR depth maps captured from mobile phone and Lucid Helios respectively ranging from indoor scenes to challenging outdoor scenes. Besides, we provide a fast depth map super-resolution (FDSR) baseline, in which the high-frequency component adaptively decomposed from RGB image to guide the depth map SR. Extensive experiments on existing public datasets demonstrate the effectiveness and efficiency of our network compared with the state-of-the-art methods. Moreover, for the real-world LR depth maps, our algorithm can produce more accurate HR depth maps with clearer boundaries and to some extent correct the depth value errors.

Related papers

• RDFC-GAN: RGB-Depth Fusion CycleGAN for Indoor Depth Completion [28.634851863097953]
  We propose a novel two-branch end-to-end fusion network named RDFC-GAN. It takes a pair of RGB and incomplete depth images as input to predict a dense and completed depth map. The first branch employs an encoder-decoder structure, by adhering to the Manhattan world assumption. The other branch applies an RGB-depth fusion CycleGAN, adept at translating RGB imagery into detailed, textured depth maps.
  arXiv  Detail & Related papers  (2023-06-06T11:03:05Z)
• Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution [52.582632746409665]
  We propose a novel Symmetric Uncertainty-aware Feature Transmission (SUFT) for color-guided DSR. Our method achieves superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-01T06:35:59Z)
• RGB-Depth Fusion GAN for Indoor Depth Completion [29.938869342958125]
  In this paper, we design a novel two-branch end-to-end fusion network, which takes a pair of RGB and incomplete depth images as input to predict a dense and completed depth map. In one branch, we propose an RGB-depth fusion GAN to transfer the RGB image to the fine-grained textured depth map. In the other branch, we adopt adaptive fusion modules named W-AdaIN to propagate the features across the two branches.
  arXiv  Detail & Related papers  (2022-03-21T10:26:38Z)
• Joint Learning of Salient Object Detection, Depth Estimation and Contour Extraction [91.43066633305662]
  We propose a novel multi-task and multi-modal filtered transformer (MMFT) network for RGB-D salient object detection (SOD) Specifically, we unify three complementary tasks: depth estimation, salient object detection and contour estimation. The multi-task mechanism promotes the model to learn the task-aware features from the auxiliary tasks. Experiments show that it not only significantly surpasses the depth-based RGB-D SOD methods on multiple datasets, but also precisely predicts a high-quality depth map and salient contour at the same time.
  arXiv  Detail & Related papers  (2022-03-09T17:20:18Z)
• Weakly-Supervised Monocular Depth Estimationwith Resolution-Mismatched Data [73.9872931307401]
  We propose a novel weakly-supervised framework to train a monocular depth estimation network. The proposed framework is composed of a sharing weight monocular depth estimation network and a depth reconstruction network for distillation. Experimental results demonstrate that our method achieves superior performance than unsupervised and semi-supervised learning based schemes.
  arXiv  Detail & Related papers  (2021-09-23T18:04:12Z)
• High-resolution Depth Maps Imaging via Attention-based Hierarchical Multi-modal Fusion [84.24973877109181]
  We propose a novel attention-based hierarchical multi-modal fusion network for guided DSR. We show that our approach outperforms state-of-the-art methods in terms of reconstruction accuracy, running speed and memory efficiency.
  arXiv  Detail & Related papers  (2021-04-04T03:28:33Z)
• Multi-Scale Progressive Fusion Learning for Depth Map Super-Resolution [11.072332820377612]
  The resolution of depth map collected by depth camera is often lower than that of its associated RGB camera. A major problem with depth map super-resolution is that there will be obvious jagged edges and excessive loss of details. We propose a multi-scale progressive fusion network for depth map SR, which possess an structure to integrate hierarchical features in different domains.
  arXiv  Detail & Related papers  (2020-11-24T03:03:07Z)
• Channel Attention based Iterative Residual Learning for Depth Map Super-Resolution [58.626803922196146]
  We argue that DSR models trained on synthetic dataset are restrictive and not effective in dealing with real-world DSR tasks. We make two contributions in tackling real-world degradation of different depth sensors. We propose a new framework for real-world DSR, which consists of four modules.
  arXiv  Detail & Related papers  (2020-06-02T09:12:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.