Discrete Cosine Transform Network for Guided Depth Map Super-Resolution
- URL: http://arxiv.org/abs/2104.06977v1
- Date: Wed, 14 Apr 2021 17:01:03 GMT
- Title: Discrete Cosine Transform Network for Guided Depth Map Super-Resolution
- Authors: Zixiang Zhao, Jiangshe Zhang, Shuang Xu, Chunxia Zhang, Junmin Liu
- Abstract summary: The goal is to use high-resolution (HR) RGB images to provide extra information on edges and object contours, so that low-resolution depth maps can be upsampled to HR ones.
We propose an advanced Discrete Cosine Transform Network (DCTNet), which is composed of four components.
We show that our method can generate accurate and HR depth maps, surpassing state-of-the-art methods.
- Score: 19.86463937632802
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Guided depth super-resolution (GDSR) is a hot topic in multi-modal image
processing. The goal is to use high-resolution (HR) RGB images to provide extra
information on edges and object contours, so that low-resolution depth maps can
be upsampled to HR ones. To solve the issues of RGB texture over-transferred,
cross-modal feature extraction difficulty and unclear working mechanism of
modules in existing methods, we propose an advanced Discrete Cosine Transform
Network (DCTNet), which is composed of four components. Firstly, the paired
RGB/depth images are input into the semi-coupled feature extraction module. The
shared convolution kernels extract the cross-modal common features, and the
private kernels extract their unique features, respectively. Then the RGB
features are input into the edge attention mechanism to highlight the edges
useful for upsampling. Subsequently, in the Discrete Cosine Transform (DCT)
module, where DCT is employed to solve the optimization problem designed for
image domain GDSR. The solution is then extended to implement the multi-channel
RGB/depth features upsampling, which increases the rationality of DCTNet, and
is more flexible and effective than conventional methods. The final depth
prediction is output by the reconstruction module. Numerous qualitative and
quantitative experiments demonstrate the effectiveness of our method, which can
generate accurate and HR depth maps, surpassing state-of-the-art methods.
Meanwhile, the rationality of modules is also proved by ablation experiments.
Related papers
- DSR-Diff: Depth Map Super-Resolution with Diffusion Model [38.68563026759223]
We present a novel CDSR paradigm that utilizes a diffusion model within the latent space to generate guidance for depth map super-resolution.
Our proposed method has shown superior performance in extensive experiments when compared to state-of-the-art methods.
arXiv Detail & Related papers (2023-11-16T14:18:10Z) - Symmetric Uncertainty-Aware Feature Transmission for Depth
Super-Resolution [52.582632746409665]
We propose a novel Symmetric Uncertainty-aware Feature Transmission (SUFT) for color-guided DSR.
Our method achieves superior performance compared to state-of-the-art methods.
arXiv Detail & Related papers (2023-06-01T06:35:59Z) - Spherical Space Feature Decomposition for Guided Depth Map
Super-Resolution [123.04455334124188]
Guided depth map super-resolution (GDSR) aims to upsample low-resolution (LR) depth maps with additional information involved in high-resolution (HR) RGB images from the same scene.
In this paper, we propose the Spherical Space feature Decomposition Network (SSDNet) to solve the above issues.
Our method can achieve state-of-the-art results on four test datasets, as well as successfully generalize to real-world scenes.
arXiv Detail & Related papers (2023-03-15T21:22:21Z) - Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient
Object Detection [67.33924278729903]
In this work, we propose Dual Swin-Transformer based Mutual Interactive Network.
We adopt Swin-Transformer as the feature extractor for both RGB and depth modality to model the long-range dependencies in visual inputs.
Comprehensive experiments on five standard RGB-D SOD benchmark datasets demonstrate the superiority of the proposed DTMINet method.
arXiv Detail & Related papers (2022-06-07T08:35:41Z) - Cross-modality Discrepant Interaction Network for RGB-D Salient Object
Detection [78.47767202232298]
We propose a novel Cross-modality Discrepant Interaction Network (CDINet) for RGB-D SOD.
Two components are designed to implement the effective cross-modality interaction.
Our network outperforms $15$ state-of-the-art methods both quantitatively and qualitatively.
arXiv Detail & Related papers (2021-08-04T11:24:42Z) - High-resolution Depth Maps Imaging via Attention-based Hierarchical
Multi-modal Fusion [84.24973877109181]
We propose a novel attention-based hierarchical multi-modal fusion network for guided DSR.
We show that our approach outperforms state-of-the-art methods in terms of reconstruction accuracy, running speed and memory efficiency.
arXiv Detail & Related papers (2021-04-04T03:28:33Z) - Bi-directional Cross-Modality Feature Propagation with
Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation.
Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion.
In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
arXiv Detail & Related papers (2020-07-17T18:35:24Z) - Fast Generation of High Fidelity RGB-D Images by Deep-Learning with
Adaptive Convolution [10.085742605397124]
We propose a deep-learning based approach to efficiently generate RGB-D images with completed information in high resolution.
As an end-to-end approach, high fidelity RGB-D images can be generated efficiently at the rate of around 21 frames per second.
arXiv Detail & Related papers (2020-02-12T16:14:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.