DSR-Diff: Depth Map Super-Resolution with Diffusion Model
- URL: http://arxiv.org/abs/2311.09919v1
- Date: Thu, 16 Nov 2023 14:18:10 GMT
- Title: DSR-Diff: Depth Map Super-Resolution with Diffusion Model
- Authors: Yuan Shi, Bin Xia, Rui Zhu, Qingmin Liao, and Wenming Yang
- Abstract summary: We present a novel CDSR paradigm that utilizes a diffusion model within the latent space to generate guidance for depth map super-resolution.
Our proposed method has shown superior performance in extensive experiments when compared to state-of-the-art methods.
- Score: 38.68563026759223
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Color-guided depth map super-resolution (CDSR) improve the spatial resolution
of a low-quality depth map with the corresponding high-quality color map,
benefiting various applications such as 3D reconstruction, virtual reality, and
augmented reality. While conventional CDSR methods typically rely on
convolutional neural networks or transformers, diffusion models (DMs) have
demonstrated notable effectiveness in high-level vision tasks. In this work, we
present a novel CDSR paradigm that utilizes a diffusion model within the latent
space to generate guidance for depth map super-resolution. The proposed method
comprises a guidance generation network (GGN), a depth map super-resolution
network (DSRN), and a guidance recovery network (GRN). The GGN is specifically
designed to generate the guidance while managing its compactness. Additionally,
we integrate a simple but effective feature fusion module and a
transformer-style feature extraction module into the DSRN, enabling it to
leverage guided priors in the extraction, fusion, and reconstruction of
multi-model images. Taking into account both accuracy and efficiency, our
proposed method has shown superior performance in extensive experiments when
compared to state-of-the-art methods. Our codes will be made available at
https://github.com/shiyuan7/DSR-Diff.
Related papers
- Cutting-Edge Techniques for Depth Map Super-Resolution [0.0]
To overcome hardware limitations in commercially available depth sensors, depth map super-resolution (DMSR) is a practical and valuable computer vision task.
Joint image filtering for DMSR has been applied using spatially-invariant and spatially-variant convolutional neural network (CNN) approaches.
In this project, we propose a novel joint image filtering DMSR algorithm using a Swin transformer architecture.
arXiv Detail & Related papers (2023-06-27T06:57:08Z) - Learning Continuous Depth Representation via Geometric Spatial
Aggregator [47.1698365486215]
We propose a novel continuous depth representation for depth map super-resolution (DSR)
The heart of this representation is our proposed Geometric Spatial Aggregator (GSA), which exploits a distance field modulated by arbitrarily upsampled target gridding.
We also present a transformer-style backbone named GeoDSR, which possesses a principled way to construct the functional mapping between local coordinates.
arXiv Detail & Related papers (2022-12-07T07:48:23Z) - Guided Depth Super-Resolution by Deep Anisotropic Diffusion [18.445649181582823]
We propose a novel approach which combines guided anisotropic diffusion with a deep convolutional network.
We achieve unprecedented results in three commonly used benchmarks for guided depth super-resolution.
arXiv Detail & Related papers (2022-11-21T15:48:13Z) - Deep Posterior Distribution-based Embedding for Hyperspectral Image
Super-resolution [75.24345439401166]
This paper focuses on how to embed the high-dimensional spatial-spectral information of hyperspectral (HS) images efficiently and effectively.
We formulate HS embedding as an approximation of the posterior distribution of a set of carefully-defined HS embedding events.
Then, we incorporate the proposed feature embedding scheme into a source-consistent super-resolution framework that is physically-interpretable.
Experiments over three common benchmark datasets demonstrate that PDE-Net achieves superior performance over state-of-the-art methods.
arXiv Detail & Related papers (2022-05-30T06:59:01Z) - BridgeNet: A Joint Learning Network of Depth Map Super-Resolution and
Monocular Depth Estimation [60.34562823470874]
We propose a joint learning network of depth map super-resolution (DSR) and monocular depth estimation (MDE) without introducing additional supervision labels.
One is the high-frequency attention bridge (HABdg) designed for the feature encoding process, which learns the high-frequency information of the MDE task to guide the DSR task.
The other is the content guidance bridge (CGBdg) designed for the depth map reconstruction process, which provides the content guidance learned from DSR task for MDE task.
arXiv Detail & Related papers (2021-07-27T01:28:23Z) - High-resolution Depth Maps Imaging via Attention-based Hierarchical
Multi-modal Fusion [84.24973877109181]
We propose a novel attention-based hierarchical multi-modal fusion network for guided DSR.
We show that our approach outperforms state-of-the-art methods in terms of reconstruction accuracy, running speed and memory efficiency.
arXiv Detail & Related papers (2021-04-04T03:28:33Z) - Accurate and Lightweight Image Super-Resolution with Model-Guided Deep
Unfolding Network [63.69237156340457]
We present and advocate an explainable approach toward SISR named model-guided deep unfolding network (MoG-DUN)
MoG-DUN is accurate (producing fewer aliasing artifacts), computationally efficient (with reduced model parameters), and versatile (capable of handling multiple degradations)
The superiority of the proposed MoG-DUN method to existing state-of-theart image methods including RCAN, SRDNF, and SRFBN is substantiated by extensive experiments on several popular datasets and various degradation scenarios.
arXiv Detail & Related papers (2020-09-14T08:23:37Z) - Interpretable Deep Multimodal Image Super-Resolution [23.48305854574444]
Multimodal image super-resolution (SR) is the reconstruction of a high resolution image given a low-resolution observation with the aid of another image modality.
We present a multimodal deep network design that integrates coupled sparse priors and allows the effective fusion of information from another modality into the reconstruction process.
arXiv Detail & Related papers (2020-09-07T14:08:35Z) - Channel Attention based Iterative Residual Learning for Depth Map
Super-Resolution [58.626803922196146]
We argue that DSR models trained on synthetic dataset are restrictive and not effective in dealing with real-world DSR tasks.
We make two contributions in tackling real-world degradation of different depth sensors.
We propose a new framework for real-world DSR, which consists of four modules.
arXiv Detail & Related papers (2020-06-02T09:12:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.