Joint Implicit Image Function for Guided Depth Super-Resolution
- URL: http://arxiv.org/abs/2107.08717v1
- Date: Mon, 19 Jul 2021 09:42:18 GMT
- Title: Joint Implicit Image Function for Guided Depth Super-Resolution
- Authors: Jiaxiang Tang, Xiaokang Chen, Gang Zeng
- Abstract summary: Guided depth super-resolution is a practical task where a low-resolution and noisy input depth map is restored to a high-resolution version.
We take the form of a general image but use a novel Joint Implicit Image Function representation to learn both the weights and values.
We demonstrate the effectiveness of our JIIF representation on guided depth super-resolution task, significantly outperforming state-of-the-art methods on three public benchmarks.
- Score: 11.325235139023931
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Guided depth super-resolution is a practical task where a low-resolution and
noisy input depth map is restored to a high-resolution version, with the help
of a high-resolution RGB guide image. Existing methods usually view this task
as a generalized guided filtering problem that relies on designing explicit
filters and objective functions, or a dense regression problem that directly
predicts the target image via deep neural networks. These methods suffer from
either model capability or interpretability. Inspired by the recent progress in
implicit neural representation, we propose to formulate the guided
super-resolution as a neural implicit image interpolation problem, where we
take the form of a general image interpolation but use a novel Joint Implicit
Image Function (JIIF) representation to learn both the interpolation weights
and values. JIIF represents the target image domain with spatially distributed
local latent codes extracted from the input image and the guide image, and uses
a graph attention mechanism to learn the interpolation weights at the same time
in one unified deep implicit function. We demonstrate the effectiveness of our
JIIF representation on guided depth super-resolution task, significantly
outperforming state-of-the-art methods on three public benchmarks. Code can be
found at \url{https://git.io/JC2sU}.
Related papers
- Solving General Noisy Inverse Problem via Posterior Sampling: A Policy Gradient Viewpoint [21.22750301965104]
We leverage a pretrained diffusion generative model to solve a wide range of image inverse tasks without task specific model fine-tuning.
To precisely estimate the guidance score function of the input image, we propose Diffusion Policy Gradient (DPG)
Experiments show that our method is robust to both Gaussian and Poisson noise degradation on multiple linear and non-linear inverse tasks.
arXiv Detail & Related papers (2024-03-15T16:38:47Z) - Guided Depth Super-Resolution by Deep Anisotropic Diffusion [18.445649181582823]
We propose a novel approach which combines guided anisotropic diffusion with a deep convolutional network.
We achieve unprecedented results in three commonly used benchmarks for guided depth super-resolution.
arXiv Detail & Related papers (2022-11-21T15:48:13Z) - Deep Generalized Unfolding Networks for Image Restoration [16.943609020362395]
We propose a Deep Generalized Unfolding Network (DGUNet) for image restoration.
We integrate a gradient estimation strategy into the gradient descent step of the Proximal Gradient Descent (PGD) algorithm.
Our method is superior in terms of state-of-the-art performance, interpretability, and generalizability.
arXiv Detail & Related papers (2022-04-28T08:39:39Z) - Learning Enriched Features for Fast Image Restoration and Enhancement [166.17296369600774]
This paper presents a holistic goal of maintaining spatially-precise high-resolution representations through the entire network.
We learn an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
Our approach achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement.
arXiv Detail & Related papers (2022-04-19T17:59:45Z) - Memory-augmented Deep Unfolding Network for Guided Image
Super-resolution [67.83489239124557]
Guided image super-resolution (GISR) aims to obtain a high-resolution (HR) target image by enhancing the spatial resolution of a low-resolution (LR) target image under the guidance of a HR image.
Previous model-based methods mainly takes the entire image as a whole, and assume the prior distribution between the HR target image and the HR guidance image.
We propose a maximal a posterior (MAP) estimation model for GISR with two types of prior on the HR target image.
arXiv Detail & Related papers (2022-02-12T15:37:13Z) - High-resolution Depth Maps Imaging via Attention-based Hierarchical
Multi-modal Fusion [84.24973877109181]
We propose a novel attention-based hierarchical multi-modal fusion network for guided DSR.
We show that our approach outperforms state-of-the-art methods in terms of reconstruction accuracy, running speed and memory efficiency.
arXiv Detail & Related papers (2021-04-04T03:28:33Z) - Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
Blind image restoration is a common yet challenging problem in computer vision.
We propose a novel blind image restoration method, aiming to integrate both the advantages of them.
Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
arXiv Detail & Related papers (2020-08-25T03:30:53Z) - Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task.
We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network.
Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
arXiv Detail & Related papers (2020-03-15T11:04:30Z) - PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of
Generative Models [77.32079593577821]
PULSE (Photo Upsampling via Latent Space Exploration) generates high-resolution, realistic images at resolutions previously unseen in the literature.
Our method outperforms state-of-the-art methods in perceptual quality at higher resolutions and scale factors than previously possible.
arXiv Detail & Related papers (2020-03-08T16:44:31Z) - Multimodal Deep Unfolding for Guided Image Super-Resolution [23.48305854574444]
Deep learning methods rely on training data to learn an end-to-end mapping from a low-resolution input to a high-resolution output.
We propose a multimodal deep learning design that incorporates sparse priors and allows the effective integration of information from another image modality into the network architecture.
Our solution relies on a novel deep unfolding operator, performing steps similar to an iterative algorithm for convolutional sparse coding with side information.
arXiv Detail & Related papers (2020-01-21T14:41:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.