End-to-End Adaptive Monte Carlo Denoising and Super-Resolution
- URL: http://arxiv.org/abs/2108.06915v1
- Date: Mon, 16 Aug 2021 06:32:23 GMT
- Title: End-to-End Adaptive Monte Carlo Denoising and Super-Resolution
- Authors: Xinyue Wei, Haozhi Huang, Yujin Shi, Hongliang Yuan, Li Shen, Jue Wang
- Abstract summary: We show that Monte Carlo path tracing can be further accelerated by joint super-resolution and denoising in post-processing.
This new type of joint filtering allows only a low-resolution and fewer-sample (thus noisy) image to be rendered by path tracing.
The main contribution of this work is a new end-to-end network architecture, specifically designed for the SRD task.
- Score: 11.928318461220355
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The classic Monte Carlo path tracing can achieve high quality rendering at
the cost of heavy computation. Recent works make use of deep neural networks to
accelerate this process, by improving either low-resolution or fewer-sample
rendering with super-resolution or denoising neural networks in
post-processing. However, denoising and super-resolution have only been
considered separately in previous work. We show in this work that Monte Carlo
path tracing can be further accelerated by joint super-resolution and denoising
(SRD) in post-processing. This new type of joint filtering allows only a
low-resolution and fewer-sample (thus noisy) image to be rendered by path
tracing, which is then fed into a deep neural network to produce a
high-resolution and clean image. The main contribution of this work is a new
end-to-end network architecture, specifically designed for the SRD task. It
contains two cascaded stages with shared components. We discover that denoising
and super-resolution require very different receptive fields, a key insight
that leads to the introduction of deformable convolution into the network
design. Extensive experiments show that the proposed method outperforms
previous methods and their variants adopted for the SRD task.
Related papers
- Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures.
This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead.
We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
arXiv Detail & Related papers (2023-03-16T21:06:13Z) - RDRN: Recursively Defined Residual Network for Image Super-Resolution [58.64907136562178]
Deep convolutional neural networks (CNNs) have obtained remarkable performance in single image super-resolution.
We propose a novel network architecture which utilizes attention blocks efficiently.
arXiv Detail & Related papers (2022-11-17T11:06:29Z) - ShuffleMixer: An Efficient ConvNet for Image Super-Resolution [88.86376017828773]
We propose ShuffleMixer, for lightweight image super-resolution that explores large convolution and channel split-shuffle operation.
Specifically, we develop a large depth-wise convolution and two projection layers based on channel splitting and shuffling as the basic component to mix features efficiently.
Experimental results demonstrate that the proposed ShuffleMixer is about 6x smaller than the state-of-the-art methods in terms of model parameters and FLOPs.
arXiv Detail & Related papers (2022-05-30T15:26:52Z) - DeepRLS: A Recurrent Network Architecture with Least Squares Implicit
Layers for Non-blind Image Deconvolution [15.986942312624]
We study the problem of non-blind image deconvolution.
We propose a novel recurrent network architecture that leads to very competitive restoration results of high image quality.
arXiv Detail & Related papers (2021-12-10T13:16:51Z) - NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor
Multi-view Stereo [97.07453889070574]
We present a new multi-view depth estimation method that utilizes both conventional SfM reconstruction and learning-based priors.
We show that our proposed framework significantly outperforms state-of-the-art methods on indoor scenes.
arXiv Detail & Related papers (2021-09-02T17:54:31Z) - Learning Frequency-aware Dynamic Network for Efficient Super-Resolution [56.98668484450857]
This paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain.
In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden.
Experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures.
arXiv Detail & Related papers (2021-03-15T12:54:26Z) - A Deep-Unfolded Reference-Based RPCA Network For Video
Foreground-Background Separation [86.35434065681925]
This paper proposes a new deep-unfolding-based network design for the problem of Robust Principal Component Analysis (RPCA)
Unlike existing designs, our approach focuses on modeling the temporal correlation between the sparse representations of consecutive video frames.
Experimentation using the moving MNIST dataset shows that the proposed network outperforms a recently proposed state-of-the-art RPCA network in the task of video foreground-background separation.
arXiv Detail & Related papers (2020-10-02T11:40:09Z) - Deep Iterative Residual Convolutional Network for Single Image
Super-Resolution [31.934084942626257]
We propose a deep Iterative Super-Resolution Residual Convolutional Network (ISRResCNet)
It exploits the powerful image regularization and large-scale optimization techniques by training the deep network in an iterative manner with a residual learning approach.
Our method with a few trainable parameters improves the results for different scaling factors in comparison with the state-of-art methods.
arXiv Detail & Related papers (2020-09-07T12:54:14Z) - OverNet: Lightweight Multi-Scale Super-Resolution with Overscaling
Network [3.6683231417848283]
We introduce OverNet, a deep but lightweight convolutional network to solve SISR at arbitrary scale factors with a single model.
We show that our network outperforms previous state-of-the-art results in standard benchmarks while using fewer parameters than previous approaches.
arXiv Detail & Related papers (2020-08-05T22:10:29Z) - AMP-Net: Denoising based Deep Unfolding for Compressive Image Sensing [44.66684590557732]
We propose a deep unfolding model dubbed AMP-Net to solve the visual image CS problem.
AMP-Net has better reconstruction accuracy than other state-of-the-art methods with high reconstruction speed and a small number of network parameters.
arXiv Detail & Related papers (2020-04-21T15:04:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.