Noise2NoiseFlow: Realistic Camera Noise Modeling without Clean Images
- URL: http://arxiv.org/abs/2206.01103v1
- Date: Thu, 2 Jun 2022 15:31:40 GMT
- Title: Noise2NoiseFlow: Realistic Camera Noise Modeling without Clean Images
- Authors: Ali Maleky, Shayan Kousha, Michael S. Brown, Marcus A. Brubaker
- Abstract summary: This paper proposes a framework for training a noise model and a denoiser simultaneously.
It relies on pairs of noisy images rather than noisy/clean paired image data.
The trained denoiser is shown to significantly improve upon both supervised and weakly supervised baseline denoising approaches.
- Score: 35.29066692454865
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Image noise modeling is a long-standing problem with many applications in
computer vision. Early attempts that propose simple models, such as
signal-independent additive white Gaussian noise or the heteroscedastic
Gaussian noise model (a.k.a., camera noise level function) are not sufficient
to learn the complex behavior of the camera sensor noise. Recently, more
complex learning-based models have been proposed that yield better results in
noise synthesis and downstream tasks, such as denoising. However, their
dependence on supervised data (i.e., paired clean images) is a limiting factor
given the challenges in producing ground-truth images. This paper proposes a
framework for training a noise model and a denoiser simultaneously while
relying only on pairs of noisy images rather than noisy/clean paired image
data. We apply this framework to the training of the Noise Flow architecture.
The noise synthesis and density estimation results show that our framework
outperforms previous signal-processing-based noise models and is on par with
its supervised counterpart. The trained denoiser is also shown to significantly
improve upon both supervised and weakly supervised baseline denoising
approaches. The results indicate that the joint training of a denoiser and a
noise model yields significant improvements in the denoiser.
Related papers
- NoiseTransfer: Image Noise Generation with Contrastive Embeddings [9.322843611215486]
We propose a new generative model that can synthesize noisy images with multiple different noise distributions.
We adopt recent contrastive learning to learn distinguishable latent features of the noise.
Our model can generate new noisy images by transferring the noise characteristics solely from a single reference noisy image.
arXiv Detail & Related papers (2023-01-31T11:09:15Z) - Deep Variation Prior: Joint Image Denoising and Noise Variance
Estimation without Clean Data [2.3061446605472558]
This paper investigates the tasks of image denoising and noise variance estimation in a single, joint learning framework.
We build upon DVP, an unsupervised deep learning framework, that simultaneously learns a denoiser and estimates noise variances.
Our method does not require any clean training images or an external step of noise estimation, and instead, approximates the minimum mean squared error denoisers using only a set of noisy images.
arXiv Detail & Related papers (2022-09-19T17:29:32Z) - Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware
Adversarial Training [50.018580462619425]
We propose a novel framework, namely Pixel-level Noise-aware Generative Adrial Network (PNGAN)
PNGAN employs a pre-trained real denoiser to map the fake and real noisy images into a nearly noise-free solution space.
For better noise fitting, we present an efficient architecture Simple Multi-versa-scale Network (SMNet) as the generator.
arXiv Detail & Related papers (2022-04-06T14:09:02Z) - Estimating Fine-Grained Noise Model via Contrastive Learning [11.626812663592416]
We propose an innovative noise model estimation and noise synthesis pipeline for realistic noisy image generation.
Our model learns a noise estimation model with fine-grained statistical noise model in a contrastive manner.
By calibrating noise models of several sensors, our model can be extended to predict other cameras.
arXiv Detail & Related papers (2022-04-03T02:35:01Z) - IDR: Self-Supervised Image Denoising via Iterative Data Refinement [66.5510583957863]
We present a practical unsupervised image denoising method to achieve state-of-the-art denoising performance.
Our method only requires single noisy images and a noise model, which is easily accessible in practical raw image denoising.
To evaluate raw image denoising performance in real-world applications, we build a high-quality raw image dataset SenseNoise-500 that contains 500 real-life scenes.
arXiv Detail & Related papers (2021-11-29T07:22:53Z) - Rethinking Noise Synthesis and Modeling in Raw Denoising [75.55136662685341]
We introduce a new perspective to synthesize noise by directly sampling from the sensor's real noise.
It inherently generates accurate raw image noise for different camera sensors.
arXiv Detail & Related papers (2021-10-10T10:45:24Z) - Physics-based Noise Modeling for Extreme Low-light Photography [63.65570751728917]
We study the noise statistics in the imaging pipeline of CMOS photosensors.
We formulate a comprehensive noise model that can accurately characterize the real noise structures.
Our noise model can be used to synthesize realistic training data for learning-based low-light denoising algorithms.
arXiv Detail & Related papers (2021-08-04T16:36:29Z) - Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images [98.82804259905478]
We present Neighbor2Neighbor to train an effective image denoising model with only noisy images.
In detail, input and target used to train a network are images sub-sampled from the same noisy image.
A denoising network is trained on sub-sampled training pairs generated in the first stage, with a proposed regularizer as additional loss for better performance.
arXiv Detail & Related papers (2021-01-08T02:03:25Z) - Adaptive noise imitation for image denoising [58.21456707617451]
We develop a new textbfadaptive noise imitation (ADANI) algorithm that can synthesize noisy data from naturally noisy images.
To produce realistic noise, a noise generator takes unpaired noisy/clean images as input, where the noisy image is a guide for noise generation.
Coupling the noisy data output from ADANI with the corresponding ground-truth, a denoising CNN is then trained in a fully-supervised manner.
arXiv Detail & Related papers (2020-11-30T02:49:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.