Joint Demosaicking and Denoising Benefits from a Two-stage Training Strategy

• URL: http://arxiv.org/abs/2009.06205v3
• Date: Wed, 19 Jul 2023 06:05:27 GMT
• Title: Joint Demosaicking and Denoising Benefits from a Two-stage Training Strategy
• Authors: Yu Guo, Qiyu Jin, Gabriele Facciolo, Tieyong Zeng, Jean-Michel Morel
• Abstract summary: Image demosaicking and denoising are the first two key steps of the color image production pipeline. In this paper, we address this problem by a hybrid machine learning method. Our demosaicking algorithm, trained on noiseless images, combines a traditional method and a residual convolutional neural network (CNN)
• Score: 28.69029171306052
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image demosaicking and denoising are the first two key steps of the color image production pipeline. The classical processing sequence has for a long time consisted of applying denoising first, and then demosaicking. Applying the operations in this order leads to oversmoothing and checkerboard effects. Yet, it was difficult to change this order, because once the image is demosaicked, the statistical properties of the noise are dramatically changed and hard to handle by traditional denoising models. In this paper, we address this problem by a hybrid machine learning method. We invert the traditional color filter array (CFA) processing pipeline by first demosaicking and then denoising. Our demosaicking algorithm, trained on noiseless images, combines a traditional method and a residual convolutional neural network (CNN). This first stage retains all known information, which is the key point to obtain faithful final results. The noisy demosaicked image is then passed through a second CNN restoring a noiseless full-color image. This pipeline order completely avoids checkerboard effects and restores fine image detail. Although CNNs can be trained to solve jointly demosaicking-denoising end-to-end, we find that this two-stage training performs better and is less prone to failure. It is shown experimentally to improve on the state of the art, both quantitatively and in terms of visual quality.

Related papers

• Combining Pre- and Post-Demosaicking Noise Removal for RAW Video [2.772895608190934]
  Denoising is one of the fundamental steps of the processing pipeline that converts data captured by a camera sensor into a display-ready image or video. We propose a self-similarity-based denoising scheme that weights both a pre- and a post-demosaicking denoiser for Bayer-patterned CFA video data. We show that a balance between the two leads to better image quality, and we empirically find that higher noise levels benefit from a higher influence pre-demosaicking.
  arXiv  Detail & Related papers  (2024-10-03T15:20:19Z)
• How to Best Combine Demosaicing and Denoising? [16.921538543268216]
  demosaicing and denoising play a critical role in the raw imaging pipeline. Most demosaicing methods address the demosaicing of noise free images. The real problem is to jointly denoise and demosaic noisy raw images.
  arXiv  Detail & Related papers  (2024-08-13T07:23:53Z)
• Self-Calibrated Variance-Stabilizing Transformations for Real-World Image Denoising [19.08732222562782]
  Supervised deep learning has become the method of choice for image denoising. We show that, contrary to popular belief, denoising networks specialized in the removal of Gaussian noise can be efficiently leveraged in favor of real-world image denoising.
  arXiv  Detail & Related papers  (2024-07-24T16:23:46Z)
• Masked Image Training for Generalizable Deep Image Denoising [53.03126421917465]
  We present a novel approach to enhance the generalization performance of denoising networks. Our method involves masking random pixels of the input image and reconstructing the missing information during training. Our approach exhibits better generalization ability than other deep learning models and is directly applicable to real-world scenarios.
  arXiv  Detail & Related papers  (2023-03-23T09:33:44Z)
• NBD-GAP: Non-Blind Image Deblurring Without Clean Target Images [79.33220095067749]
  Large amounts of blurry-clean image pairs are required for training to achieve good performance. Deep networks often fail to perform well when the blurry images and the blur kernels during testing are very different from the ones used during training.
  arXiv  Detail & Related papers  (2022-09-20T06:21:11Z)
• Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images [98.82804259905478]
  We present Neighbor2Neighbor to train an effective image denoising model with only noisy images. In detail, input and target used to train a network are images sub-sampled from the same noisy image. A denoising network is trained on sub-sampled training pairs generated in the first stage, with a proposed regularizer as additional loss for better performance.
  arXiv  Detail & Related papers  (2021-01-08T02:03:25Z)
• Unpaired Learning of Deep Image Denoising [80.34135728841382]
  This paper presents a two-stage scheme by incorporating self-supervised learning and knowledge distillation. For self-supervised learning, we suggest a dilated blind-spot network (D-BSN) to learn denoising solely from real noisy images. Experiments show that our unpaired learning method performs favorably on both synthetic noisy images and real-world noisy photographs.
  arXiv  Detail & Related papers  (2020-08-31T16:22:40Z)
• Improving Blind Spot Denoising for Microscopy [73.94017852757413]
  We present a novel way to improve the quality of self-supervised denoising. We assume the clean image to be the result of a convolution with a point spread function (PSF) and explicitly include this operation at the end of our neural network.
  arXiv  Detail & Related papers  (2020-08-19T13:06:24Z)
• Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation [52.75909685172843]
  Real-world image noise removal is a long-standing yet very challenging task in computer vision. We propose a novel unified framework to deal with the noise removal and noise generation tasks. Our method learns the joint distribution of the clean-noisy image pairs.
  arXiv  Detail & Related papers  (2020-07-12T09:16:06Z)
• Noise2Inpaint: Learning Referenceless Denoising by Inpainting Unrolling [2.578242050187029]
  We introduce Noise2Inpaint (N2I), a training approach that recasts the denoising problem into a regularized image inpainting framework. N2I performs successful denoising on real-world datasets, while better preserving details compared to its purely data-driven counterpart Noise2Self.
  arXiv  Detail & Related papers  (2020-06-16T18:46:42Z)
• A Review of an Old Dilemma: Demosaicking First, or Denoising First? [16.328866317851183]
  Denoising and demosaicking aim at reconstructing a full color image from a noisy color filter array (CFA) image. In this paper, we review the main variants of these strategies and carry-out an extensive evaluation to find the best way to reconstruct full color images from a noisy mosaic.
  arXiv  Detail & Related papers  (2020-04-24T07:32:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.