How to Best Combine Demosaicing and Denoising?

• URL: http://arxiv.org/abs/2408.06684v1
• Date: Tue, 13 Aug 2024 07:23:53 GMT
• Title: How to Best Combine Demosaicing and Denoising?
• Authors: Yu Guo, Qiyu Jin, Jean-Michel Morel, Gabriele Facciolo,
• Abstract summary: demosaicing and denoising play a critical role in the raw imaging pipeline. Most demosaicing methods address the demosaicing of noise free images. The real problem is to jointly denoise and demosaic noisy raw images.
• Score: 16.921538543268216
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image demosaicing and denoising play a critical role in the raw imaging pipeline. These processes have often been treated as independent, without considering their interactions. Indeed, most classic denoising methods handle noisy RGB images, not raw images. Conversely, most demosaicing methods address the demosaicing of noise free images. The real problem is to jointly denoise and demosaic noisy raw images. But the question of how to proceed is still not yet clarified. In this paper, we carry-out extensive experiments and a mathematical analysis to tackle this problem by low complexity algorithms. Indeed, both problems have been only addressed jointly by end-to-end heavy weight convolutional neural networks (CNNs), which are currently incompatible with low power portable imaging devices and remain by nature domain (or device) dependent. Our study leads us to conclude that, with moderate noise, demosaicing should be applied first, followed by denoising. This requires a simple adaptation of classic denoising algorithms to demosaiced noise, which we justify and specify. Although our main conclusion is ``demosaic first, then denoise'', we also discover that for high noise, there is a moderate PSNR gain by a more complex strategy: partial CFA denoising followed by demosaicing, and by a second denoising on the RGB image. These surprising results are obtained by a black-box optimization of the pipeline, which could be applied to any other pipeline. We validate our results on simulated and real noisy CFA images obtained from several benchmarks.

Related papers

• Enhancing convolutional neural network generalizability via low-rank weight approximation [6.763245393373041]
  Sufficient denoising is often an important first step for image processing. Deep neural networks (DNNs) have been widely used for image denoising. We introduce a new self-supervised framework for image denoising based on the Tucker low-rank tensor approximation.
  arXiv  Detail & Related papers  (2022-09-26T14:11:05Z)
• Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training [50.018580462619425]
  We propose a novel framework, namely Pixel-level Noise-aware Generative Adrial Network (PNGAN) PNGAN employs a pre-trained real denoiser to map the fake and real noisy images into a nearly noise-free solution space. For better noise fitting, we present an efficient architecture Simple Multi-versa-scale Network (SMNet) as the generator.
  arXiv  Detail & Related papers  (2022-04-06T14:09:02Z)
• Zero-shot Blind Image Denoising via Implicit Neural Representations [77.79032012459243]
  We propose an alternative denoising strategy that leverages the architectural inductive bias of implicit neural representations (INRs) We show that our method outperforms existing zero-shot denoising methods under an extensive set of low-noise or real-noise scenarios.
  arXiv  Detail & Related papers  (2022-04-05T12:46:36Z)
• Rethinking Noise Synthesis and Modeling in Raw Denoising [75.55136662685341]
  We introduce a new perspective to synthesize noise by directly sampling from the sensor's real noise. It inherently generates accurate raw image noise for different camera sensors.
  arXiv  Detail & Related papers  (2021-10-10T10:45:24Z)
• Noise2Score: Tweedie's Approach to Self-Supervised Image Denoising without Clean Images [35.41467558264341]
  We present a novel approach, called Noise2Score, which reveals a missing link in order to unite different approaches. Specifically, we show that image denoising problems without clean images can be addressed by finding the mode of the posterior distribution. Our method then uses the recent finding that the score function can be stably estimated from the noisy images using the amortized residual denoising autoencoder.
  arXiv  Detail & Related papers  (2021-06-13T14:41:09Z)
• Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images [98.82804259905478]
  We present Neighbor2Neighbor to train an effective image denoising model with only noisy images. In detail, input and target used to train a network are images sub-sampled from the same noisy image. A denoising network is trained on sub-sampled training pairs generated in the first stage, with a proposed regularizer as additional loss for better performance.
  arXiv  Detail & Related papers  (2021-01-08T02:03:25Z)
• Joint Demosaicking and Denoising Benefits from a Two-stage Training Strategy [28.69029171306052]
  Image demosaicking and denoising are the first two key steps of the color image production pipeline. In this paper, we address this problem by a hybrid machine learning method. Our demosaicking algorithm, trained on noiseless images, combines a traditional method and a residual convolutional neural network (CNN)
  arXiv  Detail & Related papers  (2020-09-14T05:23:58Z)
• A Review of an Old Dilemma: Demosaicking First, or Denoising First? [16.328866317851183]
  Denoising and demosaicking aim at reconstructing a full color image from a noisy color filter array (CFA) image. In this paper, we review the main variants of these strategies and carry-out an extensive evaluation to find the best way to reconstruct full color images from a noisy mosaic.
  arXiv  Detail & Related papers  (2020-04-24T07:32:17Z)
• CycleISP: Real Image Restoration via Improved Data Synthesis [166.17296369600774]
  We present a framework that models camera imaging pipeline in forward and reverse directions. By training a new image denoising network on realistic synthetic data, we achieve the state-of-the-art performance on real camera benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-17T15:20:25Z)
• Handling noise in image deblurring via joint learning [0.3407858371718068]
  Many blind deblurring methods assume blurred images are noise-free and perform unsatisfactorily on the blurry images with noise. We propose a cascaded framework consisting of a denoiser subnetwork and a deblurring subnetwork. Joint learning reduces the effect of the residual noise after denoising on deblurring, hence improves the robustness of deblurring to heavy noise.
  arXiv  Detail & Related papers  (2020-01-27T12:59:52Z)
• Variational Denoising Network: Toward Blind Noise Modeling and Removal [59.36166491196973]
  Blind image denoising is an important yet very challenging problem in computer vision. We propose a new variational inference method, which integrates both noise estimation and image denoising.
  arXiv  Detail & Related papers  (2019-08-29T15:54:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.