Self-supervised Image Denoising with Downsampled Invariance Loss and Conditional Blind-Spot Network

• URL: http://arxiv.org/abs/2304.09507v2
• Date: Fri, 28 Jul 2023 06:06:28 GMT
• Title: Self-supervised Image Denoising with Downsampled Invariance Loss and Conditional Blind-Spot Network
• Authors: Yeong Il Jang, Keuntek Lee, Gu Yong Park, Seyun Kim, Nam Ik Cho
• Abstract summary: Most representative self-supervised denoisers are based on blind-spot networks. A standard blind-spot network fails to reduce real camera noise due to the pixel-wise correlation of noise. We propose a novel self-supervised training framework that can remove real noise.
• Score: 12.478287906337194
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There have been many image denoisers using deep neural networks, which outperform conventional model-based methods by large margins. Recently, self-supervised methods have attracted attention because constructing a large real noise dataset for supervised training is an enormous burden. The most representative self-supervised denoisers are based on blind-spot networks, which exclude the receptive field's center pixel. However, excluding any input pixel is abandoning some information, especially when the input pixel at the corresponding output position is excluded. In addition, a standard blind-spot network fails to reduce real camera noise due to the pixel-wise correlation of noise, though it successfully removes independently distributed synthetic noise. Hence, to realize a more practical denoiser, we propose a novel self-supervised training framework that can remove real noise. For this, we derive the theoretic upper bound of a supervised loss where the network is guided by the downsampled blinded output. Also, we design a conditional blind-spot network (C-BSN), which selectively controls the blindness of the network to use the center pixel information. Furthermore, we exploit a random subsampler to decorrelate noise spatially, making the C-BSN free of visual artifacts that were often seen in downsample-based methods. Extensive experiments show that the proposed C-BSN achieves state-of-the-art performance on real-world datasets as a self-supervised denoiser and shows qualitatively pleasing results without any post-processing or refinement.

Related papers

• Direct Unsupervised Denoising [60.71146161035649]
  Unsupervised denoisers do not directly produce a single prediction, such as the MMSE estimate. We present an alternative approach that trains a deterministic network alongside the VAE to directly predict a central tendency.
  arXiv  Detail & Related papers  (2023-10-27T13:02:12Z)
• Random Sub-Samples Generation for Self-Supervised Real Image Denoising [9.459398471988724]
  We propose a novel self-supervised real image denoising framework named Sampling Difference As Perturbation (SDAP) We find that adding an appropriate perturbation to the training images can effectively improve the performance of BSN. The results show that it significantly outperforms other state-of-the-art self-supervised denoising methods on real-world datasets.
  arXiv  Detail & Related papers  (2023-07-31T16:39:35Z)
• Exploring Efficient Asymmetric Blind-Spots for Self-Supervised Denoising in Real-World Scenarios [44.31657750561106]
  Noise in real-world scenarios is often spatially correlated, which causes many self-supervised algorithms to perform poorly. We propose Asymmetric Tunable Blind-Spot Network (AT-BSN), where the blind-spot size can be freely adjusted. We show that our method achieves state-of-the-art, and is superior to other self-supervised algorithms in terms of computational overhead and visual effects.
  arXiv  Detail & Related papers  (2023-03-29T15:19:01Z)
• Spatially Adaptive Self-Supervised Learning for Real-World Image Denoising [73.71324390085714]
  We propose a novel perspective to solve the problem of real-world sRGB image denoising. We take into account the respective characteristics of flat and textured regions in noisy images, and construct supervisions for them separately. We present a locally aware network (LAN) to meet the requirement, while LAN itself is supervised with the output of BNN.
  arXiv  Detail & Related papers  (2023-03-27T06:18:20Z)
• I2V: Towards Texture-Aware Self-Supervised Blind Denoising using Self-Residual Learning for Real-World Images [8.763680382529412]
  pixel-shuffle downsampling (PD) has been proposed to eliminate the spatial correlation of noise. We propose self-residual learning without the PD process to maintain texture information. The results of extensive experiments show that the proposed method outperforms state-of-the-art self-supervised blind denoising approaches.
  arXiv  Detail & Related papers  (2023-02-21T08:51:17Z)
• Enhancing convolutional neural network generalizability via low-rank weight approximation [6.763245393373041]
  Sufficient denoising is often an important first step for image processing. Deep neural networks (DNNs) have been widely used for image denoising. We introduce a new self-supervised framework for image denoising based on the Tucker low-rank tensor approximation.
  arXiv  Detail & Related papers  (2022-09-26T14:11:05Z)
• Robust Deep Ensemble Method for Real-world Image Denoising [62.099271330458066]
  We propose a simple yet effective Bayesian deep ensemble (BDE) method for real-world image denoising. Our BDE achieves +0.28dB PSNR gain over the state-of-the-art denoising method. Our BDE can be extended to other image restoration tasks, and achieves +0.30dB, +0.18dB and +0.12dB PSNR gains on benchmark datasets.
  arXiv  Detail & Related papers  (2022-06-08T06:19:30Z)
• Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training [50.018580462619425]
  We propose a novel framework, namely Pixel-level Noise-aware Generative Adrial Network (PNGAN) PNGAN employs a pre-trained real denoiser to map the fake and real noisy images into a nearly noise-free solution space. For better noise fitting, we present an efficient architecture Simple Multi-versa-scale Network (SMNet) as the generator.
  arXiv  Detail & Related papers  (2022-04-06T14:09:02Z)
• Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images [98.82804259905478]
  We present Neighbor2Neighbor to train an effective image denoising model with only noisy images. In detail, input and target used to train a network are images sub-sampled from the same noisy image. A denoising network is trained on sub-sampled training pairs generated in the first stage, with a proposed regularizer as additional loss for better performance.
  arXiv  Detail & Related papers  (2021-01-08T02:03:25Z)
• Variational Denoising Network: Toward Blind Noise Modeling and Removal [59.36166491196973]
  Blind image denoising is an important yet very challenging problem in computer vision. We propose a new variational inference method, which integrates both noise estimation and image denoising.
  arXiv  Detail & Related papers  (2019-08-29T15:54:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.