MM-BSN: Self-Supervised Image Denoising for Real-World with Multi-Mask based on Blind-Spot Network

• URL: http://arxiv.org/abs/2304.01598v3
• Date: Fri, 7 Apr 2023 09:31:51 GMT
• Title: MM-BSN: Self-Supervised Image Denoising for Real-World with Multi-Mask based on Blind-Spot Network
• Authors: Dan Zhang, Fangfang Zhou, Yuwen Jiang and Zhengming Fu
• Abstract summary: In self-supervised image denoising, blind-spot network (BSN) is one of the most common methods. We propose a multi-mask strategy using multiple convolutional kernels masked in different shapes to break the noise spatial correlation. We show that different masks can cause significant performance differences, and the proposed MM-BSN can efficiently fuse the features extracted by multi-masked layers.
• Score: 2.8376287904582176
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in deep learning have been pushing image denoising techniques to a new level. In self-supervised image denoising, blind-spot network (BSN) is one of the most common methods. However, most of the existing BSN algorithms use a dot-based central mask, which is recognized as inefficient for images with large-scale spatially correlated noise. In this paper, we give the definition of large-noise and propose a multi-mask strategy using multiple convolutional kernels masked in different shapes to further break the noise spatial correlation. Furthermore, we propose a novel self-supervised image denoising method that combines the multi-mask strategy with BSN (MM-BSN). We show that different masks can cause significant performance differences, and the proposed MM-BSN can efficiently fuse the features extracted by multi-masked layers, while recovering the texture structures destroyed by multi-masking and information transmission. Our MM-BSN can be used to address the problem of large-noise denoising, which cannot be efficiently handled by other BSN methods. Extensive experiments on public real-world datasets demonstrate that the proposed MM-BSN achieves state-of-the-art performance among self-supervised and even unpaired image denoising methods for sRGB images denoising, without any labelling effort or prior knowledge. Code can be found in https://github.com/dannie125/MM-BSN.

Related papers

• Asymmetric Mask Scheme for Self-Supervised Real Image Denoising [14.18283674891189]
  We propose a single mask scheme for self-supervised denoising training, which eliminates the need for blind spot operation. Our method, featuring the asymmetric mask scheme in training and inference, achieves state-of-the-art performance on existing real noisy image datasets.
  arXiv  Detail & Related papers  (2024-07-09T03:01:28Z)
• Random Sub-Samples Generation for Self-Supervised Real Image Denoising [9.459398471988724]
  We propose a novel self-supervised real image denoising framework named Sampling Difference As Perturbation (SDAP) We find that adding an appropriate perturbation to the training images can effectively improve the performance of BSN. The results show that it significantly outperforms other state-of-the-art self-supervised denoising methods on real-world datasets.
  arXiv  Detail & Related papers  (2023-07-31T16:39:35Z)
• Enhancing convolutional neural network generalizability via low-rank weight approximation [6.763245393373041]
  Sufficient denoising is often an important first step for image processing. Deep neural networks (DNNs) have been widely used for image denoising. We introduce a new self-supervised framework for image denoising based on the Tucker low-rank tensor approximation.
  arXiv  Detail & Related papers  (2022-09-26T14:11:05Z)
• Multi-stage image denoising with the wavelet transform [125.2251438120701]
  Deep convolutional neural networks (CNNs) are used for image denoising via automatically mining accurate structure information. We propose a multi-stage image denoising CNN with the wavelet transform (MWDCNN) via three stages, i.e., a dynamic convolutional block (DCB), two cascaded wavelet transform and enhancement blocks (WEBs) and residual block (RB)
  arXiv  Detail & Related papers  (2022-09-26T03:28:23Z)
• Robust Deep Ensemble Method for Real-world Image Denoising [62.099271330458066]
  We propose a simple yet effective Bayesian deep ensemble (BDE) method for real-world image denoising. Our BDE achieves +0.28dB PSNR gain over the state-of-the-art denoising method. Our BDE can be extended to other image restoration tasks, and achieves +0.30dB, +0.18dB and +0.12dB PSNR gains on benchmark datasets.
  arXiv  Detail & Related papers  (2022-06-08T06:19:30Z)
• Zero-shot Blind Image Denoising via Implicit Neural Representations [77.79032012459243]
  We propose an alternative denoising strategy that leverages the architectural inductive bias of implicit neural representations (INRs) We show that our method outperforms existing zero-shot denoising methods under an extensive set of low-noise or real-noise scenarios.
  arXiv  Detail & Related papers  (2022-04-05T12:46:36Z)
• AP-BSN: Self-Supervised Denoising for Real-World Images via Asymmetric PD and Blind-Spot Network [60.650035708621786]
  Blind-spot network (BSN) and its variants have made significant advances in self-supervised denoising. It is challenging to deal with spatially correlated real-world noise using self-supervised BSN. Recently, pixel-shuffle downsampling (PD) has been proposed to remove the spatial correlation of real-world noise. We propose an Asymmetric PD (AP) to address this issue, which introduces different PD stride factors for training and inference.
  arXiv  Detail & Related papers  (2022-03-22T15:04:37Z)
• View Blind-spot as Inpainting: Self-Supervised Denoising with Mask Guided Residual Convolution [2.179313476241343]
  We propose a novel Mask Guided Residual Convolution (MGRConv) into common convolutional neural networks. Our MGRConv can be regarded as soft partial convolution and find a trade-off among partial convolution, learnable attention maps, and gated convolution. Experiments show that our proposed plug-and-play MGRConv can assist blind-spot based denoising network to reach promising results.
  arXiv  Detail & Related papers  (2021-09-10T16:10:08Z)
• Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images [98.82804259905478]
  We present Neighbor2Neighbor to train an effective image denoising model with only noisy images. In detail, input and target used to train a network are images sub-sampled from the same noisy image. A denoising network is trained on sub-sampled training pairs generated in the first stage, with a proposed regularizer as additional loss for better performance.
  arXiv  Detail & Related papers  (2021-01-08T02:03:25Z)
• Variational Denoising Network: Toward Blind Noise Modeling and Removal [59.36166491196973]
  Blind image denoising is an important yet very challenging problem in computer vision. We propose a new variational inference method, which integrates both noise estimation and image denoising.
  arXiv  Detail & Related papers  (2019-08-29T15:54:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.