View Blind-spot as Inpainting: Self-Supervised Denoising with Mask Guided Residual Convolution

• URL: http://arxiv.org/abs/2109.04970v1
• Date: Fri, 10 Sep 2021 16:10:08 GMT
• Title: View Blind-spot as Inpainting: Self-Supervised Denoising with Mask Guided Residual Convolution
• Authors: Yuhongze Zhou, Liguang Zhou, Tin Lun Lam, Yangsheng Xu
• Abstract summary: We propose a novel Mask Guided Residual Convolution (MGRConv) into common convolutional neural networks. Our MGRConv can be regarded as soft partial convolution and find a trade-off among partial convolution, learnable attention maps, and gated convolution. Experiments show that our proposed plug-and-play MGRConv can assist blind-spot based denoising network to reach promising results.
• Score: 2.179313476241343
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, self-supervised denoising methods have shown impressive performance, which circumvent painstaking collection procedure of noisy-clean image pairs in supervised denoising methods and boost denoising applicability in real world. One of well-known self-supervised denoising strategies is the blind-spot training scheme. However, a few works attempt to improve blind-spot based self-denoiser in the aspect of network architecture. In this paper, we take an intuitive view of blind-spot strategy and consider its process of using neighbor pixels to predict manipulated pixels as an inpainting process. Therefore, we propose a novel Mask Guided Residual Convolution (MGRConv) into common convolutional neural networks, e.g. U-Net, to promote blind-spot based denoising. Our MGRConv can be regarded as soft partial convolution and find a trade-off among partial convolution, learnable attention maps, and gated convolution. It enables dynamic mask learning with appropriate mask constrain. Different from partial convolution and gated convolution, it provides moderate freedom for network learning. It also avoids leveraging external learnable parameters for mask activation, unlike learnable attention maps. The experiments show that our proposed plug-and-play MGRConv can assist blind-spot based denoising network to reach promising results on both existing single-image based and dataset-based methods.

Related papers

• Asymmetric Mask Scheme for Self-Supervised Real Image Denoising [14.18283674891189]
  We propose a single mask scheme for self-supervised denoising training, which eliminates the need for blind spot operation. Our method, featuring the asymmetric mask scheme in training and inference, achieves state-of-the-art performance on existing real noisy image datasets.
  arXiv  Detail & Related papers  (2024-07-09T03:01:28Z)
• Masking Improves Contrastive Self-Supervised Learning for ConvNets, and Saliency Tells You Where [63.61248884015162]
  We aim to alleviate the burden of including masking operation into the contrastive-learning framework for convolutional neural networks. We propose to explicitly take the saliency constraint into consideration in which the masked regions are more evenly distributed among the foreground and background.
  arXiv  Detail & Related papers  (2023-09-22T09:58:38Z)
• Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Training [59.923672191632065]
  We propose a new self-supervised pre-training approach, named Masked and Permuted Vision Transformer (MaPeT) MaPeT employs autoregressive and permuted predictions to capture intra-patch dependencies. Our results demonstrate that MaPeT achieves competitive performance on ImageNet.
  arXiv  Detail & Related papers  (2023-06-12T18:12:19Z)
• Self-supervised Image Denoising with Downsampled Invariance Loss and Conditional Blind-Spot Network [12.478287906337194]
  Most representative self-supervised denoisers are based on blind-spot networks. A standard blind-spot network fails to reduce real camera noise due to the pixel-wise correlation of noise. We propose a novel self-supervised training framework that can remove real noise.
  arXiv  Detail & Related papers  (2023-04-19T08:55:27Z)
• Exploring Efficient Asymmetric Blind-Spots for Self-Supervised Denoising in Real-World Scenarios [44.31657750561106]
  Noise in real-world scenarios is often spatially correlated, which causes many self-supervised algorithms to perform poorly. We propose Asymmetric Tunable Blind-Spot Network (AT-BSN), where the blind-spot size can be freely adjusted. We show that our method achieves state-of-the-art, and is superior to other self-supervised algorithms in terms of computational overhead and visual effects.
  arXiv  Detail & Related papers  (2023-03-29T15:19:01Z)
• Spatially Adaptive Self-Supervised Learning for Real-World Image Denoising [73.71324390085714]
  We propose a novel perspective to solve the problem of real-world sRGB image denoising. We take into account the respective characteristics of flat and textured regions in noisy images, and construct supervisions for them separately. We present a locally aware network (LAN) to meet the requirement, while LAN itself is supervised with the output of BNN.
  arXiv  Detail & Related papers  (2023-03-27T06:18:20Z)
• Masked Image Training for Generalizable Deep Image Denoising [53.03126421917465]
  We present a novel approach to enhance the generalization performance of denoising networks. Our method involves masking random pixels of the input image and reconstructing the missing information during training. Our approach exhibits better generalization ability than other deep learning models and is directly applicable to real-world scenarios.
  arXiv  Detail & Related papers  (2023-03-23T09:33:44Z)
• I2V: Towards Texture-Aware Self-Supervised Blind Denoising using Self-Residual Learning for Real-World Images [8.763680382529412]
  pixel-shuffle downsampling (PD) has been proposed to eliminate the spatial correlation of noise. We propose self-residual learning without the PD process to maintain texture information. The results of extensive experiments show that the proposed method outperforms state-of-the-art self-supervised blind denoising approaches.
  arXiv  Detail & Related papers  (2023-02-21T08:51:17Z)
• Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images [98.82804259905478]
  We present Neighbor2Neighbor to train an effective image denoising model with only noisy images. In detail, input and target used to train a network are images sub-sampled from the same noisy image. A denoising network is trained on sub-sampled training pairs generated in the first stage, with a proposed regularizer as additional loss for better performance.
  arXiv  Detail & Related papers  (2021-01-08T02:03:25Z)
• Variational Denoising Network: Toward Blind Noise Modeling and Removal [59.36166491196973]
  Blind image denoising is an important yet very challenging problem in computer vision. We propose a new variational inference method, which integrates both noise estimation and image denoising.
  arXiv  Detail & Related papers  (2019-08-29T15:54:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.