Rethinking Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising

• URL: http://arxiv.org/abs/2404.07846v3
• Date: Tue, 17 Dec 2024 02:26:54 GMT
• Title: Rethinking Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising
• Authors: Junyi Li, Zhilu Zhang, Wangmeng Zuo,
• Abstract summary: Blind-spot networks (BSN) have been prevalent neural architectures in self-supervised image denoising (SSID) We build a Transformer-based Blind-Spot Network (TBSN) which shows strong local fitting and global perspective abilities.
• Score: 94.09442506816724
• License:
• Abstract: Blind-spot networks (BSN) have been prevalent neural architectures in self-supervised image denoising (SSID). However, most existing BSNs are conducted with convolution layers. Although transformers have shown the potential to overcome the limitations of convolutions in many image restoration tasks, the attention mechanisms may violate the blind-spot requirement, thereby restricting their applicability in BSN. To this end, we propose to analyze and redesign the channel and spatial attentions to meet the blind-spot requirement. Specifically, channel self-attention may leak the blind-spot information in multi-scale architectures, since the downsampling shuffles the spatial feature into channel dimensions. To alleviate this problem, we divide the channel into several groups and perform channel attention separately. For spatial selfattention, we apply an elaborate mask to the attention matrix to restrict and mimic the receptive field of dilated convolution. Based on the redesigned channel and window attentions, we build a Transformer-based Blind-Spot Network (TBSN), which shows strong local fitting and global perspective abilities. Furthermore, we introduce a knowledge distillation strategy that distills TBSN into smaller denoisers to improve computational efficiency while maintaining performance. Extensive experiments on real-world image denoising datasets show that TBSN largely extends the receptive field and exhibits favorable performance against state-of-theart SSID methods.

Related papers

• Hyperspectral Image Denoising via Self-Modulating Convolutional Neural Networks [15.700048595212051]
  We introduce a self-modulating convolutional neural network which utilizes correlated spectral and spatial information. At the core of the model lies a novel block, which allows the network to transform the features in an adaptive manner based on the adjacent spectral data. Experimental analysis on both synthetic and real data shows that the proposed SM-CNN outperforms other state-of-the-art HSI denoising methods.
  arXiv  Detail & Related papers  (2023-09-15T06:57:43Z)
• Joint Channel Estimation and Feedback with Masked Token Transformers in Massive MIMO Systems [74.52117784544758]
  This paper proposes an encoder-decoder based network that unveils the intrinsic frequency-domain correlation within the CSI matrix. The entire encoder-decoder network is utilized for channel compression. Our method outperforms state-of-the-art channel estimation and feedback techniques in joint tasks.
  arXiv  Detail & Related papers  (2023-06-08T06:15:17Z)
• Exploring Efficient Asymmetric Blind-Spots for Self-Supervised Denoising in Real-World Scenarios [44.31657750561106]
  Noise in real-world scenarios is often spatially correlated, which causes many self-supervised algorithms to perform poorly. We propose Asymmetric Tunable Blind-Spot Network (AT-BSN), where the blind-spot size can be freely adjusted. We show that our method achieves state-of-the-art, and is superior to other self-supervised algorithms in terms of computational overhead and visual effects.
  arXiv  Detail & Related papers  (2023-03-29T15:19:01Z)
• Spatially Adaptive Self-Supervised Learning for Real-World Image Denoising [73.71324390085714]
  We propose a novel perspective to solve the problem of real-world sRGB image denoising. We take into account the respective characteristics of flat and textured regions in noisy images, and construct supervisions for them separately. We present a locally aware network (LAN) to meet the requirement, while LAN itself is supervised with the output of BNN.
  arXiv  Detail & Related papers  (2023-03-27T06:18:20Z)
• Attention Aided CSI Wireless Localization [19.50869817974852]
  We propose attention-based CSI for robust feature learning in deep neural networks (DNNs) We evaluate the performance of attended features in centralized and distributed massive MIMO systems for ray-tracing channels in two non-stationary railway track environments.
  arXiv  Detail & Related papers  (2022-03-20T09:38:01Z)
• Towards Boosting the Channel Attention in Real Image Denoising : Sub-band Pyramid Attention [3.264560291660082]
  This paper proposes a novel Sub-band Pyramid Attention (SPA) based on wavelet sub-band pyramid to recalibrate the frequency components of the extracted features. We equip the SPA blocks on a network designed for real image denoising. Experimental results show that the proposed method achieves a remarkable improvement.
  arXiv  Detail & Related papers  (2020-12-23T04:28:33Z)
• Volumetric Transformer Networks [88.85542905676712]
  We introduce a learnable module, the volumetric transformer network (VTN) VTN predicts channel-wise warping fields so as to reconfigure intermediate CNN features spatially and channel-wisely. Our experiments show that VTN consistently boosts the features' representation power and consequently the networks' accuracy on fine-grained image recognition and instance-level image retrieval.
  arXiv  Detail & Related papers  (2020-07-18T14:00:12Z)
• Decentralized Learning for Channel Allocation in IoT Networks over Unlicensed Bandwidth as a Contextual Multi-player Multi-armed Bandit Game [134.88020946767404]
  We study a decentralized channel allocation problem in an ad-hoc Internet of Things network underlaying on the spectrum licensed to a primary cellular network. Our study maps this problem into a contextual multi-player, multi-armed bandit game, and proposes a purely decentralized, three-stage policy learning algorithm through trial-and-error.
  arXiv  Detail & Related papers  (2020-03-30T10:05:35Z)
• ADRN: Attention-based Deep Residual Network for Hyperspectral Image Denoising [52.01041506447195]
  We propose an attention-based deep residual network to learn a mapping from noisy HSI to the clean one. Experimental results demonstrate that our proposed ADRN scheme outperforms the state-of-the-art methods both in quantitative and visual evaluations.
  arXiv  Detail & Related papers  (2020-03-04T08:36:27Z)
• Channel-Attention Dense U-Net for Multichannel Speech Enhancement [21.94418736688929]
  We introduce a channel-attention mechanism inside the deep architecture to mimic beamforming. We demonstrate the superior performance of the network against the state-of-the-art approaches on the CHiME-3 dataset.
  arXiv  Detail & Related papers  (2020-01-30T19:56:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.