WaveNets: Wavelet Channel Attention Networks

• URL: http://arxiv.org/abs/2211.02695v2
• Date: Tue, 12 Mar 2024 15:12:55 GMT
• Title: WaveNets: Wavelet Channel Attention Networks
• Abstract summary: We propose Wavelet transform compression as a solution to the channel representation problem. We generalize channel attention using Wavelet compression and name it WaveNet. Our method outperforms the baseline SENet, and achieves the state-of-the-art results.
• Score: 9.842943431749191
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Channel Attention reigns supreme as an effective technique in the field of computer vision. However, the proposed channel attention by SENet suffers from information loss in feature learning caused by the use of Global Average Pooling (GAP) to represent channels as scalars. Thus, designing effective channel attention mechanisms requires finding a solution to enhance features preservation in modeling channel inter-dependencies. In this work, we utilize Wavelet transform compression as a solution to the channel representation problem. We first test wavelet transform as an Auto-Encoder model equipped with conventional channel attention module. Next, we test wavelet transform as a standalone channel compression method. We prove that global average pooling is equivalent to the recursive approximate Haar wavelet transform. With this proof, we generalize channel attention using Wavelet compression and name it WaveNet. Implementation of our method can be embedded within existing channel attention methods with a couple of lines of code. We test our proposed method using ImageNet dataset for image classification task. Our method outperforms the baseline SENet, and achieves the state-of-the-art results. Our code implementation is publicly available at https://github.com/hady1011/WaveNet-C.

Related papers

• ChWDTA: Channel-wise Wavelet-Domain Transformer Attention and Entropy Modeling for Learned Image Compression [36.75128193748412]
  We introduce channel-wise wavelet transforms into both the transformer and entropy-coding components.<n>With this configuration, the proposed scheme obtains BD-rate reductions of -17.82%, -19.15%, and -22.56% on the Kodak, CLIC Professional Validation, and Tecnick test sets.
  arXiv  Detail & Related papers  (2026-05-27T04:36:08Z)
• CP Loss: Channel-wise Perceptual Loss for Time Series Forecasting [67.3477355449697]
  We propose a Channel-wise Perceptual Loss (CP Loss) for time-series data.<n>We learn a unique perceptual space for each channel that is adapted to its characteristics.<n>Loss is calculated within these perception spaces to optimize the model.
  arXiv  Detail & Related papers  (2026-01-25T15:31:37Z)
• Wavelet-based Bi-dimensional Aggregation Network for SAR Image Change Detection [53.842568573251214]
  Experimental results on three SAR datasets demonstrate that our WBANet significantly outperforms contemporary state-of-the-art methods. Our WBANet achieves 98.33%, 96.65%, and 96.62% of percentage of correct classification (PCC) on the respective datasets.
  arXiv  Detail & Related papers  (2024-07-18T04:36:10Z)
• Joint Channel Estimation and Feedback with Masked Token Transformers in Massive MIMO Systems [74.52117784544758]
  This paper proposes an encoder-decoder based network that unveils the intrinsic frequency-domain correlation within the CSI matrix. The entire encoder-decoder network is utilized for channel compression. Our method outperforms state-of-the-art channel estimation and feedback techniques in joint tasks.
  arXiv  Detail & Related papers  (2023-06-08T06:15:17Z)
• Trainable Wavelet Neural Network for Non-Stationary Signals [0.0]
  This work introduces a wavelet neural network to learn a filter-bank specialized to fit non-stationary signals and improve interpretability and performance for digital signal processing. The network uses a wavelet transform as the first layer of a neural network where the convolution is a parameterized function of the complex Morlet wavelet.
  arXiv  Detail & Related papers  (2022-05-06T16:41:27Z)
• Attention Based Neural Networks for Wireless Channel Estimation [1.0499453838486013]
  We propose a new hybrid encoder-decoder structure (called HA02) for the first time which exploits the attention mechanism to focus on the most important input information. In particular, we implement a transformer encoder block as the encoder to achieve the sparsity in the input features and a residual neural network as the decoder respectively. Our simulations show superior estimation performance compared with other candidate neural network methods for channel estimation.
  arXiv  Detail & Related papers  (2022-04-28T12:54:19Z)
• Group Fisher Pruning for Practical Network Compression [58.25776612812883]
  We present a general channel pruning approach that can be applied to various complicated structures. We derive a unified metric based on Fisher information to evaluate the importance of a single channel and coupled channels. Our method can be used to prune any structures including those with coupled channels.
  arXiv  Detail & Related papers  (2021-08-02T08:21:44Z)
• Towards Boosting the Channel Attention in Real Image Denoising : Sub-band Pyramid Attention [3.264560291660082]
  This paper proposes a novel Sub-band Pyramid Attention (SPA) based on wavelet sub-band pyramid to recalibrate the frequency components of the extracted features. We equip the SPA blocks on a network designed for real image denoising. Experimental results show that the proposed method achieves a remarkable improvement.
  arXiv  Detail & Related papers  (2020-12-23T04:28:33Z)
• FcaNet: Frequency Channel Attention Networks [20.6292655895549]
  We rethink channel attention using frequency analysis. We propose FcaNet with novel multi-spectral channel attention. Our method could improve by 1.8% in terms of Top-1 accuracy on ImageNet.
  arXiv  Detail & Related papers  (2020-12-22T08:33:27Z)
• Wavelet Channel Attention Module with a Fusion Network for Single Image Deraining [46.62290347397139]
  Single image deraining is a crucial problem because rain severely degenerates the visibility of images. We propose the new convolutional neural network (CNN) called the wavelet channel attention module with a fusion network.
  arXiv  Detail & Related papers  (2020-07-17T18:06:13Z)
• Channel-Level Variable Quantization Network for Deep Image Compression [50.3174629451739]
  We propose a channel-level variable quantization network to dynamically allocate more convolutions for significant channels and withdraws for negligible channels. Our method achieves superior performance and can produce much better visual reconstructions.
  arXiv  Detail & Related papers  (2020-07-15T07:20:39Z)
• Wavelet Integrated CNNs for Noise-Robust Image Classification [51.18193090255933]
  We enhance CNNs by replacing max-pooling, strided-convolution, and average-pooling with Discrete Wavelet Transform (DWT) WaveCNets, the wavelet integrated versions of VGG, ResNets, and DenseNet, achieve higher accuracy and better noise-robustness than their vanilla versions.
  arXiv  Detail & Related papers  (2020-05-07T09:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.