BA-Net: Bridge Attention for Deep Convolutional Neural Networks

• URL: http://arxiv.org/abs/2112.04150v1
• Date: Wed, 8 Dec 2021 07:39:18 GMT
• Title: BA-Net: Bridge Attention for Deep Convolutional Neural Networks
• Authors: Yue Zhao, Junzhou Chen, Zirui Zhang and Ronghui Zhang
• Abstract summary: Bridge Attention Net (BA-Net) is proposed for better channel attention mechanisms. BA-Net can not only provide richer features to calculate channel weight when feedforward, but also multiply paths of parameters updating when backforward.
• Score: 4.459779019776622
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, channel attention mechanism is widely investigated for its great potential in improving the performance of deep convolutional neural networks (CNNs). However, in most existing methods, only the output of the adjacent convolution layer is fed to the attention layer for calculating the channel weights. Information from other convolution layers is ignored. With these observations, a simple strategy, named Bridge Attention Net (BA-Net), is proposed for better channel attention mechanisms. The main idea of this design is to bridge the outputs of the previous convolution layers through skip connections for channel weights generation. BA-Net can not only provide richer features to calculate channel weight when feedforward, but also multiply paths of parameters updating when backforward. Comprehensive evaluation demonstrates that the proposed approach achieves state-of-the-art performance compared with the existing methods in regards to accuracy and speed. Bridge Attention provides a fresh perspective on the design of neural network architectures and shows great potential in improving the performance of the existing channel attention mechanisms. The code is available at \url{https://github.com/zhaoy376/Attention-mechanism

Related papers

• Channel Attention for Quantum Convolutional Neural Networks [0.0]
  Quantum convolutional neural networks (QCNNs) have gathered attention as one of the most promising algorithms for quantum machine learning. We propose a channel attention mechanism for QCNNs and show the effectiveness of this approach for quantum phase classification problems.
  arXiv  Detail & Related papers  (2023-11-06T04:54:05Z)
• RDRN: Recursively Defined Residual Network for Image Super-Resolution [58.64907136562178]
  Deep convolutional neural networks (CNNs) have obtained remarkable performance in single image super-resolution. We propose a novel network architecture which utilizes attention blocks efficiently.
  arXiv  Detail & Related papers  (2022-11-17T11:06:29Z)
• Interference Cancellation GAN Framework for Dynamic Channels [74.22393885274728]
  We introduce an online training framework that can adapt to any changes in the channel. Our framework significantly outperforms recent neural network models on highly dynamic channels.
  arXiv  Detail & Related papers  (2022-08-17T02:01:18Z)
• DeepTx: Deep Learning Beamforming with Channel Prediction [8.739166282613118]
  In this study, we focus on machine learning algorithms for the transmitter. We consider beamforming and propose a CNN which, for a given uplink channel estimate as input, outputs downlink channel information to be used for beamforming. The main task of the neural network is to predict the channel evolution between uplink and downlink slots, but it can also learn to handle inefficiencies and errors in the whole chain.
  arXiv  Detail & Related papers  (2022-02-16T11:19:54Z)
• Exploring Gradient Flow Based Saliency for DNN Model Compression [21.993801817422572]
  Model pruning aims to reduce the deep neural network (DNN) model size or computational overhead. Traditional model pruning methods that evaluates the channel significance for DNN pay too much attention to the local analysis of each channel. We propose a new model pruning method from a new perspective of flow in this paper.
  arXiv  Detail & Related papers  (2021-10-24T16:09:40Z)
• CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization [61.71504948770445]
  We propose a novel channel pruning method via Class-Aware Trace Ratio Optimization (CATRO) to reduce the computational burden and accelerate the model inference. We show that CATRO achieves higher accuracy with similar cost or lower cost with similar accuracy than other state-of-the-art channel pruning algorithms. Because of its class-aware property, CATRO is suitable to prune efficient networks adaptively for various classification subtasks, enhancing handy deployment and usage of deep networks in real-world applications.
  arXiv  Detail & Related papers  (2021-10-21T06:26:31Z)
• Channel Scaling: A Scale-and-Select Approach for Transfer Learning [2.6304695993930594]
  Transfer learning with pre-trained neural networks is a common strategy for training classifiers in medical image analysis. We propose a novel approach to efficiently build small and well performing networks by introducing the channel-scaling layers. By imposing L1 regularization and thresholding on the scaling weights, this framework iteratively removes unnecessary feature channels from a pre-trained model.
  arXiv  Detail & Related papers  (2021-03-22T23:26:57Z)
• End-to-end learnable EEG channel selection with deep neural networks [72.21556656008156]
  We propose a framework to embed the EEG channel selection in the neural network itself. We deal with the discrete nature of this new optimization problem by employing continuous relaxations of the discrete channel selection parameters. This generic approach is evaluated on two different EEG tasks.
  arXiv  Detail & Related papers  (2021-02-11T13:44:07Z)
• Direct Quantization for Training Highly Accurate Low Bit-width Deep Neural Networks [73.29587731448345]
  This paper proposes two novel techniques to train deep convolutional neural networks with low bit-width weights and activations. First, to obtain low bit-width weights, most existing methods obtain the quantized weights by performing quantization on the full-precision network weights. Second, to obtain low bit-width activations, existing works consider all channels equally.
  arXiv  Detail & Related papers  (2020-12-26T15:21:18Z)
• Channel Pruning Guided by Spatial and Channel Attention for DNNs in Intelligent Edge Computing [15.248962858090431]
  A critical challenge is to determine which channels are to be removed, so that the model accuracy will not be negatively affected. We propose a new attention module combining both spatial and channel attention. With the guidance of SCA, our CPSCA approach achieves higher inference accuracy than other state-of-the-art pruning methods.
  arXiv  Detail & Related papers  (2020-11-08T02:40:06Z)
• Network Adjustment: Channel Search Guided by FLOPs Utilization Ratio [101.84651388520584]
  This paper presents a new framework named network adjustment, which considers network accuracy as a function of FLOPs. Experiments on standard image classification datasets and a wide range of base networks demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-06T15:51:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.