Global Self-Attention Networks for Image Recognition

• URL: http://arxiv.org/abs/2010.03019v2
• Date: Wed, 14 Oct 2020 05:46:59 GMT
• Title: Global Self-Attention Networks for Image Recognition
• Authors: Zhuoran Shen, Irwan Bello, Raviteja Vemulapalli, Xuhui Jia, Ching-Hui Chen
• Abstract summary: This work introduces a new global self-attention module, referred to as the GSA module, which is efficient enough to serve as the backbone component of a deep network. Based on the proposed GSA module, we introduce new standalone global attention-based deep networks that use GSA modules instead of convolutions to model pixel interactions.
• Score: 15.57942306567032
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, a series of works in computer vision have shown promising results on various image and video understanding tasks using self-attention. However, due to the quadratic computational and memory complexities of self-attention, these works either apply attention only to low-resolution feature maps in later stages of a deep network or restrict the receptive field of attention in each layer to a small local region. To overcome these limitations, this work introduces a new global self-attention module, referred to as the GSA module, which is efficient enough to serve as the backbone component of a deep network. This module consists of two parallel layers: a content attention layer that attends to pixels based only on their content and a positional attention layer that attends to pixels based on their spatial locations. The output of this module is the sum of the outputs of the two layers. Based on the proposed GSA module, we introduce new standalone global attention-based deep networks that use GSA modules instead of convolutions to model pixel interactions. Due to the global extent of the proposed GSA module, a GSA network has the ability to model long-range pixel interactions throughout the network. Our experimental results show that GSA networks outperform the corresponding convolution-based networks significantly on the CIFAR-100 and ImageNet datasets while using less parameters and computations. The proposed GSA networks also outperform various existing attention-based networks on the ImageNet dataset.

Related papers

• DDU-Net: A Domain Decomposition-based CNN for High-Resolution Image Segmentation on Multiple GPUs [46.873264197900916]
  A domain decomposition-based U-Net architecture is introduced, which partitions input images into non-overlapping patches. A communication network is added to facilitate inter-patch information exchange to enhance the understanding of spatial context. Results show that the approach achieves a $2-3,%$ higher intersection over union (IoU) score compared to the same network without inter-patch communication.
  arXiv  Detail & Related papers  (2024-07-31T01:07:21Z)
• SA2-Net: Scale-aware Attention Network for Microscopic Image Segmentation [36.286876343282565]
  Microscopic image segmentation is a challenging task, wherein the objective is to assign semantic labels to each pixel in a given microscopic image. We introduce SA2-Net, an attention-guided method that leverages multi-scale feature learning to handle diverse structures within microscopic images.
  arXiv  Detail & Related papers  (2023-09-28T17:58:05Z)
• De-coupling and De-positioning Dense Self-supervised Learning [65.56679416475943]
  Dense Self-Supervised Learning (SSL) methods address the limitations of using image-level feature representations when handling images with multiple objects. We show that they suffer from coupling and positional bias, which arise from the receptive field increasing with layer depth and zero-padding. We demonstrate the benefits of our method on COCO and on a new challenging benchmark, OpenImage-MINI, for object classification, semantic segmentation, and object detection.
  arXiv  Detail & Related papers  (2023-03-29T18:07:25Z)
• LoG-CAN: local-global Class-aware Network for semantic segmentation of remote sensing images [4.124381172041927]
  We present LoG-CAN, a multi-scale semantic segmentation network with a global class-aware (GCA) module and local class-aware (LCA) modules to remote sensing images. Specifically, the GCA module captures the global representations of class-wise context modeling to circumvent background interference; the LCA modules generate local class representations as intermediate aware elements, indirectly associating pixels with global class representations to reduce variance within a class.
  arXiv  Detail & Related papers  (2023-03-14T09:44:29Z)
• CRCNet: Few-shot Segmentation with Cross-Reference and Region-Global Conditional Networks [59.85183776573642]
  Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images. We propose a Cross-Reference and Local-Global Networks (CRCNet) for few-shot segmentation. Our network can better find the co-occurrent objects in the two images with a cross-reference mechanism.
  arXiv  Detail & Related papers  (2022-08-23T06:46:18Z)
• Multi-level Second-order Few-shot Learning [111.0648869396828]
  We propose a Multi-level Second-order (MlSo) few-shot learning network for supervised or unsupervised few-shot image classification and few-shot action recognition. We leverage so-called power-normalized second-order base learner streams combined with features that express multiple levels of visual abstraction. We demonstrate respectable results on standard datasets such as Omniglot, mini-ImageNet, tiered-ImageNet, Open MIC, fine-grained datasets such as CUB Birds, Stanford Dogs and Cars, and action recognition datasets such as HMDB51, UCF101, and mini-MIT.
  arXiv  Detail & Related papers  (2022-01-15T19:49:00Z)
• Specificity-preserving RGB-D Saliency Detection [103.3722116992476]
  We propose a specificity-preserving network (SP-Net) for RGB-D saliency detection. Two modality-specific networks and a shared learning network are adopted to generate individual and shared saliency maps. Experiments on six benchmark datasets demonstrate that our SP-Net outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-18T14:14:22Z)
• DMSANet: Dual Multi Scale Attention Network [0.0]
  We propose a new attention module that not only achieves the best performance but also has lesser parameters compared to most existing models. Our attention module can easily be integrated with other convolutional neural networks because of its lightweight nature.
  arXiv  Detail & Related papers  (2021-06-13T10:31:31Z)
• A Novel Adaptive Deep Network for Building Footprint Segmentation [0.0]
  We propose a novel network-based on Pix2Pix methodology to solve the problem of inaccurate boundaries obtained by converting satellite images into maps. Our framework includes two generators where the first generator extracts localization features in order to merge them with the boundary features extracted from the second generator to segment all detailed building edges. Different strategies are implemented to enhance the quality of the proposed networks' results, implying that the proposed network outperforms state-of-the-art networks in segmentation accuracy with a large margin for all evaluation metrics.
  arXiv  Detail & Related papers  (2021-02-27T18:13:48Z)
• Boundary-Aware Segmentation Network for Mobile and Web Applications [60.815545591314915]
  Boundary-Aware Network (BASNet) is integrated with a predict-refine architecture and a hybrid loss for highly accurate image segmentation. BASNet runs at over 70 fps on a single GPU which benefits many potential real applications. Based on BASNet, we further developed two (close to) commercial applications: AR COPY & PASTE, in which BASNet is augmented reality for "COPY" and "PASTING" real-world objects, and OBJECT CUT, which is a web-based tool for automatic object background removal.
  arXiv  Detail & Related papers  (2021-01-12T19:20:26Z)
• X-ModalNet: A Semi-Supervised Deep Cross-Modal Network for Classification of Remote Sensing Data [69.37597254841052]
  We propose a novel cross-modal deep-learning framework called X-ModalNet. X-ModalNet generalizes well, owing to propagating labels on an updatable graph constructed by high-level features on the top of the network. We evaluate X-ModalNet on two multi-modal remote sensing datasets (HSI-MSI and HSI-SAR) and achieve a significant improvement in comparison with several state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-06-24T15:29:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.