Content-Augmented Feature Pyramid Network with Light Linear Transformers

• URL: http://arxiv.org/abs/2105.09464v1
• Date: Thu, 20 May 2021 02:31:31 GMT
• Title: Content-Augmented Feature Pyramid Network with Light Linear Transformers
• Authors: Yongxiang Gu, Xiaolin Qin, Yuncong Peng, Lu Li
• Abstract summary: transformers can adaptively aggregate similar features from a global view using self-attention mechanism. For object detection, Feature Pyramid Network (FPN) proposes feature interaction across layers and proves its extremely importance. In this paper, we utilize a linearized attention function to overcome above problems and build a novel architecture, named Content-Augmented Feature Pyramid Network (CA-FPN)
• Score: 7.035864400598343
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, plenty of work has tried to introduce transformers into computer vision tasks, with good results. Unlike classic convolution networks, which extract features within a local receptive field, transformers can adaptively aggregate similar features from a global view using self-attention mechanism. For object detection, Feature Pyramid Network (FPN) proposes feature interaction across layers and proves its extremely importance. However, its interaction is still in a local manner, which leaves a lot of room for improvement. Since transformer was originally designed for NLP tasks, adapting processing subject directly from text to image will cause unaffordable computation and space overhead. In this paper, we utilize a linearized attention function to overcome above problems and build a novel architecture, named Content-Augmented Feature Pyramid Network (CA-FPN), which proposes a global content extraction module and deeply combines with FPN through light linear transformers. What's more, light transformers can further make the application of multi-head attention mechanism easier. Most importantly, our CA-FPN can be readily plugged into existing FPN-based models. Extensive experiments on the challenging COCO object detection dataset demonstrated that our CA-FPN significantly outperforms competitive baselines without bells and whistles. Code will be made publicly available.

Related papers

• DuoFormer: Leveraging Hierarchical Visual Representations by Local and Global Attention [1.5624421399300303]
  We propose a novel hierarchical transformer model that adeptly integrates the feature extraction capabilities of Convolutional Neural Networks (CNNs) with the advanced representational potential of Vision Transformers (ViTs) Addressing the lack of inductive biases and dependence on extensive training datasets in ViTs, our model employs a CNN backbone to generate hierarchical visual representations. These representations are then adapted for transformer input through an innovative patch tokenization.
  arXiv  Detail & Related papers  (2024-07-18T22:15:35Z)
• CFPFormer: Feature-pyramid like Transformer Decoder for Segmentation and Detection [1.837431956557716]
  Feature pyramids have been widely adopted in convolutional neural networks (CNNs) and transformers for tasks like medical image segmentation and object detection. We propose a novel decoder block that integrates feature pyramids and transformers. Our model achieves superior performance in detecting small objects compared to existing methods.
  arXiv  Detail & Related papers  (2024-04-23T18:46:07Z)
• A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks [60.38369406877899]
  Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data. transformer models excel in handling long dependencies between input sequence elements and enable parallel processing. Our survey encompasses the identification of the top five application domains for transformer-based models.
  arXiv  Detail & Related papers  (2023-06-11T23:13:51Z)
• Feature Shrinkage Pyramid for Camouflaged Object Detection with Transformers [34.42710399235461]
  Vision transformers have recently shown strong global context modeling capabilities in camouflaged object detection. They suffer from two major limitations: less effective locality modeling and insufficient feature aggregation in decoders. We propose a novel transformer-based Feature Shrinkage Pyramid Network (FSPNet), which aims to hierarchically decode locality-enhanced neighboring transformer features.
  arXiv  Detail & Related papers  (2023-03-26T20:50:58Z)
• Hierarchical Point Attention for Indoor 3D Object Detection [111.04397308495618]
  This work proposes two novel attention operations as generic hierarchical designs for point-based transformer detectors. First, we propose Multi-Scale Attention (MS-A) that builds multi-scale tokens from a single-scale input feature to enable more fine-grained feature learning. Second, we propose Size-Adaptive Local Attention (Local-A) with adaptive attention regions for localized feature aggregation within bounding box proposals.
  arXiv  Detail & Related papers  (2023-01-06T18:52:12Z)
• Cross-receptive Focused Inference Network for Lightweight Image Super-Resolution [64.25751738088015]
  Transformer-based methods have shown impressive performance in single image super-resolution (SISR) tasks. Transformers that need to incorporate contextual information to extract features dynamically are neglected. We propose a lightweight Cross-receptive Focused Inference Network (CFIN) that consists of a cascade of CT Blocks mixed with CNN and Transformer.
  arXiv  Detail & Related papers  (2022-07-06T16:32:29Z)
• Vision Transformer with Convolutions Architecture Search [72.70461709267497]
  We propose an architecture search method-Vision Transformer with Convolutions Architecture Search (VTCAS) The high-performance backbone network searched by VTCAS introduces the desirable features of convolutional neural networks into the Transformer architecture. It enhances the robustness of the neural network for object recognition, especially in the low illumination indoor scene.
  arXiv  Detail & Related papers  (2022-03-20T02:59:51Z)
• PnP-DETR: Towards Efficient Visual Analysis with Transformers [146.55679348493587]
  Recently, DETR pioneered the solution vision tasks with transformers, it directly translates the image feature map into the object result. Recent transformer-based image recognition model andTT show consistent efficiency gain.
  arXiv  Detail & Related papers  (2021-09-15T01:10:30Z)
• Transformers Solve the Limited Receptive Field for Monocular Depth Prediction [82.90445525977904]
  We propose TransDepth, an architecture which benefits from both convolutional neural networks and transformers. This is the first paper which applies transformers into pixel-wise prediction problems involving continuous labels.
  arXiv  Detail & Related papers  (2021-03-22T18:00:13Z)
• Toward Transformer-Based Object Detection [12.704056181392415]
  Vision Transformers can be used as a backbone by a common detection task head to produce competitive COCO results. ViT-FRCNN demonstrates several known properties associated with transformers, including large pretraining capacity and fast fine-tuning performance. We view ViT-FRCNN as an important stepping stone toward a pure-transformer solution of complex vision tasks such as object detection.
  arXiv  Detail & Related papers  (2020-12-17T22:33:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.