IncepFormer: Efficient Inception Transformer with Pyramid Pooling for Semantic Segmentation

• URL: http://arxiv.org/abs/2212.03035v1
• Date: Tue, 6 Dec 2022 15:08:00 GMT
• Title: IncepFormer: Efficient Inception Transformer with Pyramid Pooling for Semantic Segmentation
• Abstract summary: IncepFormer is a powerful semantic segmentation architecture. It harvests global context and fine localisation features simultaneously. It is superior to state-of-the-art methods in both accuracy and speed.
• Score: 18.195826175511552
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic segmentation usually benefits from global contexts, fine localisation information, multi-scale features, etc. To advance Transformer-based segmenters with these aspects, we present a simple yet powerful semantic segmentation architecture, termed as IncepFormer. IncepFormer has two critical contributions as following. First, it introduces a novel pyramid structured Transformer encoder which harvests global context and fine localisation features simultaneously. These features are concatenated and fed into a convolution layer for final per-pixel prediction. Second, IncepFormer integrates an Inception-like architecture with depth-wise convolutions, and a light-weight feed-forward module in each self-attention layer, efficiently obtaining rich local multi-scale object features. Extensive experiments on five benchmarks show that our IncepFormer is superior to state-of-the-art methods in both accuracy and speed, e.g., 1) our IncepFormer-S achieves 47.7% mIoU on ADE20K which outperforms the existing best method by 1% while only costs half parameters and fewer FLOPs. 2) Our IncepFormer-B finally achieves 82.0% mIoU on Cityscapes dataset with 39.6M parameters. Code is available:github.com/shendu0321/IncepFormer.

Related papers

• EDTformer: An Efficient Decoder Transformer for Visual Place Recognition [34.875097011568336]
  Visual place recognition (VPR) aims to determine the general geographical location of a query image.<n>We propose an Efficient Decoder Transformer (EDTformer) for feature aggregation.<n>Our EDTformer can fully utilize the contextual information within deep features.
  arXiv  Detail & Related papers  (2024-12-01T12:14:36Z)
• MSDNet: Multi-Scale Decoder for Few-Shot Semantic Segmentation via Transformer-Guided Prototyping [1.1557852082644071]
  Few-shot Semantic addresses the challenge of segmenting objects in query images with only a handful of examples. We propose a new Few-shot Semantic framework based on the transformer architecture. Our model with only 1.5 million parameters demonstrates competitive performance while overcoming limitations of existing methodologies.
  arXiv  Detail & Related papers  (2024-09-17T16:14:03Z)
• Positional Prompt Tuning for Efficient 3D Representation Learning [16.25423192020736]
  Point cloud analysis has achieved significant development and is well-performed in multiple downstream tasks like point cloud classification and segmentation, etc. Being conscious of the simplicity of the position encoding structure in Transformer-based architectures, we attach importance to the position encoding as a high-dimensional part and the patch encoder to offer multi-scale information. Our Proposed method of PEFT tasks, namely, with only 1.05% of parameters for training, gets state-of-the-art results in several mainstream datasets, such as 95.01% accuracy in the ScanObjectNN OBJ_BG dataset.
  arXiv  Detail & Related papers  (2024-08-21T12:18:34Z)
• MacFormer: Semantic Segmentation with Fine Object Boundaries [38.430631361558426]
  We introduce a new semantic segmentation architecture, MacFormer'', which features two key components. Firstly, using learnable agent tokens, a Mutual Agent Cross-Attention (MACA) mechanism effectively facilitates the bidirectional integration of features across encoder and decoder layers. Secondly, a Frequency Enhancement Module (FEM) in the decoder leverages high-frequency and low-frequency components to boost features in the frequency domain. MacFormer is demonstrated to be compatible with various network architectures and outperforms existing methods in both accuracy and efficiency on datasets benchmark ADE20K and Cityscapes.
  arXiv  Detail & Related papers  (2024-08-11T05:36:10Z)
• Prompt-Matched Semantic Segmentation [96.99924127527002]
  The objective of this work is to explore how to effectively adapt pre-trained foundation models to various downstream tasks of image semantic segmentation. We propose a novel Inter-Stage Prompt-Matched Framework, which maintains the original structure of the foundation model while generating visual prompts adaptively for task-oriented tuning. A lightweight module termed Semantic-aware Prompt Matcher is then introduced to hierarchically interpolate between two stages to learn reasonable prompts for each specific task.
  arXiv  Detail & Related papers  (2022-08-22T09:12:53Z)
• Transformer-based Context Condensation for Boosting Feature Pyramids in Object Detection [77.50110439560152]
  Current object detectors typically have a feature pyramid (FP) module for multi-level feature fusion (MFF) We propose a novel and efficient context modeling mechanism that can help existing FPs deliver better MFF results. In particular, we introduce a novel insight that comprehensive contexts can be decomposed and condensed into two types of representations for higher efficiency.
  arXiv  Detail & Related papers  (2022-07-14T01:45:03Z)
• OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers [79.09486658392626]
  OSFormer is a one-stage transformer framework for camouflaged instance segmentation (CIS) We design a location-sensing transformer (LST) to obtain the location label and instance-aware parameters. We develop a coarse-to-fine fusion (CFF) to merge diverse context information from the LST encoder and CNN backbone.
  arXiv  Detail & Related papers  (2022-07-05T18:32:21Z)
• SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers [79.646577541655]
  We present SegFormer, a semantic segmentation framework which unifies Transformers with lightweight multilayer perception (MLP) decoders. SegFormer comprises a novelly structured encoder which outputs multiscale features. The proposed decoder aggregates information from different layers, and thus combining both local attention and global attention to powerful representations.
  arXiv  Detail & Related papers  (2021-05-31T17:59:51Z)
• Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers [149.78470371525754]
  We treat semantic segmentation as a sequence-to-sequence prediction task. Specifically, we deploy a pure transformer to encode an image as a sequence of patches. With the global context modeled in every layer of the transformer, this encoder can be combined with a simple decoder to provide a powerful segmentation model, termed SEgmentation TRansformer (SETR) SETR achieves new state of the art on ADE20K (50.28% mIoU), Pascal Context (55.83% mIoU) and competitive results on Cityscapes.
  arXiv  Detail & Related papers  (2020-12-31T18:55:57Z)
• CARAFE++: Unified Content-Aware ReAssembly of FEatures [132.49582482421246]
  We propose unified Content-Aware ReAssembly of FEatures (CARAFE++), a universal, lightweight and highly effective operator to fulfill this goal. CARAFE++ generates adaptive kernels on-the-fly to enable instance-specific content-aware handling. It shows consistent and substantial gains across all the tasks with negligible computational overhead.
  arXiv  Detail & Related papers  (2020-12-07T07:34:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.