A Decoding Scheme with Successive Aggregation of Multi-Level Features for Light-Weight Semantic Segmentation

• URL: http://arxiv.org/abs/2402.11201v2
• Date: Fri, 14 Jun 2024 06:48:24 GMT
• Title: A Decoding Scheme with Successive Aggregation of Multi-Level Features for Light-Weight Semantic Segmentation
• Authors: Jiwon Yoo, Jangwon Lee, Gyeonghwan Kim,
• Abstract summary: We propose a novel decoding scheme for semantic segmentation. It takes multi-level features from the encoder with multi-scale architecture. It aims to achieve not only reduced computational expense but also higher segmentation accuracy.
• Score: 4.454210876879237
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-scale architecture, including hierarchical vision transformer, has been commonly applied to high-resolution semantic segmentation to deal with computational complexity with minimum performance loss. In this paper, we propose a novel decoding scheme for semantic segmentation in this regard, which takes multi-level features from the encoder with multi-scale architecture. The decoding scheme based on a multi-level vision transformer aims to achieve not only reduced computational expense but also higher segmentation accuracy, by introducing successive cross-attention in aggregation of the multi-level features. Furthermore, a way to enhance the multi-level features by the aggregated semantics is proposed. The effort is focused on maintaining the contextual consistency from the perspective of attention allocation and brings improved performance with significantly lower computational cost. Set of experiments on popular datasets demonstrates superiority of the proposed scheme to the state-of-the-art semantic segmentation models in terms of computational cost without loss of accuracy, and extensive ablation studies prove the effectiveness of ideas proposed.

Related papers

• Segformer++: Efficient Token-Merging Strategies for High-Resolution Semantic Segmentation [12.249546377051438]
  token merging has exhibited remarkable enhancements in inference speed, training efficiency, and memory utilization for image classification tasks. This paper facilitates the deployment of transformer-based architectures on resource-constrained devices and in real-time applications.
  arXiv  Detail & Related papers  (2024-05-23T11:54:27Z)
• Generalizable Entity Grounding via Assistance of Large Language Model [77.07759442298666]
  We propose a novel approach to densely ground visual entities from a long caption. We leverage a large multimodal model to extract semantic nouns, a class-a segmentation model to generate entity-level segmentation, and a multi-modal feature fusion module to associate each semantic noun with its corresponding segmentation mask.
  arXiv  Detail & Related papers  (2024-02-04T16:06:05Z)
• Category Feature Transformer for Semantic Segmentation [34.812688388968525]
  CFT learns unified feature embeddings for individual semantic categories from high-level features during each aggregation process. We conduct extensive experiments on popular semantic segmentation benchmarks. The proposed CFT obtains a compelling 55.1% mIoU with greatly reduced model parameters and computations on the challenging ADE20K dataset.
  arXiv  Detail & Related papers  (2023-08-10T13:44:54Z)
• Semantics-Aware Dynamic Localization and Refinement for Referring Image Segmentation [102.25240608024063]
  Referring image segments an image from a language expression. We develop an algorithm that shifts from being localization-centric to segmentation-language. Compared to its counterparts, our method is more versatile yet effective.
  arXiv  Detail & Related papers  (2023-03-11T08:42:40Z)
• Multi-scale and Cross-scale Contrastive Learning for Semantic Segmentation [5.281694565226513]
  We apply contrastive learning to enhance the discriminative power of the multi-scale features extracted by semantic segmentation networks. By first mapping the encoder's multi-scale representations to a common feature space, we instantiate a novel form of supervised local-global constraint.
  arXiv  Detail & Related papers  (2022-03-25T01:24:24Z)
• Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization [76.68866368409216]
  We propose learning to dynamically select discretization tightness conditioned on inputs. We show that dynamically varying tightness in communication bottlenecks can improve model performance on visual reasoning and reinforcement learning tasks.
  arXiv  Detail & Related papers  (2022-02-02T23:54:26Z)
• Generalizing Interactive Backpropagating Refinement for Dense Prediction [0.0]
  We introduce a set of G-BRS layers that enable both global and localized refinement for a range of dense prediction tasks. Our method can successfully generalize and significantly improve performance of existing pretrained state-of-the-art models with only a few clicks.
  arXiv  Detail & Related papers  (2021-12-21T03:52:08Z)
• Adversarial Feature Augmentation and Normalization for Visual Recognition [109.6834687220478]
  Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models. Here, we present an effective and efficient alternative that advocates adversarial augmentation on intermediate feature embeddings. We validate the proposed approach across diverse visual recognition tasks with representative backbone networks.
  arXiv  Detail & Related papers  (2021-03-22T20:36:34Z)
• A Holistically-Guided Decoder for Deep Representation Learning with Applications to Semantic Segmentation and Object Detection [74.88284082187462]
  One common strategy is to adopt dilated convolutions in the backbone networks to extract high-resolution feature maps. We propose one novel holistically-guided decoder which is introduced to obtain the high-resolution semantic-rich feature maps.
  arXiv  Detail & Related papers  (2020-12-18T10:51:49Z)
• BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation [118.46210049742993]
  We propose an efficient and effective architecture with a good trade-off between speed and accuracy, termed Bilateral spatial Network (BiSeNet V2) For a 2,048x1, input, we achieve 72.6% Mean IoU on the Cityscapes test set with a speed of 156 FPS on one NVIDIA GeForce 1080 Ti card, which is significantly faster than existing methods, yet we achieve better segmentation accuracy.
  arXiv  Detail & Related papers  (2020-04-05T10:26:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.