SSA-Seg: Semantic and Spatial Adaptive Pixel-level Classifier for Semantic Segmentation

• URL: http://arxiv.org/abs/2405.06525v2
• Date: Fri, 25 Oct 2024 09:40:47 GMT
• Title: SSA-Seg: Semantic and Spatial Adaptive Pixel-level Classifier for Semantic Segmentation
• Authors: Xiaowen Ma, Zhenliang Ni, Xinghao Chen,
• Abstract summary: In this paper, we propose a novel Semantic and Spatial Adaptive (SSA-Seg) to address the challenges of semantic segmentation. Specifically, we employ the coarse masks obtained from the fixed prototypes as a guide to adjust the fixed prototype towards the center of the semantic and spatial domains in the test image. Results show that the proposed SSA-Seg significantly improves the segmentation performance of the baseline models with only a minimal increase in computational cost.
• Score: 11.176993272867396
• License:
• Abstract: Vanilla pixel-level classifiers for semantic segmentation are based on a certain paradigm, involving the inner product of fixed prototypes obtained from the training set and pixel features in the test image. This approach, however, encounters significant limitations, \ie, feature deviation in the semantic domain and information loss in the spatial domain. The former struggles with large intra-class variance among pixel features from different images, while the latter fails to utilize the structured information of semantic objects effectively. This leads to blurred mask boundaries as well as a deficiency of fine-grained recognition capability. In this paper, we propose a novel Semantic and Spatial Adaptive Classifier (SSA-Seg) to address the above challenges. Specifically, we employ the coarse masks obtained from the fixed prototypes as a guide to adjust the fixed prototype towards the center of the semantic and spatial domains in the test image. The adapted prototypes in semantic and spatial domains are then simultaneously considered to accomplish classification decisions. In addition, we propose an online multi-domain distillation learning strategy to improve the adaption process. Experimental results on three publicly available benchmarks show that the proposed SSA-Seg significantly improves the segmentation performance of the baseline models with only a minimal increase in computational cost. Code is available at https://github.com/xwmaxwma/SSA-Seg.

Related papers

• Pixel-Level Domain Adaptation: A New Perspective for Enhancing Weakly Supervised Semantic Segmentation [13.948425538725138]
  We propose a Pixel-Level Domain Adaptation (PLDA) method to encourage the model in learning pixel-wise domain-invariant features. We experimentally demonstrate the effectiveness of our approach under a wide range of settings.
  arXiv  Detail & Related papers  (2024-08-04T14:14:54Z)
• Location-Aware Self-Supervised Transformers [74.76585889813207]
  We propose to pretrain networks for semantic segmentation by predicting the relative location of image parts. We control the difficulty of the task by masking a subset of the reference patch features visible to those of the query. Our experiments show that this location-aware pretraining leads to representations that transfer competitively to several challenging semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2022-12-05T16:24:29Z)
• SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentation [52.62441404064957]
  Domain adaptive semantic segmentation attempts to make satisfactory dense predictions on an unlabeled target domain by utilizing the model trained on a labeled source domain. Many methods tend to alleviate noisy pseudo labels, however, they ignore intrinsic connections among cross-domain pixels with similar semantic concepts. We propose Semantic-Guided Pixel Contrast (SePiCo), a novel one-stage adaptation framework that highlights the semantic concepts of individual pixel.
  arXiv  Detail & Related papers  (2022-04-19T11:16:29Z)
• AF$_2$: Adaptive Focus Framework for Aerial Imagery Segmentation [86.44683367028914]
  Aerial imagery segmentation has some unique challenges, the most critical one among which lies in foreground-background imbalance. We propose Adaptive Focus Framework (AF$), which adopts a hierarchical segmentation procedure and focuses on adaptively utilizing multi-scale representations. AF$ has significantly improved the accuracy on three widely used aerial benchmarks, as fast as the mainstream method.
  arXiv  Detail & Related papers  (2022-02-18T10:14:45Z)
• SPCL: A New Framework for Domain Adaptive Semantic Segmentation via Semantic Prototype-based Contrastive Learning [6.705297811617307]
  Domain adaptation can help in transferring knowledge from a labeled source domain to an unlabeled target domain. We propose a novel semantic prototype-based contrastive learning framework for fine-grained class alignment. Our method is easy to implement and attains superior results compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-24T09:26:07Z)
• Semantic Distribution-aware Contrastive Adaptation for Semantic Segmentation [50.621269117524925]
  Domain adaptive semantic segmentation refers to making predictions on a certain target domain with only annotations of a specific source domain. We present a semantic distribution-aware contrastive adaptation algorithm that enables pixel-wise representation alignment. We evaluate SDCA on multiple benchmarks, achieving considerable improvements over existing algorithms.
  arXiv  Detail & Related papers  (2021-05-11T13:21:25Z)
• Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation [169.82760468633236]
  We propose to build the pixel-level cycle association between source and target pixel pairs. Our method can be trained end-to-end in one stage and introduces no additional parameters.
  arXiv  Detail & Related papers  (2020-10-31T00:11:36Z)
• Affinity Space Adaptation for Semantic Segmentation Across Domains [57.31113934195595]
  In this paper, we address the problem of unsupervised domain adaptation (UDA) in semantic segmentation. Motivated by the fact that source and target domain have invariant semantic structures, we propose to exploit such invariance across domains. We develop two affinity space adaptation strategies: affinity space cleaning and adversarial affinity space alignment.
  arXiv  Detail & Related papers  (2020-09-26T10:28:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.