Semantic Segmentation via Pixel-to-Center Similarity Calculation
- URL: http://arxiv.org/abs/2301.04870v1
- Date: Thu, 12 Jan 2023 08:36:59 GMT
- Title: Semantic Segmentation via Pixel-to-Center Similarity Calculation
- Authors: Dongyue Wu, Zilin Guo, Aoyan Li, Changqian Yu, Changxin Gao, Nong Sang
- Abstract summary: We first rethink semantic segmentation from a perspective of similarity between pixels and class centers.
Under this novel view, we propose a Class Center Similarity layer (CCS layer) to address the above-mentioned challenges.
Our model performs favourably against the state-of-the-art CNN-based methods.
- Score: 40.62804702162577
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Since the fully convolutional network has achieved great success in semantic
segmentation, lots of works have been proposed focusing on extracting
discriminative pixel feature representations. However, we observe that existing
methods still suffer from two typical challenges, i.e. (i) large intra-class
feature variation in different scenes, (ii) small inter-class feature
distinction in the same scene. In this paper, we first rethink semantic
segmentation from a perspective of similarity between pixels and class centers.
Each weight vector of the segmentation head represents its corresponding
semantic class in the whole dataset, which can be regarded as the embedding of
the class center. Thus, the pixel-wise classification amounts to computing
similarity in the final feature space between pixels and the class centers.
Under this novel view, we propose a Class Center Similarity layer (CCS layer)
to address the above-mentioned challenges by generating adaptive class centers
conditioned on different scenes and supervising the similarities between class
centers. It utilizes a Adaptive Class Center Module (ACCM) to generate class
centers conditioned on each scene, which adapt the large intra-class variation
between different scenes. Specially designed loss functions are introduced to
control both inter-class and intra-class distances based on predicted
center-to-center and pixel-to-center similarity, respectively. Finally, the CCS
layer outputs the processed pixel-to-center similarity as the segmentation
prediction. Extensive experiments demonstrate that our model performs
favourably against the state-of-the-art CNN-based methods.
Related papers
- Center-guided Classifier for Semantic Segmentation of Remote Sensing Images [2.803715177543843]
CenterSeg is a novel classifier for semantic segmentation of remote sensing images.
It solves problems with multiple prototypes, direct supervision under Grassmann manifold, and interpretability strategy.
Besides the superior performance, CenterSeg has the advantages of simplicity, lightweight, compatibility, and interpretability.
arXiv Detail & Related papers (2025-03-21T09:21:37Z) - Superpixel Graph Contrastive Clustering with Semantic-Invariant
Augmentations for Hyperspectral Images [64.72242126879503]
Hyperspectral images (HSI) clustering is an important but challenging task.
We first use 3-D and 2-D hybrid convolutional neural networks to extract the high-order spatial and spectral features of HSI.
We then design a superpixel graph contrastive clustering model to learn discriminative superpixel representations.
arXiv Detail & Related papers (2024-03-04T07:40:55Z) - Feature Selection using Sparse Adaptive Bottleneck Centroid-Encoder [1.2487990897680423]
We introduce a novel nonlinear model, Sparse Adaptive Bottleneckid-Encoder (SABCE), for determining the features that discriminate between two or more classes.
The algorithm is applied to various real-world data sets, including high-dimensional biological, image, speech, and accelerometer sensor data.
arXiv Detail & Related papers (2023-06-07T21:37:21Z) - Visual Recognition with Deep Nearest Centroids [57.35144702563746]
We devise deep nearest centroids (DNC), a conceptually elegant yet surprisingly effective network for large-scale visual recognition.
Compared with parametric counterparts, DNC performs better on image classification (CIFAR-10, ImageNet) and greatly boots pixel recognition (ADE20K, Cityscapes)
arXiv Detail & Related papers (2022-09-15T15:47:31Z) - Active Domain Adaptation with Multi-level Contrastive Units for Semantic
Segmentation [22.048328293739182]
We propose a novel Active Domain Adaptation scheme with Multi-level Contrastive Units (ADA-MCU) for semantic image segmentation.
ADA-MCU is constructed from intra-image, cross-image, and cross-domain levels by using both labeled and unlabeled pixels.
We show that the proposed method achieves competitive performance against state-of-the-art SSDA methods with 50% fewer labeled pixels and significantly outperforms state-of-the-art with a large margin by using the same level of annotation cost.
arXiv Detail & Related papers (2022-05-23T10:55:39Z) - SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic
Segmentation [52.62441404064957]
Domain adaptive semantic segmentation attempts to make satisfactory dense predictions on an unlabeled target domain by utilizing the model trained on a labeled source domain.
Many methods tend to alleviate noisy pseudo labels, however, they ignore intrinsic connections among cross-domain pixels with similar semantic concepts.
We propose Semantic-Guided Pixel Contrast (SePiCo), a novel one-stage adaptation framework that highlights the semantic concepts of individual pixel.
arXiv Detail & Related papers (2022-04-19T11:16:29Z) - The Fixed Sub-Center: A Better Way to Capture Data Complexity [1.583842747998493]
We propose to use Fixed Sub-Center (F-SC) to create more discrepant sub-centers.
The experimental results show that F-SC significantly improves the accuracy of both image classification and fine-grained recognition tasks.
arXiv Detail & Related papers (2022-03-24T08:21:28Z) - Semantic Distribution-aware Contrastive Adaptation for Semantic
Segmentation [50.621269117524925]
Domain adaptive semantic segmentation refers to making predictions on a certain target domain with only annotations of a specific source domain.
We present a semantic distribution-aware contrastive adaptation algorithm that enables pixel-wise representation alignment.
We evaluate SDCA on multiple benchmarks, achieving considerable improvements over existing algorithms.
arXiv Detail & Related papers (2021-05-11T13:21:25Z) - PiCIE: Unsupervised Semantic Segmentation using Invariance and
Equivariance in Clustering [41.39557700996688]
We present a new framework for semantic segmentation without annotations via clustering.
We extend clustering from images to pixels and assign separate cluster membership to different instances within each image.
With our novel learning objective, our framework can learn high-level semantic concepts.
arXiv Detail & Related papers (2021-03-30T00:12:10Z) - Inter-Image Communication for Weakly Supervised Localization [77.2171924626778]
Weakly supervised localization aims at finding target object regions using only image-level supervision.
We propose to leverage pixel-level similarities across different objects for learning more accurate object locations.
Our method achieves the Top-1 localization error rate of 45.17% on the ILSVRC validation set.
arXiv Detail & Related papers (2020-08-12T04:14:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.