Semantic Connectivity-Driven Pseudo-labeling for Cross-domain
Segmentation
- URL: http://arxiv.org/abs/2312.06331v1
- Date: Mon, 11 Dec 2023 12:29:51 GMT
- Title: Semantic Connectivity-Driven Pseudo-labeling for Cross-domain
Segmentation
- Authors: Dong Zhao, Ruizhi Yang, Shuang Wang, Qi Zang, Yang Hu, Licheng Jiao,
Nicu Sebe, Zhun Zhong
- Abstract summary: Self-training is a prevailing approach in cross-domain semantic segmentation.
We propose a novel approach called Semantic Connectivity-driven pseudo-labeling.
This approach formulates pseudo-labels at the connectivity level and thus can facilitate learning structured and low-noise semantics.
- Score: 89.41179071022121
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Presently, self-training stands as a prevailing approach in cross-domain
semantic segmentation, enhancing model efficacy by training with pixels
assigned with reliable pseudo-labels. However, we find two critical limitations
in this paradigm. (1) The majority of reliable pixels exhibit a speckle-shaped
pattern and are primarily located in the central semantic region. This presents
challenges for the model in accurately learning semantics. (2) Category noise
in speckle pixels is difficult to locate and correct, leading to error
accumulation in self-training. To address these limitations, we propose a novel
approach called Semantic Connectivity-driven pseudo-labeling (SeCo). This
approach formulates pseudo-labels at the connectivity level and thus can
facilitate learning structured and low-noise semantics. Specifically, SeCo
comprises two key components: Pixel Semantic Aggregation (PSA) and Semantic
Connectivity Correction (SCC). Initially, PSA divides semantics into 'stuff'
and 'things' categories and aggregates speckled pseudo-labels into semantic
connectivity through efficient interaction with the Segment Anything Model
(SAM). This enables us not only to obtain accurate boundaries but also
simplifies noise localization. Subsequently, SCC introduces a simple
connectivity classification task, which enables locating and correcting
connectivity noise with the guidance of loss distribution. Extensive
experiments demonstrate that SeCo can be flexibly applied to various
cross-domain semantic segmentation tasks, including traditional unsupervised,
source-free, and black-box domain adaptation, significantly improving the
performance of existing state-of-the-art methods. The code is available at
https://github.com/DZhaoXd/SeCo.
Related papers
- Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised
Semantic Segmentation [79.05949524349005]
We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from saliency maps.
We also propose a cross-task affinity learning mechanism to learn pixel-level affinities from the saliency and segmentation feature maps.
arXiv Detail & Related papers (2024-03-02T10:03:21Z) - Dual-level Interaction for Domain Adaptive Semantic Segmentation [0.0]
We propose a dual-level interaction for domain adaptation (DIDA) in semantic segmentation.
Explicitly, we encourage the different augmented views of the same pixel to have similar class prediction.
Our method outperforms the state-of-the-art by a notable margin, especially on confusing and long-tailed classes.
arXiv Detail & Related papers (2023-07-16T07:51:18Z) - Regularizing Self-training for Unsupervised Domain Adaptation via
Structural Constraints [14.593782939242121]
We propose to incorporate structural cues from auxiliary modalities, such as depth, to regularise conventional self-training objectives.
Specifically, we introduce a contrastive pixel-level objectness constraint that pulls the pixel representations within a region of an object instance closer.
We show that our regularizer significantly improves top performing self-training methods in various UDA benchmarks for semantic segmentation.
arXiv Detail & Related papers (2023-04-29T00:12:26Z) - SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic
Segmentation [52.62441404064957]
Domain adaptive semantic segmentation attempts to make satisfactory dense predictions on an unlabeled target domain by utilizing the model trained on a labeled source domain.
Many methods tend to alleviate noisy pseudo labels, however, they ignore intrinsic connections among cross-domain pixels with similar semantic concepts.
We propose Semantic-Guided Pixel Contrast (SePiCo), a novel one-stage adaptation framework that highlights the semantic concepts of individual pixel.
arXiv Detail & Related papers (2022-04-19T11:16:29Z) - Exploiting Inter-pixel Correlations in Unsupervised Domain Adaptation
for Semantic Segmentation [32.71144601174794]
"Self-training" has become a dominant method for semantic segmentation via unsupervised domain adaptation (UDA)
We propose a method of transferring the inter-pixel correlations from the source domain to the target domain via a self-attention module.
arXiv Detail & Related papers (2021-10-21T06:11:44Z) - Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised
Semantic Segmentation [88.49669148290306]
We propose a novel weakly supervised multi-task framework called AuxSegNet to leverage saliency detection and multi-label image classification as auxiliary tasks.
Inspired by their similar structured semantics, we also propose to learn a cross-task global pixel-level affinity map from the saliency and segmentation representations.
The learned cross-task affinity can be used to refine saliency predictions and propagate CAM maps to provide improved pseudo labels for both tasks.
arXiv Detail & Related papers (2021-07-25T11:39:58Z) - More Separable and Easier to Segment: A Cluster Alignment Method for
Cross-Domain Semantic Segmentation [41.81843755299211]
We propose a new UDA semantic segmentation approach based on domain assumption closeness to alleviate the above problems.
Specifically, a prototype clustering strategy is applied to cluster pixels with the same semantic, which will better maintain associations among target domain pixels.
Experiments conducted on GTA5 and SYNTHIA proved the effectiveness of our method.
arXiv Detail & Related papers (2021-05-07T10:24:18Z) - Contrastive Learning and Self-Training for Unsupervised Domain
Adaptation in Semantic Segmentation [71.77083272602525]
UDA attempts to provide efficient knowledge transfer from a labeled source domain to an unlabeled target domain.
We propose a contrastive learning approach that adapts category-wise centroids across domains.
We extend our method with self-training, where we use a memory-efficient temporal ensemble to generate consistent and reliable pseudo-labels.
arXiv Detail & Related papers (2021-05-05T11:55:53Z) - Affinity Space Adaptation for Semantic Segmentation Across Domains [57.31113934195595]
In this paper, we address the problem of unsupervised domain adaptation (UDA) in semantic segmentation.
Motivated by the fact that source and target domain have invariant semantic structures, we propose to exploit such invariance across domains.
We develop two affinity space adaptation strategies: affinity space cleaning and adversarial affinity space alignment.
arXiv Detail & Related papers (2020-09-26T10:28:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.