PiPa: Pixel- and Patch-wise Self-supervised Learning for Domain
Adaptative Semantic Segmentation
- URL: http://arxiv.org/abs/2211.07609v1
- Date: Mon, 14 Nov 2022 18:31:24 GMT
- Title: PiPa: Pixel- and Patch-wise Self-supervised Learning for Domain
Adaptative Semantic Segmentation
- Authors: Mu Chen, Zhedong Zheng, Yi Yang, Tat-Seng Chua
- Abstract summary: Unsupervised Domain Adaptation (UDA) aims to enhance the generalization of the learned model to other domains.
We propose a unified pixel- and patch-wise self-supervised learning framework, called PiPa, for domain adaptive semantic segmentation.
- Score: 100.6343963798169
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Unsupervised Domain Adaptation (UDA) aims to enhance the generalization of
the learned model to other domains. The domain-invariant knowledge is
transferred from the model trained on labeled source domain, e.g., video game,
to unlabeled target domains, e.g., real-world scenarios, saving annotation
expenses. Existing UDA methods for semantic segmentation usually focus on
minimizing the inter-domain discrepancy of various levels, e.g., pixels,
features, and predictions, for extracting domain-invariant knowledge. However,
the primary intra-domain knowledge, such as context correlation inside an
image, remains underexplored. In an attempt to fill this gap, we propose a
unified pixel- and patch-wise self-supervised learning framework, called PiPa,
for domain adaptive semantic segmentation that facilitates intra-image
pixel-wise correlations and patch-wise semantic consistency against different
contexts. The proposed framework exploits the inherent structures of
intra-domain images, which: (1) explicitly encourages learning the
discriminative pixel-wise features with intra-class compactness and inter-class
separability, and (2) motivates the robust feature learning of the identical
patch against different contexts or fluctuations. Extensive experiments verify
the effectiveness of the proposed method, which obtains competitive accuracy on
the two widely-used UDA benchmarks, i.e., 75.6 mIoU on GTA to Cityscapes and
68.2 mIoU on Synthia to Cityscapes. Moreover, our method is compatible with
other UDA approaches to further improve the performance without introducing
extra parameters.
Related papers
- PiPa++: Towards Unification of Domain Adaptive Semantic Segmentation via Self-supervised Learning [34.786268652516355]
Unsupervised domain adaptive segmentation aims to improve the segmentation accuracy of models on target domains without relying on labeled data from those domains.
It seeks to align the feature representations of the source domain (where labeled data is available) and the target domain (where only unlabeled data is present)
arXiv Detail & Related papers (2024-07-24T08:53:29Z) - Unsupervised Domain Adaptation for Semantic Segmentation using One-shot
Image-to-Image Translation via Latent Representation Mixing [9.118706387430883]
We propose a new unsupervised domain adaptation method for the semantic segmentation of very high resolution images.
An image-to-image translation paradigm is proposed, based on an encoder-decoder principle where latent content representations are mixed across domains.
Cross-city comparative experiments have shown that the proposed method outperforms state-of-the-art domain adaptation methods.
arXiv Detail & Related papers (2022-12-07T18:16:17Z) - Exploiting Inter-pixel Correlations in Unsupervised Domain Adaptation
for Semantic Segmentation [32.71144601174794]
"Self-training" has become a dominant method for semantic segmentation via unsupervised domain adaptation (UDA)
We propose a method of transferring the inter-pixel correlations from the source domain to the target domain via a self-attention module.
arXiv Detail & Related papers (2021-10-21T06:11:44Z) - A New Bidirectional Unsupervised Domain Adaptation Segmentation
Framework [27.13101555533594]
unsupervised domain adaptation (UDA) techniques are proposed to bridge the gap between different domains.
In this paper, we propose a bidirectional UDA framework based on disentangled representation learning for equally competent two-way UDA performances.
arXiv Detail & Related papers (2021-08-18T05:25:11Z) - AFAN: Augmented Feature Alignment Network for Cross-Domain Object
Detection [90.18752912204778]
Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications.
We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training.
Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
arXiv Detail & Related papers (2021-06-10T05:01:20Z) - Margin Preserving Self-paced Contrastive Learning Towards Domain
Adaptation for Medical Image Segmentation [51.93711960601973]
We propose a novel margin preserving self-paced contrastive Learning model for cross-modal medical image segmentation.
With the guidance of progressively refined semantic prototypes, a novel margin preserving contrastive loss is proposed to boost the discriminability of embedded representation space.
Experiments on cross-modal cardiac segmentation tasks demonstrate that MPSCL significantly improves semantic segmentation performance.
arXiv Detail & Related papers (2021-03-15T15:23:10Z) - Pixel-Level Cycle Association: A New Perspective for Domain Adaptive
Semantic Segmentation [169.82760468633236]
We propose to build the pixel-level cycle association between source and target pixel pairs.
Our method can be trained end-to-end in one stage and introduces no additional parameters.
arXiv Detail & Related papers (2020-10-31T00:11:36Z) - Affinity Space Adaptation for Semantic Segmentation Across Domains [57.31113934195595]
In this paper, we address the problem of unsupervised domain adaptation (UDA) in semantic segmentation.
Motivated by the fact that source and target domain have invariant semantic structures, we propose to exploit such invariance across domains.
We develop two affinity space adaptation strategies: affinity space cleaning and adversarial affinity space alignment.
arXiv Detail & Related papers (2020-09-26T10:28:11Z) - Adaptively-Accumulated Knowledge Transfer for Partial Domain Adaptation [66.74638960925854]
Partial domain adaptation (PDA) deals with a realistic and challenging problem when the source domain label space substitutes the target domain.
We propose an Adaptively-Accumulated Knowledge Transfer framework (A$2$KT) to align the relevant categories across two domains.
arXiv Detail & Related papers (2020-08-27T00:53:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.