Context-Aware Domain Adaptation in Semantic Segmentation

• URL: http://arxiv.org/abs/2003.04010v1
• Date: Mon, 9 Mar 2020 09:57:24 GMT
• Title: Context-Aware Domain Adaptation in Semantic Segmentation
• Authors: Jinyu Yang, Weizhi An, Chaochao Yan, Peilin Zhao, Junzhou Huang
• Abstract summary: We consider the problem of unsupervised domain adaptation in the semantic segmentation. Existing methods mainly focus on adapting domain-invariant features (what to transfer) through adversarial learning. We propose a cross-attention mechanism based on self-attention to capture context dependencies between two domains and adapt transferable context.
• Score: 59.79348089240319
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we consider the problem of unsupervised domain adaptation in the semantic segmentation. There are two primary issues in this field, i.e., what and how to transfer domain knowledge across two domains. Existing methods mainly focus on adapting domain-invariant features (what to transfer) through adversarial learning (how to transfer). Context dependency is essential for semantic segmentation, however, its transferability is still not well understood. Furthermore, how to transfer contextual information across two domains remains unexplored. Motivated by this, we propose a cross-attention mechanism based on self-attention to capture context dependencies between two domains and adapt transferable context. To achieve this goal, we design two cross-domain attention modules to adapt context dependencies from both spatial and channel views. Specifically, the spatial attention module captures local feature dependencies between each position in the source and target image. The channel attention module models semantic dependencies between each pair of cross-domain channel maps. To adapt context dependencies, we further selectively aggregate the context information from two domains. The superiority of our method over existing state-of-the-art methods is empirically proved on "GTA5 to Cityscapes" and "SYNTHIA to Cityscapes".

Related papers

• Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval [55.122020263319634]
  Video moment retrieval (VMR) aims to localize the target moment from an untrimmed video according to a given language query. In this paper, we focus on a novel task: cross-domain VMR, where fully-annotated datasets are available in one domain but the domain of interest only contains unannotated datasets. We propose a novel Multi-Modal Cross-Domain Alignment network to transfer the annotation knowledge from the source domain to the target domain.
  arXiv  Detail & Related papers  (2022-09-23T12:58:20Z)
• Unsupervised Domain Adaptation for Point Cloud Semantic Segmentation via Graph Matching [14.876681993079062]
  We propose a graph-based framework to explore the local-level feature alignment between the two domains. We also formulate a category-guided contrastive loss to guide the segmentation model to learn discriminative features on the target domain.
  arXiv  Detail & Related papers  (2022-08-09T02:30:15Z)
• DecoupleNet: Decoupled Network for Domain Adaptive Semantic Segmentation [78.30720731968135]
  Unsupervised domain adaptation in semantic segmentation has been raised to alleviate the reliance on expensive pixel-wise annotations. We propose DecoupleNet that alleviates source domain overfitting and enables the final model to focus more on the segmentation task. We also put forward Self-Discrimination (SD) and introduce an auxiliary classifier to learn more discriminative target domain features with pseudo labels.
  arXiv  Detail & Related papers  (2022-07-20T15:47:34Z)
• DRANet: Disentangling Representation and Adaptation Networks for Unsupervised Cross-Domain Adaptation [23.588766224169493]
  DRANet is a network architecture that disentangles image representations and transfers the visual attributes in a latent space for unsupervised cross-domain adaptation. Our model encodes individual representations of content (scene structure) and style (artistic appearance) from both source and target images. It adapts the domain by incorporating the transferred style factor into the content factor along with learnable weights specified for each domain.
  arXiv  Detail & Related papers  (2021-03-24T18:54:23Z)
• Variational Interaction Information Maximization for Cross-domain Disentanglement [34.08140408283391]
  Cross-domain disentanglement is the problem of learning representations partitioned into domain-invariant and domain-specific representations. We cast the simultaneous learning of domain-invariant and domain-specific representations as a joint objective of multiple information constraints. We show that our model achieves the state-of-the-art performance in the zero-shot sketch based image retrieval task.
  arXiv  Detail & Related papers  (2020-12-08T07:11:35Z)
• Affinity Space Adaptation for Semantic Segmentation Across Domains [57.31113934195595]
  In this paper, we address the problem of unsupervised domain adaptation (UDA) in semantic segmentation. Motivated by the fact that source and target domain have invariant semantic structures, we propose to exploit such invariance across domains. We develop two affinity space adaptation strategies: affinity space cleaning and adversarial affinity space alignment.
  arXiv  Detail & Related papers  (2020-09-26T10:28:11Z)
• Domain Adaptation for Semantic Parsing [68.81787666086554]
  We propose a novel semantic for domain adaptation, where we have much fewer annotated data in the target domain compared to the source domain. Our semantic benefits from a two-stage coarse-to-fine framework, thus can provide different and accurate treatments for the two stages. Experiments on a benchmark dataset show that our method consistently outperforms several popular domain adaptation strategies.
  arXiv  Detail & Related papers  (2020-06-23T14:47:41Z)
• Differential Treatment for Stuff and Things: A Simple Unsupervised Domain Adaptation Method for Semantic Segmentation [105.96860932833759]
  State-of-the-art approaches prove that performing semantic-level alignment is helpful in tackling the domain shift issue. We propose to improve the semantic-level alignment with different strategies for stuff regions and for things. In addition to our proposed method, we show that our method can help ease this issue by minimizing the most similar stuff and instance features between the source and the target domains.
  arXiv  Detail & Related papers  (2020-03-18T04:43:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.