MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation

• URL: http://arxiv.org/abs/2212.01322v2
• Date: Fri, 24 Mar 2023 15:26:35 GMT
• Title: MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation
• Authors: Lukas Hoyer, Dengxin Dai, Haoran Wang, Luc Van Gool
• Abstract summary: In unsupervised domain adaptation (UDA), a model trained on source data (e.g. synthetic) is adapted to target data (e.g. real-world) without access to target annotation. We propose a Masked Image Consistency (MIC) module to enhance UDA by learning spatial context relations of the target domain. MIC significantly improves the state-of-the-art performance across the different recognition tasks for synthetic-to-real, day-to-nighttime, and clear-to-adverse-weather UDA.
• Score: 104.40114562948428
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In unsupervised domain adaptation (UDA), a model trained on source data (e.g. synthetic) is adapted to target data (e.g. real-world) without access to target annotation. Most previous UDA methods struggle with classes that have a similar visual appearance on the target domain as no ground truth is available to learn the slight appearance differences. To address this problem, we propose a Masked Image Consistency (MIC) module to enhance UDA by learning spatial context relations of the target domain as additional clues for robust visual recognition. MIC enforces the consistency between predictions of masked target images, where random patches are withheld, and pseudo-labels that are generated based on the complete image by an exponential moving average teacher. To minimize the consistency loss, the network has to learn to infer the predictions of the masked regions from their context. Due to its simple and universal concept, MIC can be integrated into various UDA methods across different visual recognition tasks such as image classification, semantic segmentation, and object detection. MIC significantly improves the state-of-the-art performance across the different recognition tasks for synthetic-to-real, day-to-nighttime, and clear-to-adverse-weather UDA. For instance, MIC achieves an unprecedented UDA performance of 75.9 mIoU and 92.8% on GTA-to-Cityscapes and VisDA-2017, respectively, which corresponds to an improvement of +2.1 and +3.0 percent points over the previous state of the art. The implementation is available at https://github.com/lhoyer/MIC.

Related papers

• SiamSeg: Self-Training with Contrastive Learning for Unsupervised Domain Adaptation Semantic Segmentation in Remote Sensing [14.007392647145448]
  UDA enables models to learn from unlabeled target domain data while training on labeled source domain data. We propose integrating contrastive learning into UDA, enhancing the model's capacity to capture semantic information. Our SimSeg method outperforms existing approaches, achieving state-of-the-art results.
  arXiv  Detail & Related papers  (2024-10-17T11:59:39Z)
• C^2DA: Contrastive and Context-aware Domain Adaptive Semantic Segmentation [11.721696305235767]
  Unsupervised domain adaptive semantic segmentation (UDA-SS) aims to train a model on the source domain data and adapt the model to predict target domain data. Most existing UDA-SS methods only focus on inter-domain knowledge to mitigate the data-shift problem. We propose a UDA-SS framework that learns both intra-domain and context-aware knowledge.
  arXiv  Detail & Related papers  (2024-10-10T15:51:35Z)
• MICDrop: Masking Image and Depth Features via Complementary Dropout for Domain-Adaptive Semantic Segmentation [155.0797148367653]
  Unsupervised Domain Adaptation (UDA) is the task of bridging the domain gap between a labeled source domain and an unlabeled target domain. We propose to leverage geometric information, i.e., depth predictions, as depth discontinuities often coincide with segmentation boundaries. We show that our method can be plugged into various recent UDA methods and consistently improve results across standard UDA benchmarks.
  arXiv  Detail & Related papers  (2024-08-29T12:15:10Z)
• Adaptive Face Recognition Using Adversarial Information Network [57.29464116557734]
  Face recognition models often degenerate when training data are different from testing data. We propose a novel adversarial information network (AIN) to address it.
  arXiv  Detail & Related papers  (2023-05-23T02:14:11Z)
• Focus on Your Target: A Dual Teacher-Student Framework for Domain-adaptive Semantic Segmentation [210.46684938698485]
  We study unsupervised domain adaptation (UDA) for semantic segmentation. We find that, by decreasing/increasing the proportion of training samples from the target domain, the 'learning ability' is strengthened/weakened. We propose a novel dual teacher-student (DTS) framework and equip it with a bidirectional learning strategy.
  arXiv  Detail & Related papers  (2023-03-16T05:04:10Z)
• PiPa: Pixel- and Patch-wise Self-supervised Learning for Domain Adaptative Semantic Segmentation [100.6343963798169]
  Unsupervised Domain Adaptation (UDA) aims to enhance the generalization of the learned model to other domains. We propose a unified pixel- and patch-wise self-supervised learning framework, called PiPa, for domain adaptive semantic segmentation.
  arXiv  Detail & Related papers  (2022-11-14T18:31:24Z)
• CLUDA : Contrastive Learning in Unsupervised Domain Adaptation for Semantic Segmentation [3.4123736336071864]
  CLUDA is a simple, yet novel method for performing unsupervised domain adaptation (UDA) for semantic segmentation. We extract a multi-level fused-feature map from the encoder, and apply contrastive loss across different classes and different domains. We produce state-of-the-art results on GTA $rightarrow$ Cityscapes (74.4 mIOU, +0.6) and Synthia $rightarrow$ Cityscapes (67.2 mIOU, +1.4) datasets.
  arXiv  Detail & Related papers  (2022-08-27T05:13:14Z)
• Semi-Supervised Domain Adaptation with Prototypical Alignment and Consistency Learning [86.6929930921905]
  This paper studies how much it can help address domain shifts if we further have a few target samples labeled. To explore the full potential of landmarks, we incorporate a prototypical alignment (PA) module which calculates a target prototype for each class from the landmarks. Specifically, we severely perturb the labeled images, making PA non-trivial to achieve and thus promoting model generalizability.
  arXiv  Detail & Related papers  (2021-04-19T08:46:08Z)
• Continual Unsupervised Domain Adaptation for Semantic Segmentation [14.160280479726921]
  Unsupervised Domain Adaptation (UDA) for semantic segmentation has been favorably applied to real-world scenarios in which pixel-level labels are hard to be obtained. We propose Continual UDA for semantic segmentation based on a newly designed Expanding Target-specific Memory (ETM) framework. Our novel ETM framework contains Target-specific Memory (TM) for each target domain to alleviate catastrophic forgetting.
  arXiv  Detail & Related papers  (2020-10-19T05:59:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.