Domain Invariant Masked Autoencoders for Self-supervised Learning from Multi-domains

• URL: http://arxiv.org/abs/2205.04771v1
• Date: Tue, 10 May 2022 09:49:40 GMT
• Title: Domain Invariant Masked Autoencoders for Self-supervised Learning from Multi-domains
• Authors: Haiyang Yang, Meilin Chen, Yizhou Wang, Shixiang Tang, Feng Zhu, Lei Bai, Rui Zhao, Wanli Ouyang
• Abstract summary: We propose a Domain-invariant Masked AutoEncoder (DiMAE) for self-supervised learning from multi-domains. The core idea is to augment the input image with style noise from different domains and then reconstruct the image from the embedding of the augmented image. Experiments on PACS and DomainNet illustrate that DiMAE achieves considerable gains compared with recent state-of-the-art methods.
• Score: 73.54897096088149
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Generalizing learned representations across significantly different visual domains is a fundamental yet crucial ability of the human visual system. While recent self-supervised learning methods have achieved good performances with evaluation set on the same domain as the training set, they will have an undesirable performance decrease when tested on a different domain. Therefore, the self-supervised learning from multiple domains task is proposed to learn domain-invariant features that are not only suitable for evaluation on the same domain as the training set but also can be generalized to unseen domains. In this paper, we propose a Domain-invariant Masked AutoEncoder (DiMAE) for self-supervised learning from multi-domains, which designs a new pretext task, \emph{i.e.,} the cross-domain reconstruction task, to learn domain-invariant features. The core idea is to augment the input image with style noise from different domains and then reconstruct the image from the embedding of the augmented image, regularizing the encoder to learn domain-invariant features. To accomplish the idea, DiMAE contains two critical designs, 1) content-preserved style mix, which adds style information from other domains to input while persevering the content in a parameter-free manner, and 2) multiple domain-specific decoders, which recovers the corresponding domain style of input to the encoded domain-invariant features for reconstruction. Experiments on PACS and DomainNet illustrate that DiMAE achieves considerable gains compared with recent state-of-the-art methods.

Related papers

• Domain-Scalable Unpaired Image Translation via Latent Space Anchoring [88.7642967393508]
  Unpaired image-to-image translation (UNIT) aims to map images between two visual domains without paired training data. We propose a new domain-scalable UNIT method, termed as latent space anchoring. Our method anchors images of different domains to the same latent space of frozen GANs by learning lightweight encoder and regressor models. In the inference phase, the learned encoders and decoders of different domains can be arbitrarily combined to translate images between any two domains without fine-tuning.
  arXiv  Detail & Related papers  (2023-06-26T17:50:02Z)
• Aggregation of Disentanglement: Reconsidering Domain Variations in Domain Generalization [9.577254317971933]
  We argue that the domain variantions also contain useful information, ie, classification-aware information, for downstream tasks. We propose a novel paradigm called Domain Disentanglement Network (DDN) to disentangle the domain expert features from the source domain images. We also propound a new contrastive learning method to guide the domain expert features to form a more balanced and separable feature space.
  arXiv  Detail & Related papers  (2023-02-05T09:48:57Z)
• Domain Generalization by Learning and Removing Domain-specific Features [15.061481139046952]
  Domain generalization aims to tackle this issue by learning a model that can generalize to unseen domains. We propose a new approach that aims to explicitly remove domain-specific features for domain generalization. We develop an encoder-decoder network to map each input image into a new image space where the learned domain-specific features are removed.
  arXiv  Detail & Related papers  (2022-12-14T08:46:46Z)
• PiPa: Pixel- and Patch-wise Self-supervised Learning for Domain Adaptative Semantic Segmentation [100.6343963798169]
  Unsupervised Domain Adaptation (UDA) aims to enhance the generalization of the learned model to other domains. We propose a unified pixel- and patch-wise self-supervised learning framework, called PiPa, for domain adaptive semantic segmentation.
  arXiv  Detail & Related papers  (2022-11-14T18:31:24Z)
• Dynamic Instance Domain Adaptation [109.53575039217094]
  Most studies on unsupervised domain adaptation assume that each domain's training samples come with domain labels. We develop a dynamic neural network with adaptive convolutional kernels to generate instance-adaptive residuals to adapt domain-agnostic deep features to each individual instance. Our model, dubbed DIDA-Net, achieves state-of-the-art performance on several commonly used single-source and multi-source UDA datasets.
  arXiv  Detail & Related papers  (2022-03-09T20:05:54Z)
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection [90.18752912204778]
  Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
  arXiv  Detail & Related papers  (2021-06-10T05:01:20Z)
• Boosting Binary Masks for Multi-Domain Learning through Affine Transformations [49.25451497933657]
  The goal of multi-domain learning is to produce a single model performing a task in all the domains together. Recent works showed how we can address this problem by masking the internal weights of a given original conv-net through learned binary variables. We provide a general formulation of binary mask based models for multi-domain learning by affine transformations of the original network parameters.
  arXiv  Detail & Related papers  (2021-03-25T14:54:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.