A Broad Study of Pre-training for Domain Generalization and Adaptation

• URL: http://arxiv.org/abs/2203.11819v1
• Date: Tue, 22 Mar 2022 15:38:36 GMT
• Title: A Broad Study of Pre-training for Domain Generalization and Adaptation
• Authors: Donghyun Kim, Kaihong Wang, Stan Sclaroff, Kate Saenko
• Abstract summary: We provide a broad study and in-depth analysis of pre-training for domain adaptation and generalization. We observe that simply using a state-of-the-art backbone outperforms existing state-of-the-art domain adaptation baselines.
• Score: 69.38359595534807
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep models must learn robust and transferable representations in order to perform well on new domains. While domain transfer methods (e.g., domain adaptation, domain generalization) have been proposed to learn transferable representations across domains, they are typically applied to ResNet backbones pre-trained on ImageNet. Thus, existing works pay little attention to the effects of pre-training on domain transfer tasks. In this paper, we provide a broad study and in-depth analysis of pre-training for domain adaptation and generalization, namely: network architectures, size, pre-training loss, and datasets. We observe that simply using a state-of-the-art backbone outperforms existing state-of-the-art domain adaptation baselines and set new baselines on Office-Home and DomainNet improving by 10.7\% and 5.5\%. We hope that this work can provide more insights for future domain transfer research.

Related papers

• Adapting to Distribution Shift by Visual Domain Prompt Generation [34.19066857066073]
  We adapt a model at test-time using a few unlabeled data to address distribution shifts. We build a knowledge bank to learn the transferable knowledge from source domains. The proposed method outperforms previous work on 5 large-scale benchmarks including WILDS and DomainNet.
  arXiv  Detail & Related papers  (2024-05-05T02:44:04Z)
• DomainVerse: A Benchmark Towards Real-World Distribution Shifts For Tuning-Free Adaptive Domain Generalization [27.099706316752254]
  We establish a novel dataset DomainVerse for Adaptive Domain Generalization (ADG) Benefiting from the introduced hierarchical definition of domain shifts, DomainVerse consists of about 0.5 million images from 390 fine-grained realistic domains. We propose two methods called Domain CLIP and Domain++ CLIP for tuning-free adaptive domain generalization.
  arXiv  Detail & Related papers  (2024-03-05T07:10:25Z)
• On the Transferability of Visually Grounded PCFGs [35.64371385720051]
  Visually-grounded Compound PCFGcitepzhao-titov-2020-visually. We consider a zero-shot transfer learning setting where a model is trained on the source domain and is directly applied to target domains, without any further training. Our experimental results suggest that: the benefits from using visual groundings transfer to text in a domain similar to the training domain but fail to transfer to remote domains.
  arXiv  Detail & Related papers  (2023-10-21T20:19:51Z)
• Improving Domain Generalization with Domain Relations [77.63345406973097]
  This paper focuses on domain shifts, which occur when the model is applied to new domains that are different from the ones it was trained on. We propose a new approach called D$3$G to learn domain-specific models. Our results show that D$3$G consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-06T08:11:16Z)
• Unsupervised Domain Generalization for Person Re-identification: A Domain-specific Adaptive Framework [50.88463458896428]
  Domain generalization (DG) has attracted much attention in person re-identification (ReID) recently. Existing methods usually need the source domains to be labeled, which could be a significant burden for practical ReID tasks. We propose a simple and efficient domain-specific adaptive framework, and realize it with an adaptive normalization module.
  arXiv  Detail & Related papers  (2021-11-30T02:35:51Z)
• Pre-training Graph Neural Network for Cross Domain Recommendation [58.057687253631826]
  A recommender system predicts users' potential interests in items, where the core is to learn user/item embeddings. Inspired by the contemporary arts in pre-training from graph representation learning, we propose a pre-training and fine-tuning diagram for cross-domain recommendation. We devise a novel Pre-training Graph Neural Network for Cross-Domain Recommendation (PCRec), which adopts the contrastive self-supervised pre-training of a graph encoder.
  arXiv  Detail & Related papers  (2021-11-16T07:34:42Z)
• f-Domain-Adversarial Learning: Theory and Algorithms [82.97698406515667]
  Unsupervised domain adaptation is used in many machine learning applications where, during training, a model has access to unlabeled data in the target domain. We derive a novel generalization bound for domain adaptation that exploits a new measure of discrepancy between distributions based on a variational characterization of f-divergences.
  arXiv  Detail & Related papers  (2021-06-21T18:21:09Z)
• DAVOS: Semi-Supervised Video Object Segmentation via Adversarial Domain Adaptation [2.9407987406005263]
  Domain shift has always been one of the primary issues in video object segmentation (VOS) We propose a novel method to tackle domain shift by first introducing adversarial domain adaptation to the VOS task. Our model achieves state-of-the-art performance on DAVIS2016 with 82.6% mean IoU score after supervised training.
  arXiv  Detail & Related papers  (2021-05-21T08:23:51Z)
• Supervised Domain Adaptation using Graph Embedding [86.3361797111839]
  Domain adaptation methods assume that distributions between the two domains are shifted and attempt to realign them. We propose a generic framework based on graph embedding. We show that the proposed approach leads to a powerful Domain Adaptation framework.
  arXiv  Detail & Related papers  (2020-03-09T12:25:13Z)
• Learning to adapt class-specific features across domains for semantic segmentation [36.36210909649728]
  In this thesis, we present a novel architecture, which learns to adapt features across domains by taking into account per class information. We adopt the recently introduced StarGAN architecture as image translation backbone, since it is able to perform translations across multiple domains by means of a single generator network.
  arXiv  Detail & Related papers  (2020-01-22T23:51:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.