Bayesian Domain Invariant Learning via Posterior Generalization of Parameter Distributions

• URL: http://arxiv.org/abs/2310.16277v1
• Date: Wed, 25 Oct 2023 01:17:08 GMT
• Title: Bayesian Domain Invariant Learning via Posterior Generalization of Parameter Distributions
• Authors: Shiyu Shen, Bin Pan, Tianyang Shi, Tao Li, Zhenwei Shi
• Abstract summary: PosTerior Generalization (PTG) shows competitive performance on various domain generalization benchmarks on DomainBed. PTG fully exploits variational inference to approximate parameter distributions, including the invariant posterior and the posteriors on training domains.
• Score: 29.018103152856792
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Domain invariant learning aims to learn models that extract invariant features over various training domains, resulting in better generalization to unseen target domains. Recently, Bayesian Neural Networks have achieved promising results in domain invariant learning, but most works concentrate on aligning features distributions rather than parameter distributions. Inspired by the principle of Bayesian Neural Network, we attempt to directly learn the domain invariant posterior distribution of network parameters. We first propose a theorem to show that the invariant posterior of parameters can be implicitly inferred by aggregating posteriors on different training domains. Our assumption is more relaxed and allows us to extract more domain invariant information. We also propose a simple yet effective method, named PosTerior Generalization (PTG), that can be used to estimate the invariant parameter distribution. PTG fully exploits variational inference to approximate parameter distributions, including the invariant posterior and the posteriors on training domains. Furthermore, we develop a lite version of PTG for widespread applications. PTG shows competitive performance on various domain generalization benchmarks on DomainBed. Additionally, PTG can use any existing domain generalization methods as its prior, and combined with previous state-of-the-art method the performance can be further improved. Code will be made public.

Related papers

• NormAUG: Normalization-guided Augmentation for Domain Generalization [60.159546669021346]
  We propose a simple yet effective method called NormAUG (Normalization-guided Augmentation) for deep learning. Our method introduces diverse information at the feature level and improves the generalization of the main path. In the test stage, we leverage an ensemble strategy to combine the predictions from the auxiliary path of our model, further boosting performance.
  arXiv  Detail & Related papers  (2023-07-25T13:35:45Z)
• Improving Domain Generalization with Domain Relations [77.63345406973097]
  This paper focuses on domain shifts, which occur when the model is applied to new domains that are different from the ones it was trained on. We propose a new approach called D$3$G to learn domain-specific models. Our results show that D$3$G consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-06T08:11:16Z)
• Learning to Learn Domain-invariant Parameters for Domain Generalization [29.821634033299855]
  Domain generalization (DG) aims to overcome this issue by capturing domain-invariant representations from source domains. We propose two modules of Domain Decoupling and Combination (DDC) and Domain-invariance-guided Backpropagation (DIGB) Our proposed method has achieved state-of-the-art performance with strong generalization capability.
  arXiv  Detail & Related papers  (2022-11-04T07:19:34Z)
• Domain Generalization through the Lens of Angular Invariance [44.76809026901016]
  Domain generalization (DG) aims at generalizing a classifier trained on multiple source domains to an unseen target domain with domain shift. We propose a novel deep DG method called Angular Invariance Domain Generalization Network (AIDGN)
  arXiv  Detail & Related papers  (2022-10-28T02:05:38Z)
• Constrained Maximum Cross-Domain Likelihood for Domain Generalization [14.91361835243516]
  Domain generalization aims to learn a generalizable model on multiple source domains, which is expected to perform well on unseen test domains. In this paper, we propose a novel domain generalization method, which minimizes the KL-divergence between posterior distributions from different domains. Experiments on four standard benchmark datasets, i.e., Digits-DG, PACS, Office-Home and miniDomainNet, highlight the superior performance of our method.
  arXiv  Detail & Related papers  (2022-10-09T03:41:02Z)
• Adaptive Domain Generalization via Online Disagreement Minimization [17.215683606365445]
  Domain Generalization aims to safely transfer a model to unseen target domains. AdaODM adaptively modifies the source model at test time for different target domains. Results show AdaODM stably improves the generalization capacity on unseen domains.
  arXiv  Detail & Related papers  (2022-08-03T11:51:11Z)
• META: Mimicking Embedding via oThers' Aggregation for Generalizable Person Re-identification [68.39849081353704]
  Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time. This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
  arXiv  Detail & Related papers  (2021-12-16T08:06:50Z)
• Learning Transferable Parameters for Unsupervised Domain Adaptation [29.962241958947306]
  Untrivial domain adaptation (UDA) enables a learning machine to adapt from a labeled source domain to an unlabeled domain under the distribution shift. We propose Transferable Learning (TransPar) to reduce the side effect brought by domain-specific information in the learning process.
  arXiv  Detail & Related papers  (2021-08-13T09:09:15Z)
• A Bit More Bayesian: Domain-Invariant Learning with Uncertainty [111.22588110362705]
  Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data. In this paper, we address both challenges with a probabilistic framework based on variational Bayesian inference. We derive domain-invariant representations and classifiers, which are jointly established in a two-layer Bayesian neural network.
  arXiv  Detail & Related papers  (2021-05-09T21:33:27Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• Learning to Learn with Variational Information Bottleneck for Domain Generalization [128.90691697063616]
  Domain generalization models learn to generalize to previously unseen domains, but suffer from prediction uncertainty and domain shift. We introduce a probabilistic meta-learning model for domain generalization, in which parameters shared across domains are modeled as distributions. To deal with domain shift, we learn domain-invariant representations by the proposed principle of meta variational information bottleneck, we call MetaVIB.
  arXiv  Detail & Related papers  (2020-07-15T12:05:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.