Domain-Indexing Variational Bayes: Interpretable Domain Index for Domain
Adaptation
- URL: http://arxiv.org/abs/2302.02561v5
- Date: Sat, 10 Jun 2023 13:28:41 GMT
- Title: Domain-Indexing Variational Bayes: Interpretable Domain Index for Domain
Adaptation
- Authors: Zihao Xu, Guang-Yuan Hao, Hao He, Hao Wang
- Abstract summary: We propose an adversarial variational Bayesian framework that infers domain indices from multi-domain data.
Our theoretical analysis shows that our framework finds the optimal domain index at equilibrium.
- Score: 8.46755868848403
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Previous studies have shown that leveraging domain index can significantly
boost domain adaptation performance (arXiv:2007.01807, arXiv:2202.03628).
However, such domain indices are not always available. To address this
challenge, we first provide a formal definition of domain index from the
probabilistic perspective, and then propose an adversarial variational Bayesian
framework that infers domain indices from multi-domain data, thereby providing
additional insight on domain relations and improving domain adaptation
performance. Our theoretical analysis shows that our adversarial variational
Bayesian framework finds the optimal domain index at equilibrium. Empirical
results on both synthetic and real data verify that our model can produce
interpretable domain indices which enable us to achieve superior performance
compared to state-of-the-art domain adaptation methods. Code is available at
https://github.com/Wang-ML-Lab/VDI.
Related papers
- Domain Generalization via Causal Adjustment for Cross-Domain Sentiment
Analysis [59.73582306457387]
We focus on the problem of domain generalization for cross-domain sentiment analysis.
We propose a backdoor adjustment-based causal model to disentangle the domain-specific and domain-invariant representations.
A series of experiments show the great performance and robustness of our model.
arXiv Detail & Related papers (2024-02-22T13:26:56Z) - DAOT: Domain-Agnostically Aligned Optimal Transport for Domain-Adaptive
Crowd Counting [35.83485358725357]
Domain adaptation is commonly employed in crowd counting to bridge the domain gaps between different datasets.
Existing domain adaptation methods tend to focus on inter-dataset differences while overlooking the intra-differences within the same dataset.
We propose a Domain-agnostically Aligned Optimal Transport (DAOT) strategy that aligns domain-agnostic factors between domains.
arXiv Detail & Related papers (2023-08-10T02:59:40Z) - Efficient Hierarchical Domain Adaptation for Pretrained Language Models [77.02962815423658]
Generative language models are trained on diverse, general domain corpora.
We introduce a method to scale domain adaptation to many diverse domains using a computationally efficient adapter approach.
arXiv Detail & Related papers (2021-12-16T11:09:29Z) - A Bit More Bayesian: Domain-Invariant Learning with Uncertainty [111.22588110362705]
Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data.
In this paper, we address both challenges with a probabilistic framework based on variational Bayesian inference.
We derive domain-invariant representations and classifiers, which are jointly established in a two-layer Bayesian neural network.
arXiv Detail & Related papers (2021-05-09T21:33:27Z) - Gradient Matching for Domain Generalization [93.04545793814486]
A critical requirement of machine learning systems is their ability to generalize to unseen domains.
We propose an inter-domain gradient matching objective that targets domain generalization.
We derive a simpler first-order algorithm named Fish that approximates its optimization.
arXiv Detail & Related papers (2021-04-20T12:55:37Z) - Domain2Vec: Domain Embedding for Unsupervised Domain Adaptation [56.94873619509414]
Conventional unsupervised domain adaptation studies the knowledge transfer between a limited number of domains.
We propose a novel Domain2Vec model to provide vectorial representations of visual domains based on joint learning of feature disentanglement and Gram matrix.
We demonstrate that our embedding is capable of predicting domain similarities that match our intuition about visual relations between different domains.
arXiv Detail & Related papers (2020-07-17T22:05:09Z) - Learning to Learn with Variational Information Bottleneck for Domain
Generalization [128.90691697063616]
Domain generalization models learn to generalize to previously unseen domains, but suffer from prediction uncertainty and domain shift.
We introduce a probabilistic meta-learning model for domain generalization, in which parameters shared across domains are modeled as distributions.
To deal with domain shift, we learn domain-invariant representations by the proposed principle of meta variational information bottleneck, we call MetaVIB.
arXiv Detail & Related papers (2020-07-15T12:05:52Z) - Continuously Indexed Domain Adaptation [24.09142831355124]
We propose the first method for continuously indexed domain adaptation.
Our approach combines traditional adversarial adaptation with a novel discriminator that models the encoding-conditioned domain index distribution.
Our empirical results show that our approach outperforms the state-of-the-art domain adaption methods on both synthetic and real-world medical datasets.
arXiv Detail & Related papers (2020-07-03T16:53:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.