Learning to Learn with Variational Information Bottleneck for Domain Generalization

• URL: http://arxiv.org/abs/2007.07645v1
• Date: Wed, 15 Jul 2020 12:05:52 GMT
• Title: Learning to Learn with Variational Information Bottleneck for Domain Generalization
• Authors: Yingjun Du, Jun Xu, Huan Xiong, Qiang Qiu, Xiantong Zhen, Cees G. M. Snoek, Ling Shao
• Abstract summary: Domain generalization models learn to generalize to previously unseen domains, but suffer from prediction uncertainty and domain shift. We introduce a probabilistic meta-learning model for domain generalization, in which parameters shared across domains are modeled as distributions. To deal with domain shift, we learn domain-invariant representations by the proposed principle of meta variational information bottleneck, we call MetaVIB.
• Score: 128.90691697063616
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Domain generalization models learn to generalize to previously unseen domains, but suffer from prediction uncertainty and domain shift. In this paper, we address both problems. We introduce a probabilistic meta-learning model for domain generalization, in which classifier parameters shared across domains are modeled as distributions. This enables better handling of prediction uncertainty on unseen domains. To deal with domain shift, we learn domain-invariant representations by the proposed principle of meta variational information bottleneck, we call MetaVIB. MetaVIB is derived from novel variational bounds of mutual information, by leveraging the meta-learning setting of domain generalization. Through episodic training, MetaVIB learns to gradually narrow domain gaps to establish domain-invariant representations, while simultaneously maximizing prediction accuracy. We conduct experiments on three benchmarks for cross-domain visual recognition. Comprehensive ablation studies validate the benefits of MetaVIB for domain generalization. The comparison results demonstrate our method outperforms previous approaches consistently.

Related papers

• Domain Generalization via Causal Adjustment for Cross-Domain Sentiment Analysis [59.73582306457387]
  We focus on the problem of domain generalization for cross-domain sentiment analysis. We propose a backdoor adjustment-based causal model to disentangle the domain-specific and domain-invariant representations. A series of experiments show the great performance and robustness of our model.
  arXiv  Detail & Related papers  (2024-02-22T13:26:56Z)
• Improving Domain Generalization with Domain Relations [77.63345406973097]
  This paper focuses on domain shifts, which occur when the model is applied to new domains that are different from the ones it was trained on. We propose a new approach called D$3$G to learn domain-specific models. Our results show that D$3$G consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-06T08:11:16Z)
• Domain Generalization via Selective Consistency Regularization for Time Series Classification [16.338176636365752]
  Domain generalization methods aim to learn models robust to domain shift with data from a limited number of source domains. We propose a novel representation learning methodology that selectively enforces prediction consistency between source domains.
  arXiv  Detail & Related papers  (2022-06-16T01:57:35Z)
• Discriminative Domain-Invariant Adversarial Network for Deep Domain Generalization [33.84004077585957]
  We propose a discriminative domain-invariant adversarial network (DDIAN) for domain generalization. DDIAN achieves better prediction on unseen target data during training compared to state-of-the-art domain generalization approaches.
  arXiv  Detail & Related papers  (2021-08-20T04:24:12Z)
• A Bit More Bayesian: Domain-Invariant Learning with Uncertainty [111.22588110362705]
  Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data. In this paper, we address both challenges with a probabilistic framework based on variational Bayesian inference. We derive domain-invariant representations and classifiers, which are jointly established in a two-layer Bayesian neural network.
  arXiv  Detail & Related papers  (2021-05-09T21:33:27Z)
• Model-Based Domain Generalization [96.84818110323518]
  We propose a novel approach for the domain generalization problem called Model-Based Domain Generalization. Our algorithms beat the current state-of-the-art methods on the very-recently-proposed WILDS benchmark by up to 20 percentage points.
  arXiv  Detail & Related papers  (2021-02-23T00:59:02Z)
• Domain Invariant Representation Learning with Domain Density Transformations [30.29600757980369]
  Domain generalization refers to the problem where we aim to train a model on data from a set of source domains so that the model can generalize to unseen target domains. We show how to use generative adversarial networks to learn such domain transformations to implement our method in practice.
  arXiv  Detail & Related papers  (2021-02-09T19:25:32Z)
• Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation [102.42638795864178]
  We propose a principled meta-learning based approach to OCDA for semantic segmentation. We cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. We learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization.
  arXiv  Detail & Related papers  (2020-12-15T13:21:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.