Information Geometrically Generalized Covariate Shift Adaptation

• URL: http://arxiv.org/abs/2304.09387v1
• Date: Wed, 19 Apr 2023 02:52:54 GMT
• Title: Information Geometrically Generalized Covariate Shift Adaptation
• Authors: Masanari Kimura and Hideitsu Hino
• Abstract summary: Many machine learning methods assume that the training and test data follow the same distribution. We show that the well-known family of covariate shift adaptation methods is unified in the framework of information geometry. Numerical experiments show that our generalization can achieve better performance than the existing methods it encompasses.
• Score: 5.990174495635326
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many machine learning methods assume that the training and test data follow the same distribution. However, in the real world, this assumption is very often violated. In particular, the phenomenon that the marginal distribution of the data changes is called covariate shift, one of the most important research topics in machine learning. We show that the well-known family of covariate shift adaptation methods is unified in the framework of information geometry. Furthermore, we show that parameter search for geometrically generalized covariate shift adaptation method can be achieved efficiently. Numerical experiments show that our generalization can achieve better performance than the existing methods it encompasses.

Related papers

• Learning Divergence Fields for Shift-Robust Graph Representations [73.11818515795761]
  In this work, we propose a geometric diffusion model with learnable divergence fields for the challenging problem with interdependent data. We derive a new learning objective through causal inference, which can guide the model to learn generalizable patterns of interdependence that are insensitive across domains.
  arXiv  Detail & Related papers  (2024-06-07T14:29:21Z)
• High-Dimensional Undirected Graphical Models for Arbitrary Mixed Data [2.2871867623460207]
  In many applications data span variables of different types, whose principled joint analysis is nontrivial. Recent advances have shown how the binary-continuous case can be tackled, but the general mixed variable type regime remains challenging. We propose flexible and scalable methodology for data with variables of entirely general mixed type.
  arXiv  Detail & Related papers  (2022-11-21T18:21:31Z)
• SARAH-based Variance-reduced Algorithm for Stochastic Finite-sum Cocoercive Variational Inequalities [137.6408511310322]
  We consider the problem of finite-sum cocoercive variational inequalities. For strongly monotone problems it is possible to achieve linear convergence to a solution using this method.
  arXiv  Detail & Related papers  (2022-10-12T08:04:48Z)
• Equivariant Disentangled Transformation for Domain Generalization under Combination Shift [91.38796390449504]
  Combinations of domains and labels are not observed during training but appear in the test environment. We provide a unique formulation of the combination shift problem based on the concepts of homomorphism, equivariance, and a refined definition of disentanglement.
  arXiv  Detail & Related papers  (2022-08-03T12:31:31Z)
• Equivariance Discovery by Learned Parameter-Sharing [153.41877129746223]
  We study how to discover interpretable equivariances from data. Specifically, we formulate this discovery process as an optimization problem over a model's parameter-sharing schemes. Also, we theoretically analyze the method for Gaussian data and provide a bound on the mean squared gap between the studied discovery scheme and the oracle scheme.
  arXiv  Detail & Related papers  (2022-04-07T17:59:19Z)
• Deep invariant networks with differentiable augmentation layers [87.22033101185201]
  Methods for learning data augmentation policies require held-out data and are based on bilevel optimization problems. We show that our approach is easier and faster to train than modern automatic data augmentation techniques.
  arXiv  Detail & Related papers  (2022-02-04T14:12:31Z)
• The Role of Mutual Information in Variational Classifiers [47.10478919049443]
  We study the generalization error of classifiers relying on encodings trained on the cross-entropy loss. We derive bounds to the generalization error showing that there exists a regime where the generalization error is bounded by the mutual information.
  arXiv  Detail & Related papers  (2020-10-22T12:27:57Z)
• Effective Sample Size, Dimensionality, and Generalization in Covariate Shift Adaptation [0.0]
  We show how effective sample size, dimensionality, and model performance/generalization are formally related in supervised learning. In this paper, we focus on building a unified view connecting the ESS, data dimensionality, and generalization.
  arXiv  Detail & Related papers  (2020-10-02T20:22:59Z)
• Spherical Feature Transform for Deep Metric Learning [58.35971328774927]
  This work proposes a novel spherical feature transform approach. It relaxes the assumption of identical covariance between classes to an assumption of similar covariances of different classes on a hypersphere. We provide a simple and effective training method, and in depth analysis on the relation between the two different transforms.
  arXiv  Detail & Related papers  (2020-08-04T11:32:23Z)
• Density Fixing: Simple yet Effective Regularization Method based on the Class Prior [2.3859169601259347]
  We propose a framework of regularization methods, called density-fixing, that can be used commonly for supervised and semi-supervised learning. Our proposed regularization method improves the generalization performance by forcing the model to approximate the class's prior distribution or the frequency of occurrence.
  arXiv  Detail & Related papers  (2020-07-08T04:58:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.