Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift

• URL: http://arxiv.org/abs/2406.00661v1
• Date: Sun, 2 Jun 2024 08:11:35 GMT
• Title: Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift
• Authors: Jiayun Wu, Jiashuo Liu, Peng Cui, Zhiwei Steven Wu,
• Abstract summary: We establish a new model-agnostic optimization framework for out-of-distribution generalization via multicalibration. We propose MC-Pseudolabel, a post-processing algorithm to achieve both extended multicalibration and out-of-distribution generalization.
• Score: 44.708914058803224
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We establish a new model-agnostic optimization framework for out-of-distribution generalization via multicalibration, a criterion that ensures a predictor is calibrated across a family of overlapping groups. Multicalibration is shown to be associated with robustness of statistical inference under covariate shift. We further establish a link between multicalibration and robustness for prediction tasks both under and beyond covariate shift. We accomplish this by extending multicalibration to incorporate grouping functions that consider covariates and labels jointly. This leads to an equivalence of the extended multicalibration and invariance, an objective for robust learning in existence of concept shift. We show a linear structure of the grouping function class spanned by density ratios, resulting in a unifying framework for robust learning by designing specific grouping functions. We propose MC-Pseudolabel, a post-processing algorithm to achieve both extended multicalibration and out-of-distribution generalization. The algorithm, with lightweight hyperparameters and optimization through a series of supervised regression steps, achieves superior performance on real-world datasets with distribution shift.

Related papers

• Semiparametric conformal prediction [79.6147286161434]
  Risk-sensitive applications require well-calibrated prediction sets over multiple, potentially correlated target variables. We treat the scores as random vectors and aim to construct the prediction set accounting for their joint correlation structure. We report desired coverage and competitive efficiency on a range of real-world regression problems.
  arXiv  Detail & Related papers  (2024-11-04T14:29:02Z)
• CHASE: Learning Convex Hull Adaptive Shift for Skeleton-based Multi-Entity Action Recognition [10.045163723630159]
  CHASE operates as a sample-adaptive normalization method to mitigate inter-entity distribution discrepancies. Our approach seamlessly adapts to single-entity backbones and boosts their performance in multi-entity scenarios.
  arXiv  Detail & Related papers  (2024-10-09T17:55:43Z)
• Self-Supervised Graph Embedding Clustering [70.36328717683297]
  K-means one-step dimensionality reduction clustering method has made some progress in addressing the curse of dimensionality in clustering tasks. We propose a unified framework that integrates manifold learning with K-means, resulting in the self-supervised graph embedding framework.
  arXiv  Detail & Related papers  (2024-09-24T08:59:51Z)
• On the Convergence of Coordinate Ascent Variational Inference [11.166959724276337]
  We consider the common coordinate ascent variational inference (CAVI) algorithm for implementing the mean-field (MF) VI. We provide general conditions for certifying global or local exponential convergence of CAVI. New notion of generalized correlation for characterizing the interaction between the constituting blocks in influencing the VI objective functional is introduced.
  arXiv  Detail & Related papers  (2023-06-01T20:19:30Z)
• A Unifying Perspective on Multi-Calibration: Game Dynamics for Multi-Objective Learning [63.20009081099896]
  We provide a unifying framework for the design and analysis of multicalibrated predictors. We exploit connections to game dynamics to achieve state-of-the-art guarantees for a diverse set of multicalibration learning problems.
  arXiv  Detail & Related papers  (2023-02-21T18:24:17Z)
• An Exploration of Multicalibration Uniform Convergence Bounds [25.500680663483624]
  We present a framework which yields multicalibration error uniform convergence bounds by reparametrizing sample complexities for Empirical Risk Minimization learning. From this framework, we demonstrate that multicalibration error exhibits dependence on the classifier architecture as well as the underlying data distribution.
  arXiv  Detail & Related papers  (2022-02-09T15:48:10Z)
• A Unified Framework for Multi-distribution Density Ratio Estimation [101.67420298343512]
  Binary density ratio estimation (DRE) provides the foundation for many state-of-the-art machine learning algorithms. We develop a general framework from the perspective of Bregman minimization divergence. We show that our framework leads to methods that strictly generalize their counterparts in binary DRE.
  arXiv  Detail & Related papers  (2021-12-07T01:23:20Z)
• Generalized Adversarially Learned Inference [42.40405470084505]
  We develop methods of inference of latent variables in GANs by adversarially training an image generator along with an encoder to match two joint distributions of image and latent vector pairs. We incorporate multiple layers of feedback on reconstructions, self-supervision, and other forms of supervision based on prior or learned knowledge about the desired solutions.
  arXiv  Detail & Related papers  (2020-06-15T02:18:13Z)
• Robust Grouped Variable Selection Using Distributionally Robust Optimization [11.383869751239166]
  We propose a Distributionally Robust Optimization (DRO) formulation with a Wasserstein-based uncertainty set for selecting grouped variables under perturbations. We prove probabilistic bounds on the out-of-sample loss and the estimation bias, and establish the grouping effect of our estimator. We show that our formulation produces an interpretable and parsimonious model that encourages sparsity at a group level.
  arXiv  Detail & Related papers  (2020-06-10T22:32:52Z)
• Ensemble Model with Batch Spectral Regularization and Data Blending for Cross-Domain Few-Shot Learning with Unlabeled Data [75.94147344921355]
  We build a multi-branch ensemble framework by using diverse feature transformation matrices. We propose a data blending method to exploit the unlabeled data and augment the sparse support set in the target domain.
  arXiv  Detail & Related papers  (2020-06-08T02:27:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.