General Quantification of Covariate and Concept Shifts

• URL: http://arxiv.org/abs/2609.11918v1
• Date: Thu, 10 Sep 2026 17:57:52 GMT
• Title: General Quantification of Covariate and Concept Shifts
• Abstract summary: Generalization under distribution shift remains a core challenge in modern machine learning.<n>In this paper, we bridge the gap between theory and practical applications.
• Score: 3.7878751488248383
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generalization under distribution shift remains a core challenge in modern machine learning, yet existing learning bound theory is limited to narrow, idealized settings and is non-estimable from samples. In this paper, we bridge the gap between theory and practical applications. We first show that existing definition of concept shift breaks when the source and target supports mismatch. Leveraging entropic optimal transport, we propose a key notion: $γ^{*}\!$-concept shifts, and derive a general error bound unifying covariate and $γ^{*}\!$-concept shifts, which applies to broad loss functions, label spaces, and stochastic labeling. We further develop estimators for these shifts with concentration guarantees, and the DataShifts algorithm, which can quantify distribution shifts and estimate the error bound in most applications - a rigorous and general tool for analyzing learning error under distribution shift.

Related papers

• Adversarial Label Invariant Graph Data Augmentations for Out-of-Distribution Generalization [5.709371904922988]
  Out-of-distribution (OoD) generalization occurs when representation learning encounters a distribution shift.<n>We propose RIA - Regularization for Invariance with Adversarial training.<n>Motivated by an analogy to $Q$-learning, it performs an adversarial exploration for training data environments.
  arXiv  Detail & Related papers  (2026-04-09T16:02:07Z)
• General and Estimable Learning Bound Unifying Covariate and Concept Shifts [3.7878751488248383]
  We bridge the gap between theory and practical applications by developing a novel unified error bound that applies to broad loss functions, label spaces, and labeling.<n>We also develop an algorithm that can quantify the error bound in most distribution shifts -- a rigorous and general tool for analyzing learning error under distribution shift.
  arXiv  Detail & Related papers  (2025-06-15T12:18:05Z)
• Beyond Progress Measures: Theoretical Insights into the Mechanism of Grokking [50.465604300990904]
  Grokking refers to the abrupt improvement in test accuracy after extended overfitting.<n>In this work, we investigate the grokking mechanism underlying the Transformer in the task of prime number operations.
  arXiv  Detail & Related papers  (2025-04-04T04:42:38Z)
• Navigating Semantic Drift in Task-Agnostic Class-Incremental Learning [51.177789437682954]
  Class-incremental learning (CIL) seeks to enable a model to sequentially learn new classes while retaining knowledge of previously learned ones.<n> Balancing flexibility and stability remains a significant challenge, particularly when the task ID is unknown.<n>We propose a novel semantic drift calibration method that incorporates mean shift compensation and covariance calibration.
  arXiv  Detail & Related papers  (2025-02-11T13:57:30Z)
• Towards Understanding Extrapolation: a Causal Lens [53.15488984371969]
  We provide a theoretical understanding of when extrapolation is possible and offer principled methods to achieve it.<n>Under this formulation, we cast the extrapolation problem into a latent-variable identification problem.<n>Our theory reveals the intricate interplay between the underlying manifold's smoothness and the shift properties.
  arXiv  Detail & Related papers  (2025-01-15T21:29:29Z)
• When Invariant Representation Learning Meets Label Shift: Insufficiency and Theoretical Insights [16.72787996847537]
  Generalized label shift (GLS) is the latest developed one which shows great potential to deal with the complex factors within the shift. Main results show the insufficiency of invariant representation learning, and prove the sufficiency and necessity of GLS correction for generalization. We propose a kernel embedding-based correction algorithm (KECA) to minimize the generalization error and achieve successful knowledge transfer.
  arXiv  Detail & Related papers  (2024-06-24T12:47:21Z)
• Boosted Control Functions: Distribution generalization and invariance in confounded models [10.503777692702952]
  We introduce a strong notion of invariance that allows for distribution generalization even in the presence of nonlinear, non-identifiable structural functions.<n>We propose the ControlTwicing algorithm to estimate the Boosted Control Function (BCF) using flexible machine-learning techniques.
  arXiv  Detail & Related papers  (2023-10-09T15:43:46Z)
• Generalized Schrödinger Bridge Matching [54.171931505066]
  Generalized Schr"odinger Bridge (GSB) problem setup is prevalent in many scientific areas both within and without machine learning. We propose Generalized Schr"odinger Bridge Matching (GSBM), a new matching algorithm inspired by recent advances. We show that such a generalization can be cast as solving conditional optimal control, for which variational approximations can be used.
  arXiv  Detail & Related papers  (2023-10-03T17:42:11Z)
• Adapting to Latent Subgroup Shifts via Concepts and Proxies [82.01141290360562]
  We show that the optimal target predictor can be non-parametrically identified with the help of concept and proxy variables available only in the source domain. For continuous observations, we propose a latent variable model specific to the data generation process at hand.
  arXiv  Detail & Related papers  (2022-12-21T18:30:22Z)
• Mitigating Both Covariate and Conditional Shift for Domain Generalization [14.91361835243516]
  Domain generalization (DG) aims to learn a model on several source domains, hoping that the model can generalize well to unseen target domains. In this paper, a novel DG method is proposed to deal with the distribution shift via Visual Alignment and Uncertainty-guided belief Ensemble (VAUE)
  arXiv  Detail & Related papers  (2022-09-17T05:13:56Z)
• Causally-motivated Shortcut Removal Using Auxiliary Labels [63.686580185674195]
  Key challenge to learning such risk-invariant predictors is shortcut learning. We propose a flexible, causally-motivated approach to address this challenge. We show both theoretically and empirically that this causally-motivated regularization scheme yields robust predictors.
  arXiv  Detail & Related papers  (2021-05-13T16:58:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.