Revisiting Theory of Contrastive Learning for Domain Generalization

• URL: http://arxiv.org/abs/2512.02831v1
• Date: Tue, 02 Dec 2025 14:39:06 GMT
• Title: Revisiting Theory of Contrastive Learning for Domain Generalization
• Authors: Ali Alvandi, Mina Rezaei,
• Abstract summary: We introduce novel generalization bounds that explicitly account for both types of mismatch: domain shift and domain generalization.<n>Our analysis reveals how the performance of contrastively learned representations depends on the statistical discrepancy between pretraining and downstream distributions.
• Score: 2.6935872912818297
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Contrastive learning is among the most popular and powerful approaches for self-supervised representation learning, where the goal is to map semantically similar samples close together while separating dissimilar ones in the latent space. Existing theoretical methods assume that downstream task classes are drawn from the same latent class distribution used during the pretraining phase. However, in real-world settings, downstream tasks may not only exhibit distributional shifts within the same label space but also introduce new or broader label spaces, leading to domain generalization challenges. In this work, we introduce novel generalization bounds that explicitly account for both types of mismatch: domain shift and domain generalization. Specifically, we analyze scenarios where downstream tasks either (i) draw classes from the same latent class space but with shifted distributions, or (ii) involve new label spaces beyond those seen during pretraining. Our analysis reveals how the performance of contrastively learned representations depends on the statistical discrepancy between pretraining and downstream distributions. This extended perspective allows us to derive provable guarantees on the performance of learned representations on average classification tasks involving class distributions outside the pretraining latent class set.

Related papers

• Balanced Learning for Domain Adaptive Semantic Segmentation [37.70100155953312]
  Unsupervised domain adaptation (UDA) for semantic segmentation aims to transfer knowledge from a labeled source domain to an unlabeled target domain.<n>Despite the effectiveness of self-training techniques in UDA, they struggle to learn each class in a balanced manner due to inherent class imbalance and distribution shift in both data and label space between domains.<n>We propose Balanced Learning for Domain Adaptation (BLDA), a novel approach to directly assess and alleviate class bias without requiring prior knowledge about the distribution shift.
  arXiv  Detail & Related papers  (2025-12-07T15:21:22Z)
• Discriminative Subspace Emersion from learning feature relevances across different populations [35.35606520517552]
  We propose a new Discriminative Subspace Emersion (DSE) method to extend subspace learning toward a general relevance learning framework.<n>DSE allows us to identify the most relevant features in distinguishing the classification task across two populations, even in cases of high overlap between classes.
  arXiv  Detail & Related papers  (2025-03-31T19:33:39Z)
• Guidance Not Obstruction: A Conjugate Consistent Enhanced Strategy for Domain Generalization [50.04665252665413]
  We argue that acquiring discriminative generalization between classes within domains is crucial.<n>In contrast to seeking distribution alignment, we endeavor to safeguard domain-related between-class discrimination.<n>We employ a novel distribution-level Universum strategy to generate supplementary diverse domain-related class-conditional distributions.
  arXiv  Detail & Related papers  (2024-12-13T12:25:16Z)
• Class Distribution Shifts in Zero-Shot Learning: Learning Robust Representations [3.8980564330208662]
  We propose and analyze a model that assumes that the attribute responsible for the shift is unknown in advance.<n>We show that our algorithm improves generalization to diverse class distributions in both simulations and experiments on real-world datasets.
  arXiv  Detail & Related papers  (2023-11-30T14:14:31Z)
• Towards Distribution-Agnostic Generalized Category Discovery [51.52673017664908]
  Data imbalance and open-ended distribution are intrinsic characteristics of the real visual world. We propose a Self-Balanced Co-Advice contrastive framework (BaCon) BaCon consists of a contrastive-learning branch and a pseudo-labeling branch, working collaboratively to provide interactive supervision to resolve the DA-GCD task.
  arXiv  Detail & Related papers  (2023-10-02T17:39:58Z)
• Multi-Domain Long-Tailed Learning by Augmenting Disentangled Representations [80.76164484820818]
  There is an inescapable long-tailed class-imbalance issue in many real-world classification problems. We study this multi-domain long-tailed learning problem and aim to produce a model that generalizes well across all classes and domains. Built upon a proposed selective balanced sampling strategy, TALLY achieves this by mixing the semantic representation of one example with the domain-associated nuisances of another.
  arXiv  Detail & Related papers  (2022-10-25T21:54:26Z)
• Contrastive Learning for Fair Representations [50.95604482330149]
  Trained classification models can unintentionally lead to biased representations and predictions. Existing debiasing methods for classification models, such as adversarial training, are often expensive to train and difficult to optimise. We propose a method for mitigating bias by incorporating contrastive learning, in which instances sharing the same class label are encouraged to have similar representations.
  arXiv  Detail & Related papers  (2021-09-22T10:47:51Z)
• A Theory of Label Propagation for Subpopulation Shift [61.408438422417326]
  We propose a provably effective framework for domain adaptation based on label propagation. We obtain end-to-end finite-sample guarantees on the entire algorithm. We extend our theoretical framework to a more general setting of source-to-target transfer based on a third unlabeled dataset.
  arXiv  Detail & Related papers  (2021-02-22T17:27:47Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• From Anchor Generation to Distribution Alignment: Learning a Discriminative Embedding Space for Zero-Shot Recognition [46.47620562161315]
  In zero-shot learning (ZSL), the samples to be classified are usually projected into side information templates such as attributes. We propose a novel framework called Discriminative Anchor Generation and Distribution Alignment Model (DAGDA) Firstly, in order to rectify the distribution of original templates, a diffusion based graph convolutional network, which can explicitly model the interaction between class and side information, is proposed to produce discriminative anchors. Secondly, to further align the samples with the corresponding anchors in anchor space, which aims to refine the distribution in a fine-grained manner, we introduce a semantic relation regularization
  arXiv  Detail & Related papers  (2020-02-10T05:25:33Z)
• Few-Shot Learning as Domain Adaptation: Algorithm and Analysis [120.75020271706978]
  Few-shot learning uses prior knowledge learned from the seen classes to recognize the unseen classes. This class-difference-caused distribution shift can be considered as a special case of domain shift. We propose a prototypical domain adaptation network with attention (DAPNA) to explicitly tackle such a domain shift problem in a meta-learning framework.
  arXiv  Detail & Related papers  (2020-02-06T01:04:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.