Unbiased Open World Regularization for Fair Self-Supervised Learning

• URL: http://arxiv.org/abs/2607.22149v1
• Date: Fri, 24 Jul 2026 09:51:22 GMT
• Title: Unbiased Open World Regularization for Fair Self-Supervised Learning
• Abstract summary: Self-supervised learning models are susceptible to learning spurious biases in the dataset.<n>We propose Unbiased Open World Regularization (UOWReg), an encoder-only framework.<n>We empirically validate this framework across both Gaussian and spherical latent spaces.
• Score: 11.74837809839014
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite recent advances, self-supervised learning (SSL) models and Joint-Embedding Predictive Architectures (JEPAs) remain susceptible to learning spurious biases in the dataset. These techniques rely on regularization, which prevents representation collapse by enforcing a global target distribution such as a multivariate Gaussian or a uniform distribution on the sphere. However, these global constraints are insufficient to prevent bias entanglement, as task-irrelevant features can still segregate the latent space into distinct sub-regions. While recent approaches like Entangling and Disentangling (EnD) and Fair Supervised Contrastive Learning (FSCL) empirically debias the latent space, we show that they act as partial approximations of conditional distribution matching. To enforce this matching explicitly, we propose Unbiased Open World Regularization (UOWReg), an encoder-only framework. We show that this shift from a global to a conditional objective guarantees statistical independence between the learned representations and the targeted attributes, regardless of the chosen target distribution. We empirically validate this framework across both Gaussian and spherical latent spaces, using statistical measures to enforce these target distributions. While conditional matching successfully mitigates bias with both distributions, we demonstrate that enforcing conditional uniformity on the sphere yields a lower linearprobing classification error. Empirically, UOWReg reduces Equalized Odds violations on the CelebA benchmark while maintaining competitive classification accuracy compared to existing encoder-only baselines. Furthermore, we introduce the Synthetic Engraving Task-a novel setting in which a dominant macro-structure masks a fine-grained micro-signature. We show that UOWReg effectively prevents the subpopulation collapse observed in standard SSL, successfully isolating micro-signatures even when heavily entangled with the global structure.

Related papers

• OmniISR: A Unified Framework for Centralized and Federated Learning via Intermediate Supervision and Regularization [58.03221830946145]
  We propose OmniISR, a unified framework that fuses pure CL, pure FL, and hybrid CL-FL training modes.<n>We show that OmniISR consistently improves model performance in both centralized and federated paradigms.
  arXiv  Detail & Related papers  (2026-05-19T04:13:27Z)
• Beyond Distribution Estimation: Simplex Anchored Structural Inference Towards Universal Semi-Supervised Learning [47.92504362652887]
  Semi-supervised learning faces significant challenges in realistic scenarios where labeled data is scarce and unlabeled data follows unknown, arbitrary distributions.<n>We formalize this critical yet under-explored paradigm as Universal Semi-supervised Learning (UniSSL)<n>We introduce a weighting strategy based on distribution-agnostic metrics to prioritize reliable pseudo-labels and an auxiliary branch to isolate potentially erroneous pseudo-labels.
  arXiv  Detail & Related papers  (2026-05-08T10:33:09Z)
• Controlling Distributional Bias in Multi-Round LLM Generation via KL-Optimized Fine-Tuning [37.39836347901149]
  We assess whether Large Language Models (LLMs) can generate outputs that adhere to a desired target distribution.<n>We formulate distribution alignment using the attributes of gender, race, and sentiment within occupational contexts.<n>We propose a novel fine-tuning framework that couples Steering Token with Semantic Alignment.
  arXiv  Detail & Related papers  (2026-04-07T11:58:16Z)
• Fine-Grained Bias Exploration and Mitigation for Group-Robust Classification [11.525201208566925]
  Bias Exploration via Overfitting (BEO) captures each distribution in greater detail by modeling it as a mixture of latent groups.<n>We introduce a fine-grained variant of CCDB, termed FG-CCDB, which performs more precise distribution matching and balancing within each group.<n>Our method performs on par with bias-supervised approaches on binary classification tasks and significantly outperforms them in highly biased multi-class scenarios.
  arXiv  Detail & Related papers  (2025-05-11T04:01:34Z)
• Mind the Gap: Confidence Discrepancy Can Guide Federated Semi-Supervised Learning Across Pseudo-Mismatch [50.632535091877706]
  Federated Semi-Supervised Learning (FSSL) aims to leverage unlabeled data across clients with limited labeled data to train a global model with strong generalization ability.<n>Most FSSL methods rely on consistency regularization with pseudo-labels, converting predictions from local or global models into hard pseudo-labels as supervisory signals.<n>We show that the quality of pseudo-label is largely deteriorated by data heterogeneity, an intrinsic facet of federated learning.
  arXiv  Detail & Related papers  (2025-03-17T14:41:51Z)
• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional dependencies for general score-mismatched diffusion samplers.<n>We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions.<n>This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Mitigating Group Bias in Federated Learning for Heterogeneous Devices [1.181206257787103]
  Federated Learning is emerging as a privacy-preserving model training approach in distributed edge applications. Our work proposes a group-fair FL framework that minimizes group-bias while preserving privacy and without resource utilization overhead.
  arXiv  Detail & Related papers  (2023-09-13T16:53:48Z)
• Chaos to Order: A Label Propagation Perspective on Source-Free Domain Adaptation [8.27771856472078]
  We present Chaos to Order (CtO), a novel approach for source-free domain adaptation (SFDA) CtO strives to constrain semantic credibility and propagate label information among target subpopulations. Empirical evidence demonstrates that CtO outperforms the state of the arts on three public benchmarks.
  arXiv  Detail & Related papers  (2023-01-20T03:39:35Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.