Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression

• URL: http://arxiv.org/abs/2606.00512v1
• Date: Sat, 30 May 2026 04:01:14 GMT
• Title: Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression
• Abstract summary: We show that fast labeled sample rates are recovered when proxy is controlled.<n>We also show that distribution regression is a direct special case, with analogous guarantees when the finite bag size is large enough.
• Score: 6.759672187503063
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In many modern machine learning pipelines, abundant pretrained representations serve as noisy proxy covariates, while task-specific labels remain scarce. We study semi-supervised regression in this setting, and propose a simple two stage estimator that learns kernel eigenfeatures from all proxy covariates and fits a ridge predictor on labeled data. We derive finite sample bounds showing that fast labeled sample rates are recovered when proxy perturbation is controlled and unlabeled proxy covariates are sufficiently abundant. We also show that distribution regression is a direct special case, with analogous guarantees when the finite bag size is large enough. Experiments show consistent gains over supervised and semi-supervised baselines, especially in low label regimes.

Related papers

• Self-Balancing Sequential Sampling: Fast Convergence with Controlled Predictability [0.7734726150561088]
  sequential sampling rules that adaptively bias sampling probabilities in order to achieve faster convergence of the empirical distribution to a desired target law.<n>Self-balancing sampler is simple to implement, arises naturally among a class of Markovian samplers.<n>Results support a practical framework for reducing repeated selections and long gaps in coverage without making future selections overly predictable.
  arXiv  Detail & Related papers  (2026-07-23T01:10:42Z)
• Beyond Distribution Estimation: Simplex Anchored Structural Inference Towards Universal Semi-Supervised Learning [47.92504362652887]
  Semi-supervised learning faces significant challenges in realistic scenarios where labeled data is scarce and unlabeled data follows unknown, arbitrary distributions.<n>We formalize this critical yet under-explored paradigm as Universal Semi-supervised Learning (UniSSL)<n>We introduce a weighting strategy based on distribution-agnostic metrics to prioritize reliable pseudo-labels and an auxiliary branch to isolate potentially erroneous pseudo-labels.
  arXiv  Detail & Related papers  (2026-05-08T10:33:09Z)
• Sharp Convergence Rates for Masked Diffusion Models [53.117058231393834]
  We develop a total-variation based analysis for the Euler method that overcomes limitations.<n>Our results relax assumptions on score estimation, improve parameter dependencies, and establish convergence guarantees.<n>Overall, our analysis introduces a direct TV-based error decomposition along the CTMC trajectory and a decoupling-based path-wise analysis for FHS.
  arXiv  Detail & Related papers  (2026-02-26T00:47:51Z)
• Computable Bernstein Certificates for Cross-Fitted Clipped Covariance Estimation [0.0]
  We propose a cross-fitted clipped covariance estimator equipped with emphfully computable Bernstein-type deviation certificates.<n>The resulting procedure adapts to intrinsic complexity measures such as effective rank under mild tail regularity.
  arXiv  Detail & Related papers  (2026-02-15T06:53:40Z)
• The Implicit Bias of Logit Regularization [4.588028371034407]
  We analyze a class of logit regularizers in the context of linear classification.<n>We demonstrate that they induce an implicit bias of logit clustering around finite per-sample targets.<n>Our results extend the theoretical understanding of label smoothing and highlight the efficacy of a broader class of logit-regularization methods.
  arXiv  Detail & Related papers  (2026-02-12T15:06:08Z)
• Combating Noisy Labels through Fostering Self- and Neighbor-Consistency [120.4394402099635]
  Label noise is pervasive in various real-world scenarios, posing challenges in supervised deep learning.<n>We propose a noise-robust method named Jo-SNC (textbfJoint sample selection and model regularization based on textbfSelf- and textbfNeighbor-textbfConsistency)<n>We design a self-adaptive, data-driven thresholding scheme to adjust per-class selection thresholds.
  arXiv  Detail & Related papers  (2026-01-19T07:55:29Z)
• Improved Sample Complexity for Full Coverage in Compact and Continuous Spaces [0.0]
  We study uniform random sampling on the $d$-dimensional unit hypercube.<n>We derive a sample complexity bound with a logarithmic dependence on the failure probability.<n>Our findings offer a sharper theoretical tool for algorithms that rely on grid-based coverage guarantees.
  arXiv  Detail & Related papers  (2025-11-21T21:06:14Z)
• Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning [59.44422468242455]
  We propose a novel method dubbed ShrinkMatch to learn uncertain samples. For each uncertain sample, it adaptively seeks a shrunk class space, which merely contains the original top-1 class. We then impose a consistency regularization between a pair of strongly and weakly augmented samples in the shrunk space to strive for discriminative representations.
  arXiv  Detail & Related papers  (2023-08-13T14:05:24Z)
• Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
  Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios. We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
  arXiv  Detail & Related papers  (2022-04-13T12:43:12Z)
• A Characterization of Semi-Supervised Adversarially-Robust PAC Learnability [57.502573663108535]
  We study the problem of learning an adversarially robust predictor to test time attacks in the semi-supervised PAC model. We show that there is a significant benefit in semi-supervised robust learning even in the worst-case distribution-free model.
  arXiv  Detail & Related papers  (2022-02-11T03:01:45Z)
• RATT: Leveraging Unlabeled Data to Guarantee Generalization [96.08979093738024]
  We introduce a method that leverages unlabeled data to produce generalization bounds. We prove that our bound is valid for 0-1 empirical risk minimization. This work provides practitioners with an option for certifying the generalization of deep nets even when unseen labeled data is unavailable.
  arXiv  Detail & Related papers  (2021-05-01T17:05:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.