Iterative Chow Filtering for Learning with Distribution Shift

• URL: http://arxiv.org/abs/2605.17251v1
• Date: Sun, 17 May 2026 04:18:17 GMT
• Title: Iterative Chow Filtering for Learning with Distribution Shift
• Abstract summary: We show that the weaker notion of $cal L$ sandwiching suffices for efficient PQ learning.<n>We obtain the first quasipolynomial-time PQ learning algorithm for DNFs under uniform distribution.
• Score: 25.270503886184795
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent work due to Goel et al. gave the first efficient algorithms for learning with distribution shift in the challenging PQ framework. In this setting, a learner receives labeled training examples, unlabeled test examples, and must make correct predictions on the test set but is allowed to abstain from predicting on out-of-distribution points. Their results rely on ${\cal L}_2$ sandwiching approximations, a strong requirement that leads to poor bounds for several basic function classes such as DNF formulas. Here, we show that the weaker notion of ${\cal L}_1$ sandwiching suffices for efficient PQ learning. As a consequence, we obtain the first quasipolynomial-time PQ learning algorithm for DNFs under the uniform distribution and essentially match the guarantees known for ordinary PAC learning. More broadly, our bounds provide exponential improvements for several classes including constant depth circuits and constant degree polynomial threshold functions. Our main technical ingredient is Iterative Chow Filtering, a new procedure that uses low-degree Chow parameters to identify and remove test points incompatible with the training distribution.

Related papers

• Equivalence of Coarse and Fine-Grained Models for Learning with Distribution Shift [21.679363840452087]
  We give an efficient black-box reduction from PQ learning to TDS learning for any Boolean concept class.<n>This equivalence implies the first hardness results for distribution-free TDS learning of basic classes such as halfspaces.<n>We also show that giving a learner access to membership queries sidesteps these hardness results and allows for efficient, distribution-free PQ learnability of halfspaces.
  arXiv  Detail & Related papers  (2026-05-07T22:37:29Z)
• On the Learning Curves of Revenue Maximization [62.087200798198786]
  A learning curve plots the decay of an algorithm's error for a fixed underlying distribution as a function of the number of training samples.<n>Prior work on revenue-maximizing learning algorithms adopts a distribution-free perspective, which parallels the PAC learning framework in learning theory.<n>We show that there exists a Bayes-consistent algorithm, meaning that its learning curve converges to zero for any arbitrary valuation distribution.
  arXiv  Detail & Related papers  (2026-04-29T17:38:25Z)
• Learning Intersections of Two Margin Halfspaces under Factorizable Distributions [56.51474048985742]
  Learning intersections of halfspaces is a central problem in Computational Learning Theory.<n>Even for just two halfspaces, it remains a major open question whether learning is possible in time.<n>We introduce a novel algorithm that provably circumvents the CSQ hardness barrier.
  arXiv  Detail & Related papers  (2025-11-13T00:28:24Z)
• Outcome-Based Online Reinforcement Learning: Algorithms and Fundamental Limits [58.63897489864948]
  Reinforcement learning with outcome-based feedback faces a fundamental challenge.<n>How do we assign credit to the right actions?<n>This paper provides the first comprehensive analysis of this problem in online RL with general function approximation.
  arXiv  Detail & Related papers  (2025-05-26T17:44:08Z)
• Tolerant Algorithms for Learning with Arbitrary Covariate Shift [18.37181965815327]
  We study the problem of learning under arbitrary distribution shift, where the learner is trained on a labeled set from one distribution but evaluated on a different, potentially adversarially generated test distribution. We focus on two frameworks: PQ learning [Goldwasser, A. Kalai, Y. Kalai, Montasser NeurIPS 2020], allowing abstention on adversarially generated parts of the test distribution, and TDS learning [Klivans, Stavropoulos, Vasilyan COLT 2024], permitting abstention on the entire test distribution if distribution shift is detected.
  arXiv  Detail & Related papers  (2024-06-04T19:50:05Z)
• Sample Complexity Bounds for Robustly Learning Decision Lists against Evasion Attacks [25.832511407411637]
  A fundamental problem in adversarial machine learning is to quantify how much training data is needed in the presence of evasion attacks. We work with probability distributions on the input data that satisfy a Lipschitz condition: nearby points have similar probability. For every fixed $k$ the class of $k$-decision lists has sample complexity against a $log(n)$-bounded adversary.
  arXiv  Detail & Related papers  (2022-05-12T14:40:18Z)
• Transformers Can Do Bayesian Inference [56.99390658880008]
  We present Prior-Data Fitted Networks (PFNs) PFNs leverage in-context learning in large-scale machine learning techniques to approximate a large set of posteriors. We demonstrate that PFNs can near-perfectly mimic Gaussian processes and also enable efficient Bayesian inference for intractable problems.
  arXiv  Detail & Related papers  (2021-12-20T13:07:39Z)
• Efficient Learning with Arbitrary Covariate Shift [19.230859892728784]
  We give an efficient algorithm for learning a binary function in a class C of bounded VC dimension, with training data distributed according to P and test data according to Q. The present work gives a time PQ-learning algorithm that uses an oracle to a "reliable learner" for C, where reliable learning is a model of learning with one-sided noise.
  arXiv  Detail & Related papers  (2021-02-15T19:14:25Z)
• A Polynomial Time Algorithm for Learning Halfspaces with Tsybakov Noise [55.45544638410858]
  We study the problem of PAC learning homogeneous halfspaces in the presence of Tsybakov noise. Our algorithm learns the true halfspace within any accuracy $epsilon$.
  arXiv  Detail & Related papers  (2020-10-04T22:19:06Z)
• Provably Efficient Reward-Agnostic Navigation with Linear Value Iteration [143.43658264904863]
  We show how iteration under a more standard notion of low inherent Bellman error, typically employed in least-square value-style algorithms, can provide strong PAC guarantees on learning a near optimal value function. We present a computationally tractable algorithm for the reward-free setting and show how it can be used to learn a near optimal policy for any (linear) reward function.
  arXiv  Detail & Related papers  (2020-08-18T04:34:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.