Distributionally-Constrained Adversaries in Online Learning

• URL: http://arxiv.org/abs/2506.10293v2
• Date: Tue, 17 Jun 2025 18:02:12 GMT
• Title: Distributionally-Constrained Adversaries in Online Learning
• Authors: Moïse Blanchard, Samory Kpotufe,
• Abstract summary: We consider the more general and flexible framework of distributionally constrained adversaries in which instances are drawn from distributions chosen by an adversary.<n>We give a characterization for which distribution classes are learnable in this context against both oblivious and adaptive adversaries.
• Score: 7.903539618132857
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There has been much recent interest in understanding the continuum from adversarial to stochastic settings in online learning, with various frameworks including smoothed settings proposed to bridge this gap. We consider the more general and flexible framework of distributionally constrained adversaries in which instances are drawn from distributions chosen by an adversary within some constrained distribution class [RST11]. Compared to smoothed analysis, we consider general distributional classes which allows for a fine-grained understanding of learning settings between fully stochastic and fully adversarial for which a learner can achieve non-trivial regret. We give a characterization for which distribution classes are learnable in this context against both oblivious and adaptive adversaries, providing insights into the types of interplay between the function class and distributional constraints on adversaries that enable learnability. In particular, our results recover and generalize learnability for known smoothed settings. Further, we show that for several natural function classes including linear classifiers, learning can be achieved without any prior knowledge of the distribution class -- in other words, a learner can simultaneously compete against any constrained adversary within learnable distribution classes.

Related papers

• Deep Fair Learning: A Unified Framework for Fine-tuning Representations with Sufficient Networks [8.616743904155419]
  We propose a framework that integrates sufficient dimension reduction with deep learning to construct fair and informative representations.<n>By introducing a novel penalty term during fine-tuning, our method enforces conditional independence between sensitive attributes and learned representations.<n>Our approach achieves a superior balance between fairness and utility, significantly outperforming state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-04-08T22:24:22Z)
• Generative Modeling of Class Probability for Multi-Modal Representation Learning [7.5696616045063845]
  Multi-modal understanding plays a crucial role in artificial intelligence by enabling models to jointly interpret inputs from different modalities.<n>We propose a novel class anchor alignment approach that leverages class probability distributions for multi-modal representation learning.<n>Our method, Class-anchor-ALigned generative Modeling (CALM), encodes class anchors as prompts to generate and align class probability distributions for each modality.
  arXiv  Detail & Related papers  (2025-03-21T01:17:44Z)
• Regularized Neural Ensemblers [55.15643209328513]
  In this study, we explore employing regularized neural networks as ensemble methods.<n>Motivated by the risk of learning low-diversity ensembles, we propose regularizing the ensembling model by randomly dropping base model predictions.<n>We demonstrate this approach provides lower bounds for the diversity within the ensemble, reducing overfitting and improving generalization capabilities.
  arXiv  Detail & Related papers  (2024-10-06T15:25:39Z)
• Towards Distribution-Agnostic Generalized Category Discovery [51.52673017664908]
  Data imbalance and open-ended distribution are intrinsic characteristics of the real visual world. We propose a Self-Balanced Co-Advice contrastive framework (BaCon) BaCon consists of a contrastive-learning branch and a pseudo-labeling branch, working collaboratively to provide interactive supervision to resolve the DA-GCD task.
  arXiv  Detail & Related papers  (2023-10-02T17:39:58Z)
• Adversarial Rewards in Universal Learning for Contextual Bandits [32.14208422566497]
  We study the limits of learning in contextual bandits, where a learner's rewards depend on their actions and a known context. We show that optimistic universal learning for contextual bandits with adversarial rewards is impossible in general.
  arXiv  Detail & Related papers  (2023-02-14T16:54:22Z)
• Optimal Online Generalized Linear Regression with Stochastic Noise and Its Application to Heteroscedastic Bandits [88.6139446295537]
  We study the problem of online generalized linear regression in the setting of a generalized linear model with possibly unbounded additive noise. We provide a sharp analysis of the classical follow-the-regularized-leader (FTRL) algorithm to cope with the label noise. We propose an algorithm based on FTRL to achieve the first variance-aware regret bound.
  arXiv  Detail & Related papers  (2022-02-28T08:25:26Z)
• Adversarial Robustness with Semi-Infinite Constrained Learning [177.42714838799924]
  Deep learning to inputs perturbations has raised serious questions about its use in safety-critical domains. We propose a hybrid Langevin Monte Carlo training approach to mitigate this issue. We show that our approach can mitigate the trade-off between state-of-the-art performance and robust robustness.
  arXiv  Detail & Related papers  (2021-10-29T13:30:42Z)
• Contrastive Learning for Fair Representations [50.95604482330149]
  Trained classification models can unintentionally lead to biased representations and predictions. Existing debiasing methods for classification models, such as adversarial training, are often expensive to train and difficult to optimise. We propose a method for mitigating bias by incorporating contrastive learning, in which instances sharing the same class label are encouraged to have similar representations.
  arXiv  Detail & Related papers  (2021-09-22T10:47:51Z)
• A Low Rank Promoting Prior for Unsupervised Contrastive Learning [108.91406719395417]
  We construct a novel probabilistic graphical model that effectively incorporates the low rank promoting prior into the framework of contrastive learning. Our hypothesis explicitly requires that all the samples belonging to the same instance class lie on the same subspace with small dimension. Empirical evidences show that the proposed algorithm clearly surpasses the state-of-the-art approaches on multiple benchmarks.
  arXiv  Detail & Related papers  (2021-08-05T15:58:25Z)
• Certifiably-Robust Federated Adversarial Learning via Randomized Smoothing [16.528628447356496]
  In this paper, we incorporate smoothing techniques into federated adversarial training to enable data-private distributed learning. Our experiments show that such an advanced federated adversarial learning framework can deliver models as robust as those trained by the centralized training.
  arXiv  Detail & Related papers  (2021-03-30T02:19:45Z)
• Beyond cross-entropy: learning highly separable feature distributions for robust and accurate classification [22.806324361016863]
  We propose a novel approach for training deep robust multiclass classifiers that provides adversarial robustness. We show that the regularization of the latent space based on our approach yields excellent classification accuracy.
  arXiv  Detail & Related papers  (2020-10-29T11:15:17Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.