Optimal Regularization for Performative Learning

• URL: http://arxiv.org/abs/2510.12249v1
• Date: Tue, 14 Oct 2025 08:00:08 GMT
• Title: Optimal Regularization for Performative Learning
• Authors: Edwige Cyffers, Alireza Mirrokni, Marco Mondelli,
• Abstract summary: We show how regularization can help cope with performative effects by studying its impact in high-dimensional ridge regression.<n>We show that the optimal regularization scales with the overall strength of the performative effect, making it possible to set the regularization in anticipation of this effect.
• Score: 29.2228276896028
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In performative learning, the data distribution reacts to the deployed model - for example, because strategic users adapt their features to game it - which creates a more complex dynamic than in classical supervised learning. One should thus not only optimize the model for the current data but also take into account that the model might steer the distribution in a new direction, without knowing the exact nature of the potential shift. We explore how regularization can help cope with performative effects by studying its impact in high-dimensional ridge regression. We show that, while performative effects worsen the test risk in the population setting, they can be beneficial in the over-parameterized regime where the number of features exceeds the number of samples. We show that the optimal regularization scales with the overall strength of the performative effect, making it possible to set the regularization in anticipation of this effect. We illustrate this finding through empirical evaluations of the optimal regularization parameter on both synthetic and real-world datasets.

Related papers

• Sequential Data Augmentation for Generative Recommendation [54.765568804267645]
  Generative recommendation plays a crucial role in personalized systems, predicting users' future interactions from their historical behavior sequences.<n>Data augmentation, the process of constructing training data from user interaction histories, is a critical yet underexplored factor in training these models.<n>We propose GenPAS, a principled framework that models augmentation as a sampling process and enables flexible control of the resulting training distribution.<n>Our experiments on benchmark and industrial datasets demonstrate that GenPAS yields superior accuracy, data efficiency, and parameter efficiency compared to existing strategies.
  arXiv  Detail & Related papers  (2025-09-17T02:53:25Z)
• ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisition [28.843447812536677]
  Amortized Active Learning and Inference Engine (ALINE) is a unified framework for amortized Bayesian inference and active data acquisition.<n>ALINE delivers both instant and accurate inference along with efficient selection of informative points.
  arXiv  Detail & Related papers  (2025-06-08T19:15:34Z)
• Feasible Learning [78.6167929413604]
  We introduce Feasible Learning (FL), a sample-centric learning paradigm where models are trained by solving a feasibility problem that bounds the loss for each training sample.<n>Our empirical analysis, spanning image classification, age regression, and preference optimization in large language models, demonstrates that models trained via FL can learn from data while displaying improved tail behavior compared to ERM, with only a marginal impact on average performance.
  arXiv  Detail & Related papers  (2025-01-24T20:39:38Z)
• Optimal Classification under Performative Distribution Shift [13.508249764979075]
  We propose a novel view in which performative effects are modelled as push-forward measures. We prove the convexity of the performative risk under a new set of assumptions. We also establish a connection with adversarially robust classification by reformulating the minimization of the performative risk as a min-max variational problem.
  arXiv  Detail & Related papers  (2024-11-04T12:20:13Z)
• Restoring balance: principled under/oversampling of data for optimal classification [0.0]
  Class imbalance in real-world data poses a common bottleneck for machine learning tasks.<n> Mitigation strategies, such as under or oversampling the data depending on their abundances, are routinely proposed and tested empirically.<n>We provide a sharp prediction of the effects of under/oversampling strategies depending on class imbalance, first and second moments of the data, and the metrics of performance considered.
  arXiv  Detail & Related papers  (2024-05-15T17:45:34Z)
• Boosting Model Resilience via Implicit Adversarial Data Augmentation [20.768174896574916]
  We propose to augment the deep features of samples by incorporating adversarial and anti-adversarial perturbation distributions. We then theoretically reveal that our augmentation process approximates the optimization of a surrogate loss function. We conduct extensive experiments across four common biased learning scenarios.
  arXiv  Detail & Related papers  (2024-04-25T03:22:48Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Improved Fine-tuning by Leveraging Pre-training Data: Theory and Practice [52.11183787786718]
  Fine-tuning a pre-trained model on the target data is widely used in many deep learning applications. Recent studies have empirically shown that training from scratch has the final performance that is no worse than this pre-training strategy. We propose a novel selection strategy to select a subset from pre-training data to help improve the generalization on the target task.
  arXiv  Detail & Related papers  (2021-11-24T06:18:32Z)
• Provably Efficient Causal Reinforcement Learning with Confounded Observational Data [135.64775986546505]
  We study how to incorporate the dataset (observational data) collected offline, which is often abundantly available in practice, to improve the sample efficiency in the online setting. We propose the deconfounded optimistic value iteration (DOVI) algorithm, which incorporates the confounded observational data in a provably efficient manner.
  arXiv  Detail & Related papers  (2020-06-22T14:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.