Extrapolation in Statistical Learning with Extreme Value Theory

• URL: http://arxiv.org/abs/2605.01909v1
• Date: Sun, 03 May 2026 14:43:51 GMT
• Title: Extrapolation in Statistical Learning with Extreme Value Theory
• Abstract summary: Extreme value theory provides rigorous theory and statistical tools for machine learning.<n>A broad range of tasks benefit from these advances, including regression and classification beyond the training data.<n>This review synthesizes recent developments in these fields at the intersection of statistical learning and extreme value theory.
• Score: 2.5306981887848514
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Extreme value theory provides rigorous theory and statistical tools for extrapolation in machine learning, particularly in settings where traditional methods struggle due to data scarcity in the tails. A broad range of tasks benefit from these advances, including regression and classification beyond the training data, extreme quantile regression, supervised and unsupervised dimension reduction, generative artificial intelligence and anomaly detection. This review synthesizes recent developments in these fields at the intersection of statistical learning and extreme value theory, with a focus on principled methods based on asymptotically motivated representations of the tail of univariate and multivariate distributions. We consider different theoretical frameworks for both asymptotically dependent and independent data and discuss how they translate into efficient statistical methods for extrapolation to extreme regions. By addressing both theoretical and practical aspects, we offer a comprehensive overview of the state-of-the-art in this quickly evolving field, and identify promising directions for future research.

Related papers

• Fused Multinomial Logistic Regression Utilizing Summary-Level External Machine-learning Information [3.2819213801147282]
  We propose a general empirical-likelihood framework that incorporates external predictions through moment constraints.<n>We focus on multinomial logistic regression as the primary model and address common data-quality issues in external sources.<n>We provide mild sufficient conditions under which incorporating external predictions delivers a strict efficiency gain relative to the primary-only estimator.
  arXiv  Detail & Related papers  (2026-04-05T02:37:23Z)
• Takeuchi's Information Criteria as Generalization Measures for DNNs Close to NTK Regime [56.89793618576349]
  Generalization measures have been studied extensively in the machine learning community to better characterize generalization gaps.<n>This study focuses on Takeuchi's information criterion (TIC) to investigate the conditions under which this classical measure can effectively explain the generalization gaps of deep neural networks (DNNs)
  arXiv  Detail & Related papers  (2026-02-26T17:01:14Z)
• Imperfect Influence, Preserved Rankings: A Theory of TRAK for Data Attribution [10.662938465357158]
  We provide a theoretical analysis of the TRAK algorithm, characterizing its performance and quantifying the errors introduced by the approximations on which the method relies.<n>We show that although the approximations incur significant errors, TRAK's estimated influence remains highly correlated with the original influence and therefore largely preserves the relative ranking of data points.
  arXiv  Detail & Related papers  (2026-02-01T16:11:49Z)
• Multiply Robust Conformal Risk Control with Coarsened Data [0.0]
  Conformal Prediction (CP) has recently received a tremendous amount of interest.<n>In this paper, we consider the general problem of obtaining distribution-free valid prediction regions for an outcome given coarsened data.<n>Our principled use of semiparametric theory has the key advantage of facilitating flexible machine learning methods.
  arXiv  Detail & Related papers  (2025-08-21T12:14:44Z)
• Global Convergence of Continual Learning on Non-IID Data [51.99584235667152]
  We provide a general and comprehensive theoretical analysis for continual learning of regression models.<n>We establish the almost sure convergence results of continual learning under a general data condition for the first time.
  arXiv  Detail & Related papers  (2025-03-24T10:06:07Z)
• In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention [52.159541540613915]
  We study how multi-head softmax attention models are trained to perform in-context learning on linear data.<n>Our results reveal that in-context learning ability emerges from the trained transformer as an aggregated effect of its architecture and the underlying data distribution.
  arXiv  Detail & Related papers  (2025-03-17T02:00:49Z)
• A Mathematics Framework of Artificial Shifted Population Risk and Its Further Understanding Related to Consistency Regularization [7.944280447232545]
  This paper introduces a more comprehensive mathematical framework for data augmentation.<n>We establish that the expected risk of the shifted population is the sum of the original population risk and a gap term.<n>The paper also provides a theoretical understanding of this gap, highlighting its negative effects on the early stages of training.
  arXiv  Detail & Related papers  (2025-02-15T08:26:49Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• On Regression in Extreme Regions [0.7974430263940756]
  We establish a statistical learning theoretical framework aimed at extrapolation, or out-of-domain generalization, on the unobserved tails of covariates.<n>We address the stylized problem of nonparametric least squares regression with predictors chosen from a Vapnik-Chervonenkis class.<n>We quantify the predictive performance on tail regions in terms of excess risk, presenting it as a finite sample risk bound with a clear bias-variance decomposition.
  arXiv  Detail & Related papers  (2023-03-06T12:55:38Z)
• On the Benefits of Invariance in Neural Networks [56.362579457990094]
  We show that training with data augmentation leads to better estimates of risk and thereof gradients, and we provide a PAC-Bayes generalization bound for models trained with data augmentation. We also show that compared to data augmentation, feature averaging reduces generalization error when used with convex losses, and tightens PAC-Bayes bounds.
  arXiv  Detail & Related papers  (2020-05-01T02:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.