Sharper Sub-Weibull Concentrations: Non-asymptotic Bai-Yin's Theorem

• URL: http://arxiv.org/abs/2102.02450v1
• Date: Thu, 4 Feb 2021 07:16:27 GMT
• Title: Sharper Sub-Weibull Concentrations: Non-asymptotic Bai-Yin's Theorem
• Authors: Huiming Zhang, Haoyu Wei
• Abstract summary: Non-asymptotic concentration inequalities play an essential role in the finite-sample theory of machine learning and statistics. We obtain a sharper and constants-specified concentration inequality for the summation of independent sub-Weibull random variables. In the application of negative binomial regressions, we gives the $ell$-error with sparse structures, which is a new result for negative binomial regressions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Arising in high-dimensional probability, non-asymptotic concentration inequalities play an essential role in the finite-sample theory of machine learning and high-dimensional statistics. In this article, we obtain a sharper and constants-specified concentration inequality for the summation of independent sub-Weibull random variables, which leads to a mixture of two tails: sub-Gaussian for small deviations and sub-Weibull for large deviations (from mean). These bounds improve existing bounds with sharper constants. In the application of random matrices, we derive non-asymptotic versions of Bai-Yin's theorem for sub-Weibull entries and it extends the previous result in terms of sub-Gaussian entries. In the application of negative binomial regressions, we gives the $\ell_2$-error of the estimated coefficients when covariate vector $X$ is sub-Weibull distributed with sparse structures, which is a new result for negative binomial regressions.

Related papers

• Efficient Certificates of Anti-Concentration Beyond Gaussians [15.709968227246453]
  This work presents a new (and arguably the most natural) formulation for anti-concentration. We give quasi-polynomial time verifiable sum-of-squares certificates of anti-concentration that hold for a wide class of non-Gaussian distributions. Our approach constructs a canonical integer program for anti-concentration and analysis a sum-of-squares relaxation of it, independent of the intended application.
  arXiv  Detail & Related papers  (2024-05-23T22:13:44Z)
• Wasserstein F-tests for Fréchet regression on Bures-Wasserstein manifolds [0.9514940899499753]
  Fr'echet regression on the Bures-Wasserstein manifold is developed. A test for the null hypothesis of no association is proposed. Results show that the proposed test has the desired level of significanceally.
  arXiv  Detail & Related papers  (2024-04-05T04:01:51Z)
• Asymptotic Inference for Infinitely Imbalanced Logistic Regression [4.981260380070016]
  We show that the variance of the the limiting slope depends exponentially on the z-score of the average of the minority class's points with respect to the majority class's distribution. We confirm our results by Monte Carlo simulations.
  arXiv  Detail & Related papers  (2022-04-27T23:52:42Z)
• Distribution of Classification Margins: Are All Data Equal? [61.16681488656473]
  We motivate theoretically and show empirically that the area under the curve of the margin distribution on the training set is in fact a good measure of generalization. The resulting subset of "high capacity" features is not consistent across different training runs.
  arXiv  Detail & Related papers  (2021-07-21T16:41:57Z)
• Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensions [79.35722941720734]
  Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks. We prove exacts characterising the estimator in high-dimensions via empirical risk minimisation. We discuss how our theory can be applied beyond the scope of synthetic data.
  arXiv  Detail & Related papers  (2021-06-07T16:53:56Z)
• Optimization Variance: Exploring Generalization Properties of DNNs [83.78477167211315]
  The test error of a deep neural network (DNN) often demonstrates double descent. We propose a novel metric, optimization variance (OV), to measure the diversity of model updates.
  arXiv  Detail & Related papers  (2021-06-03T09:34:17Z)
• Spectral clustering under degree heterogeneity: a case for the random walk Laplacian [83.79286663107845]
  This paper shows that graph spectral embedding using the random walk Laplacian produces vector representations which are completely corrected for node degree. In the special case of a degree-corrected block model, the embedding concentrates about K distinct points, representing communities.
  arXiv  Detail & Related papers  (2021-05-03T16:36:27Z)
• Concentration Inequalities for Statistical Inference [3.236217153362305]
  This paper gives a review of concentration inequalities which are widely employed in non-asymptotical analyses of mathematical statistics. We aim to illustrate the concentration inequalities with known constants and to improve existing bounds with sharper constants.
  arXiv  Detail & Related papers  (2020-11-04T12:54:06Z)
• On Linear Stochastic Approximation: Fine-grained Polyak-Ruppert and Non-Asymptotic Concentration [115.1954841020189]
  We study the inequality and non-asymptotic properties of approximation procedures with Polyak-Ruppert averaging. We prove a central limit theorem (CLT) for the averaged iterates with fixed step size and number of iterations going to infinity.
  arXiv  Detail & Related papers  (2020-04-09T17:54:18Z)
• Sharp Concentration Results for Heavy-Tailed Distributions [17.510560590853576]
  We obtain concentration and large deviation for the sums of independent and identically distributed random variables with heavy-tailed distributions. Our main theorem can not only recover some of the existing results, such as the concentration of the sum of subWeibull random variables, but it can also produce new results for the sum of random variables with heavier tails.
  arXiv  Detail & Related papers  (2020-03-30T21:05:29Z)
• The Vector Poisson Channel: On the Linearity of the Conditional Mean Estimator [82.5577471797883]
  This work studies properties of the conditional mean estimator in vector Poisson noise. The first result shows that the conditional mean estimator cannot be linear when the dark current parameter of the Poisson noise is non-zero. The second result produces a quantitative refinement of the first result.
  arXiv  Detail & Related papers  (2020-03-19T18:21:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.