Concentration of measure and generalized product of random vectors with an application to Hanson-Wright-like inequalities

• URL: http://arxiv.org/abs/2102.08020v5
• Date: Sun, 25 Jun 2023 13:33:35 GMT
• Title: Concentration of measure and generalized product of random vectors with an application to Hanson-Wright-like inequalities
• Authors: Cosme Louart and Romain Couillet
• Abstract summary: This article provides an expression of the concentration of functionals $phi(Z_1,ldots, Z_m)$ where the variations of $phi$ on each variable depend on the product of the norms (or semi-norms) of the other variables. We illustrate the importance of this result through various generalizations of the Hanson-Wright concentration inequality as well as through a study of the random matrix $XDXT$ and its resolvent $Q =.
• Score: 45.24358490877106
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Starting from concentration of measure hypotheses on $m$ random vectors $Z_1,\ldots, Z_m$, this article provides an expression of the concentration of functionals $\phi(Z_1,\ldots, Z_m)$ where the variations of $\phi$ on each variable depend on the product of the norms (or semi-norms) of the other variables (as if $\phi$ were a product). We illustrate the importance of this result through various generalizations of the Hanson-Wright concentration inequality as well as through a study of the random matrix $XDX^T$ and its resolvent $Q = (I_p - \frac{1}{n}XDX^T)^{-1}$, where $X$ and $D$ are random, which have fundamental interest in statistical machine learning applications.

Related papers

• Near-Optimal Mean Estimation with Unknown, Heteroskedastic Variances [15.990720051907864]
  Subset-of-Signals model serves as a benchmark for heteroskedastic mean estimation. Our algorithm resolves this open question up to logarithmic factors. Even for $d=2$, our techniques enable rates comparable to knowing the variance of each sample.
  arXiv  Detail & Related papers  (2023-12-05T01:13:10Z)
• $L^1$ Estimation: On the Optimality of Linear Estimators [70.75102576909295]
  This work shows that the only prior distribution on $X$ that induces linearity in the conditional median is Gaussian. In particular, it is demonstrated that if the conditional distribution $P_X|Y=y$ is symmetric for all $y$, then $X$ must follow a Gaussian distribution.
  arXiv  Detail & Related papers  (2023-09-17T01:45:13Z)
• Universality laws for Gaussian mixtures in generalized linear models [22.154969876570238]
  We investigate the joint statistics of the family of generalized linear estimators $(Theta_1, dots, Theta_M)$. This allow us to prove the universality of different quantities of interest, such as the training and generalization errors. We discuss the applications of our results to different machine learning tasks of interest, such as ensembling and uncertainty.
  arXiv  Detail & Related papers  (2023-02-17T15:16:06Z)
• Bivariate moments of the two-point correlation function for embedded Gaussian unitary ensemble with $k$-body interactions [0.0]
  Two-point correlation function in eigenvalues of a random matrix ensemble is the ensemble average of the product of the density of eigenvalues at two eigenvalues. Fluctuation measures such as the number variance and Dyson-Mehta $Delta_3$ statistic are defined by the two-point function.
  arXiv  Detail & Related papers  (2022-08-24T05:37:47Z)
• Spectral properties of sample covariance matrices arising from random matrices with independent non identically distributed columns [50.053491972003656]
  It was previously shown that the functionals $texttr(AR(z))$, for $R(z) = (frac1nXXT- zI_p)-1$ and $Ain mathcal M_p$ deterministic, have a standard deviation of order $O(|A|_* / sqrt n)$. Here, we show that $|mathbb E[R(z)] - tilde R(z)|_F
  arXiv  Detail & Related papers  (2021-09-06T14:21:43Z)
• Convex regularization in statistical inverse learning problems [1.7778609937758323]
  We consider Tikhonov regularization with general convex and $p$-homogeneous penalty functionals. We derive concrete rates for Besov norm penalties and numerically demonstrate the correspondence with the observed rates in the context of X-ray tomography.
  arXiv  Detail & Related papers  (2021-02-18T18:12:08Z)
• The Sample Complexity of Robust Covariance Testing [56.98280399449707]
  We are given i.i.d. samples from a distribution of the form $Z = (1-epsilon) X + epsilon B$, where $X$ is a zero-mean and unknown covariance Gaussian $mathcalN(0, Sigma)$. In the absence of contamination, prior work gave a simple tester for this hypothesis testing task that uses $O(d)$ samples. We prove a sample complexity lower bound of $Omega(d2)$ for $epsilon$ an arbitrarily small constant and $gamma
  arXiv  Detail & Related papers  (2020-12-31T18:24:41Z)
• Sparse sketches with small inversion bias [79.77110958547695]
  Inversion bias arises when averaging estimates of quantities that depend on the inverse covariance. We develop a framework for analyzing inversion bias, based on our proposed concept of an $(epsilon,delta)$-unbiased estimator for random matrices. We show that when the sketching matrix $S$ is dense and has i.i.d. sub-gaussian entries, the estimator $(epsilon,delta)$-unbiased for $(Atop A)-1$ with a sketch of size $m=O(d+sqrt d/
  arXiv  Detail & Related papers  (2020-11-21T01:33:15Z)
• Multivariate mean estimation with direction-dependent accuracy [8.147652597876862]
  We consider the problem of estimating the mean of a random vector based on $N$ independent, identically distributed observations. We prove an estimator that has a near-optimal error in all directions in which the variance of the one dimensional marginal of the random vector is not too small.
  arXiv  Detail & Related papers  (2020-10-22T17:52:45Z)
• Locally Private Hypothesis Selection [96.06118559817057]
  We output a distribution from $mathcalQ$ whose total variation distance to $p$ is comparable to the best such distribution. We show that the constraint of local differential privacy incurs an exponential increase in cost. Our algorithms result in exponential improvements on the round complexity of previous methods.
  arXiv  Detail & Related papers  (2020-02-21T18:30:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.