Sub-Gaussian Concentration and Entropic Normality of the Maximum Likelihood Estimator

• URL: http://arxiv.org/abs/2605.07107v1
• Date: Fri, 08 May 2026 01:34:03 GMT
• Title: Sub-Gaussian Concentration and Entropic Normality of the Maximum Likelihood Estimator
• Abstract summary: It is well known that, under standard regularity conditions, the maximum likelihood estimator satisfies a central limit theorem.<n>This paper develops several stronger forms of normality for the normalized MLE.<n> auxiliary tools that may be of independent interest, including exponential consistency bounds, high-moment estimates, and entropy-control arguments for the estimator.
• Score: 14.681656119120584
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: It is well known that, under standard regularity conditions, the maximum likelihood estimator (MLE) satisfies a central limit theorem and converges in distribution to a Gaussian random variable as the sample size grows. This paper strengthens this classical result by developing several stronger forms of asymptotic normality for the normalized MLE. With additional assumptions on the score, we first establish sub-Gaussian tail bounds and convergence of all moments for the normalized estimation error. We then prove an entropic central limit theorem for a smoothed version of the estimator, showing convergence in relative entropy to the limiting Gaussian law. When the Fisher information of the normalized estimate is bounded, or its density has bounded first derivative, we further show that the smoothing can be removed, yielding entropic normality of the MLE itself. The proofs develop auxiliary tools that may be of independent interest, including exponential consistency bounds, high-moment estimates, and entropy-control arguments for the estimator.

Related papers

• Central Limit Theorems for Outcome Records in Disordered Quantum Trajectories [0.0]
  We prove CLT for finite pattern counts in the measurement record of discrete-time quantum trajectories generated by repeated measurements in a disordered environment.<n>We provide criteria and verify the assumptions across a broad family of examples, including disordered walk-type models generated by finite group actions.
  arXiv  Detail & Related papers  (2026-03-30T18:17:02Z)
• Universality of General Spiked Tensor Models [9.454986540713655]
  We study the rank-one spiked tensor model in the high-dimensional regime.<n>We show that their high-dimensional spectral behavior and statistical limits are robust to non-Gaussian noise.
  arXiv  Detail & Related papers  (2026-02-04T11:59:30Z)
• Statistical Limits in Random Tensors with Multiple Correlated Spikes [6.614637831308917]
  We use tools from random matrix theory to study the multi-spiked tensor model.<n>We study the phase transition phenomenon for finding critical points of the corresponding optimization problem.<n>We propose a new estimator of the rank-$r$ weights by solving a system of equations.
  arXiv  Detail & Related papers  (2025-03-05T10:37:54Z)
• Limit Distribution Theory for Quantum Divergences [8.11839312231511]
  We show that a limit distribution theory which characterizes the fluctuations of the estimation error is still premature. As an application of our results, we consider an estimator of quantum relative entropy based on Pauli tomography of quantum states and show that the resulting distribution is a normal, with its variance characterized in terms of the Pauli operators and states. We utilize the knowledge of the aforementioned limit distribution to obtain performance guarantees for a multi-hypothesis testing problem.
  arXiv  Detail & Related papers  (2023-11-22T21:06:41Z)
• Tight Non-asymptotic Inference via Sub-Gaussian Intrinsic Moment Norm [12.400847321321912]
  In non-asymptotic learning, variance-type parameters of sub-Gaussian distributions are of paramount importance. We suggest using the sub-Gaussian intrinsic moment norm achieved by maximizing a sequence of normalized moments.
  arXiv  Detail & Related papers  (2023-03-13T17:03:19Z)
• Nonconvex Stochastic Scaled-Gradient Descent and Generalized Eigenvector Problems [98.34292831923335]
  Motivated by the problem of online correlation analysis, we propose the emphStochastic Scaled-Gradient Descent (SSD) algorithm. We bring these ideas together in an application to online correlation analysis, deriving for the first time an optimal one-time-scale algorithm with an explicit rate of local convergence to normality.
  arXiv  Detail & Related papers  (2021-12-29T18:46:52Z)
• Tight Exponential Analysis for Smoothing the Max-Relative Entropy and for Quantum Privacy Amplification [56.61325554836984]
  The max-relative entropy together with its smoothed version is a basic tool in quantum information theory. We derive the exact exponent for the decay of the small modification of the quantum state in smoothing the max-relative entropy based on purified distance.
  arXiv  Detail & Related papers  (2021-11-01T16:35:41Z)
• Mean-Square Analysis with An Application to Optimal Dimension Dependence of Langevin Monte Carlo [60.785586069299356]
  This work provides a general framework for the non-asymotic analysis of sampling error in 2-Wasserstein distance. Our theoretical analysis is further validated by numerical experiments.
  arXiv  Detail & Related papers  (2021-09-08T18:00:05Z)
• Spectral clustering under degree heterogeneity: a case for the random walk Laplacian [83.79286663107845]
  This paper shows that graph spectral embedding using the random walk Laplacian produces vector representations which are completely corrected for node degree. In the special case of a degree-corrected block model, the embedding concentrates about K distinct points, representing communities.
  arXiv  Detail & Related papers  (2021-05-03T16:36:27Z)
• Benign Overfitting of Constant-Stepsize SGD for Linear Regression [122.70478935214128]
  inductive biases are central in preventing overfitting empirically. This work considers this issue in arguably the most basic setting: constant-stepsize SGD for linear regression. We reflect on a number of notable differences between the algorithmic regularization afforded by (unregularized) SGD in comparison to ordinary least squares.
  arXiv  Detail & Related papers  (2021-03-23T17:15:53Z)
• Amortized Conditional Normalized Maximum Likelihood: Reliable Out of Distribution Uncertainty Estimation [99.92568326314667]
  We propose the amortized conditional normalized maximum likelihood (ACNML) method as a scalable general-purpose approach for uncertainty estimation. Our algorithm builds on the conditional normalized maximum likelihood (CNML) coding scheme, which has minimax optimal properties according to the minimum description length principle. We demonstrate that ACNML compares favorably to a number of prior techniques for uncertainty estimation in terms of calibration on out-of-distribution inputs.
  arXiv  Detail & Related papers  (2020-11-05T08:04:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.