Entropic CLT for Order Statistics

• URL: http://arxiv.org/abs/2205.04621v1
• Date: Tue, 10 May 2022 01:37:55 GMT
• Title: Entropic CLT for Order Statistics
• Authors: Martina Cardone and Alex Dytso and Cynthia Rush
• Abstract summary: It is well known that central order statistics exhibit a central limit behavior and converge to a Gaussian distribution as the sample size grows. This paper strengthens this known result by establishing an entropic version of the CLT that ensures a stronger mode of convergence using the relative entropy.
• Score: 39.42561801744333
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: It is well known that central order statistics exhibit a central limit behavior and converge to a Gaussian distribution as the sample size grows. This paper strengthens this known result by establishing an entropic version of the CLT that ensures a stronger mode of convergence using the relative entropy. In particular, an order $O(1/\sqrt{n})$ rate of convergence is established under mild conditions on the parent distribution of the sample generating the order statistics. To prove this result, ancillary results on order statistics are derived, which might be of independent interest.

Related papers

• Convergence Rates for Distribution Matching with Sliced Optimal Transport [10.117027375572627]
  We study the slice-matching scheme, an efficient iterative method for distribution matching based on sliced optimal transport.<n>A key challenge is to control along the trajectory the constants in these inequalities.<n>We show that this becomes tractable for Gaussian distributions.
  arXiv  Detail & Related papers  (2026-02-11T09:47:52Z)
• Pivotal CLTs for Pseudolikelihood via Conditional Centering in Dependent Random Fields [1.3875545441867139]
  We study fluctuations of conditionally centered statistics of the form $$N-1/2sum_i=1N c_i(g(sigma_i)-mathbbE_N[g(sigma_i)|sigma_j,jneq i])$$ where $(sigma_j,ldots,sigma_N) are sampled from a dependent random field.<n>We develop a general framework for maximum pseudolikelihood inference in dependent random fields.
  arXiv  Detail & Related papers  (2025-10-06T16:06:45Z)
• CLT and Edgeworth Expansion for m-out-of-n Bootstrap Estimators of The Studentized Median [4.174296652683762]
  The m-out-of-n bootstrap approximates the distribution of a statistic by repeatedly drawing m subsamples without replacement from an original sample of size n.<n>Despite its broad applicability across econometrics, biostatistics, and machine learning, rigorous parameter-free guarantees for the soundness of the bootstrap have remained elusive.<n>This paper establishes such guarantees by analyzing the estimator of sample quantiles obtained from m-out-of-n resampling of a dataset of size n.
  arXiv  Detail & Related papers  (2025-05-16T22:14:49Z)
• Wasserstein Bounds for generative diffusion models with Gaussian tail targets [0.0]
  We present an estimate of the Wasserstein distance between the data distribution and the generation of score-based generative models. The complexity bound in dimension is $O(sqrtd)$, with a logarithmic constant.
  arXiv  Detail & Related papers  (2024-12-15T17:20:42Z)
• Statistical features of quantum chaos using the Krylov operator complexity [7.338134750636499]
  We study the statistical properties of Lanczos coefficients over an ensemble of random initial operators generating the Krylov space. We propose two statistical quantities that are important in characterizing the complexity. Their resulting statistics are the Wishart distribution and the (rescaled) chi-square distribution respectively.
  arXiv  Detail & Related papers  (2024-11-27T15:20:24Z)
• Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods [59.779795063072655]
  Chain-of-Thought (CoT) prompting and its variants have gained popularity as effective methods for solving multi-step reasoning problems. We analyze CoT prompting from a statistical estimation perspective, providing a comprehensive characterization of its sample complexity.
  arXiv  Detail & Related papers  (2024-08-25T04:07:18Z)
• Signal reconstruction using determinantal sampling [13.531952725283027]
  We study the approximation of a square-integrable function from a finite number of evaluations on a random set of nodes. We show that determinantal point processes and mixtures thereof can yield fast convergence rates.
  arXiv  Detail & Related papers  (2023-10-13T23:02:57Z)
• Adaptive Annealed Importance Sampling with Constant Rate Progress [68.8204255655161]
  Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution. We propose the Constant Rate AIS algorithm and its efficient implementation for $alpha$-divergences.
  arXiv  Detail & Related papers  (2023-06-27T08:15:28Z)
• A High-dimensional Convergence Theorem for U-statistics with Applications to Kernel-based Testing [3.469038201881982]
  We prove a convergence theorem for U-statistics of degree two, where the data dimension $d$ is allowed to scale with sample size $n$. We apply our theory to two popular kernel-based distribution tests, MMD and KSD, whose high-dimensional performance has been challenging to study.
  arXiv  Detail & Related papers  (2023-02-11T12:49:46Z)
• Statistical Efficiency of Score Matching: The View from Isoperimetry [96.65637602827942]
  We show a tight connection between statistical efficiency of score matching and the isoperimetric properties of the distribution being estimated. We formalize these results both in the sample regime and in the finite regime.
  arXiv  Detail & Related papers  (2022-10-03T06:09:01Z)
• Statistical Properties of the Entropy from Ordinal Patterns [55.551675080361335]
  Knowing the joint distribution of the pair Entropy-Statistical Complexity for a large class of time series models would allow statistical tests that are unavailable to date. We characterize the distribution of the empirical Shannon's Entropy for any model under which the true normalized Entropy is neither zero nor one. We present a bilateral test that verifies if there is enough evidence to reject the hypothesis that two signals produce ordinal patterns with the same Shannon's Entropy.
  arXiv  Detail & Related papers  (2022-09-15T23:55:58Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Nearest neighbor empirical processes [7.034466417392574]
  An empirical measure based on the responses from the nearest neighbors to a given point $x$ is introduced and studied as a central statistical quantity. A uniform non-asymptotic bound is established under a well-known condition, often referred to as Vapnik-Chervonenkis, on the uniform entropy numbers. This suggests the possibility of using standard formulas to estimate the variance by using only the nearest neighbors instead of the full data.
  arXiv  Detail & Related papers  (2021-10-27T08:15:20Z)
• On the Estimation of Information Measures of Continuous Distributions [25.395010130602287]
  estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. We provide confidence bounds for simple histogram based estimation of differential entropy from a fixed number of samples. Our focus is on differential entropy, but we provide examples that show that similar results hold for mutual information and relative entropy as well.
  arXiv  Detail & Related papers  (2020-02-07T15:36:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.