Lower Bounds on the Total Variation Distance Between Mixtures of Two Gaussians

• URL: http://arxiv.org/abs/2109.01064v1
• Date: Thu, 2 Sep 2021 16:32:16 GMT
• Title: Lower Bounds on the Total Variation Distance Between Mixtures of Two Gaussians
• Authors: Sami Davies, Arya Mazumdar, Soumyabrata Pal, Cyrus Rashtchian
• Abstract summary: We exploit a connection between total variation distance and the characteristic function of the mixture. We derive new lower bounds on the total variation distance between pairs of two-component Gaussian mixtures.
• Score: 45.392805695921666
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mixtures of high dimensional Gaussian distributions have been studied extensively in statistics and learning theory. While the total variation distance appears naturally in the sample complexity of distribution learning, it is analytically difficult to obtain tight lower bounds for mixtures. Exploiting a connection between total variation distance and the characteristic function of the mixture, we provide fairly tight functional approximations. This enables us to derive new lower bounds on the total variation distance between pairs of two-component Gaussian mixtures that have a shared covariance matrix.

Related papers

• Learning general Gaussian mixtures with efficient score matching [16.06356123715737]
  We study the problem of learning mixtures of $k$ Gaussians in $d$ dimensions. We make no separation assumptions on the underlying mixture components. We give an algorithm that draws $dmathrmpoly(k/varepsilon)$ samples from the target mixture, runs in sample-polynomial time, and constructs a sampler.
  arXiv  Detail & Related papers  (2024-04-29T17:30:36Z)
• Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
  We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data. Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
  arXiv  Detail & Related papers  (2024-04-24T09:04:36Z)
• On the best approximation by finite Gaussian mixtures [7.084611118322622]
  We consider the problem of approximating a general Gaussian location mixture by finite mixtures. The minimum order of finite mixtures that achieve a prescribed accuracy is determined within constant factors.
  arXiv  Detail & Related papers  (2024-04-13T06:57:44Z)
• A numerical approximation method for the Fisher-Rao distance between multivariate normal distributions [12.729120803225065]
  We use discretizing curves joining normal distributions and approximating Rao's distances between successive nearby normal distributions on the curves by the square root of Jeffreys divergence. We report on our experiments and assess the quality of our approximation technique by comparing the numerical approximations with both lower and upper bounds.
  arXiv  Detail & Related papers  (2023-02-16T09:44:55Z)
• Theoretical Error Analysis of Entropy Approximation for Gaussian Mixture [0.7499722271664147]
  In this paper, we analyze the approximation error between the true entropy and the approximate one to reveal when this approximation works effectively. Our results provide a guarantee that this approximation works well in higher dimension problems.
  arXiv  Detail & Related papers  (2022-02-26T04:49:01Z)
• A Unified Framework for Multi-distribution Density Ratio Estimation [101.67420298343512]
  Binary density ratio estimation (DRE) provides the foundation for many state-of-the-art machine learning algorithms. We develop a general framework from the perspective of Bregman minimization divergence. We show that our framework leads to methods that strictly generalize their counterparts in binary DRE.
  arXiv  Detail & Related papers  (2021-12-07T01:23:20Z)
• Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond [63.59034509960994]
  We study shuffling-based variants: minibatch and local Random Reshuffling, which draw gradients without replacement. For smooth functions satisfying the Polyak-Lojasiewicz condition, we obtain convergence bounds which show that these shuffling-based variants converge faster than their with-replacement counterparts. We propose an algorithmic modification called synchronized shuffling that leads to convergence rates faster than our lower bounds in near-homogeneous settings.
  arXiv  Detail & Related papers  (2021-10-20T02:25:25Z)
• Uniform Convergence Rates for Maximum Likelihood Estimation under Two-Component Gaussian Mixture Models [13.769786711365104]
  We derive uniform convergence rates for the maximum likelihood estimator and minimax lower bounds for parameter estimation. We assume the mixing proportions of the mixture are known and fixed, but make no separation assumption on the underlying mixture components.
  arXiv  Detail & Related papers  (2020-06-01T04:13:48Z)
• Minimax Optimal Estimation of KL Divergence for Continuous Distributions [56.29748742084386]
  Esting Kullback-Leibler divergence from identical and independently distributed samples is an important problem in various domains. One simple and effective estimator is based on the k nearest neighbor between these samples.
  arXiv  Detail & Related papers  (2020-02-26T16:37:37Z)
• Algebraic and Analytic Approaches for Parameter Learning in Mixture Models [66.96778152993858]
  We present two different approaches for parameter learning in several mixture models in one dimension. For some of these distributions, our results represent the first guarantees for parameter estimation.
  arXiv  Detail & Related papers  (2020-01-19T05:10:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.