Consistent Density Estimation Under Discrete Mixture Models
- URL: http://arxiv.org/abs/2105.01108v1
- Date: Mon, 3 May 2021 18:30:02 GMT
- Title: Consistent Density Estimation Under Discrete Mixture Models
- Authors: Luc Devroye and Alex Dytso
- Abstract summary: This work considers a problem of estimating a mixing probability density $f$ in the setting of discrete mixture models.
In particular, it is shown that there exists an estimator $f_n$ such that for every density $f$ $lim_nto infty mathbbE left[ int |f_n -f | right]=0$.
- Score: 20.935152220339056
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: This work considers a problem of estimating a mixing probability density $f$
in the setting of discrete mixture models. The paper consists of three parts.
The first part focuses on the construction of an $L_1$ consistent estimator
of $f$. In particular, under the assumptions that the probability measure $\mu$
of the observation is atomic, and the map from $f$ to $\mu$ is bijective, it is
shown that there exists an estimator $f_n$ such that for every density $f$
$\lim_{n\to \infty} \mathbb{E} \left[ \int |f_n -f | \right]=0$.
The second part discusses the implementation details. Specifically, it is
shown that the consistency for every $f$ can be attained with a computationally
feasible estimator.
The third part, as a study case, considers a Poisson mixture model. In
particular, it is shown that in the Poisson noise setting, the bijection
condition holds and, hence, estimation can be performed consistently for every
$f$.
Related papers
- Estimating the Mixing Coefficients of Geometrically Ergodic Markov
Processes [5.00389879175348]
We estimate the individual $beta$-mixing coefficients of a real-valued geometrically ergodic Markov process from a single sample-path.
Naturally no density assumptions are required in this setting; the expected error rate is shown to be of order $mathcal O(log(n) n-1/2)$.
arXiv Detail & Related papers (2024-02-11T20:17:10Z) - A Unified Framework for Uniform Signal Recovery in Nonlinear Generative
Compressed Sensing [68.80803866919123]
Under nonlinear measurements, most prior results are non-uniform, i.e., they hold with high probability for a fixed $mathbfx*$ rather than for all $mathbfx*$ simultaneously.
Our framework accommodates GCS with 1-bit/uniformly quantized observations and single index models as canonical examples.
We also develop a concentration inequality that produces tighter bounds for product processes whose index sets have low metric entropy.
arXiv Detail & Related papers (2023-09-25T17:54:19Z) - $L^1$ Estimation: On the Optimality of Linear Estimators [70.75102576909295]
This work shows that the only prior distribution on $X$ that induces linearity in the conditional median is Gaussian.
In particular, it is demonstrated that if the conditional distribution $P_X|Y=y$ is symmetric for all $y$, then $X$ must follow a Gaussian distribution.
arXiv Detail & Related papers (2023-09-17T01:45:13Z) - Data Structures for Density Estimation [66.36971978162461]
Given a sublinear (in $n$) number of samples from $p$, our main result is the first data structure that identifies $v_i$ in time sublinear in $k$.
We also give an improved version of the algorithm of Acharya et al. that reports $v_i$ in time linear in $k$.
arXiv Detail & Related papers (2023-06-20T06:13:56Z) - The Sketched Wasserstein Distance for mixture distributions [13.643197515573029]
Sketched Wasserstein Distance ($WS$) is a new probability distance specifically tailored to finite mixture distributions.
We show that $WS$ is defined to be the most discriminative of this metric to the space $mathcalS = textrmconv(mathcalA)$ of mixtures of elements of $mathcalA$.
arXiv Detail & Related papers (2022-06-26T02:33:40Z) - Mean Estimation in High-Dimensional Binary Markov Gaussian Mixture
Models [12.746888269949407]
We consider a high-dimensional mean estimation problem over a binary hidden Markov model.
We establish a nearly minimax optimal (up to logarithmic factors) estimation error rate, as a function of $|theta_*|,delta,d,n$.
arXiv Detail & Related papers (2022-06-06T09:34:04Z) - Beyond Black Box Densities: Parameter Learning for the Deviated
Components [15.501680326749515]
A known density function estimate may have been previously obtained by a black box method.
The increased complexity of the data set may result in the true density being deviated from the known estimate by a mixture distribution.
We establish rates of convergence for the maximum likelihood estimates of $lambda*$ and $G*$ under Wasserstein metric.
arXiv Detail & Related papers (2022-02-05T22:44:20Z) - Universal Regular Conditional Distributions via Probability
Measure-Valued Deep Neural Models [3.8073142980733]
We find that any model built using the proposed framework is dense in the space $C(mathcalX,mathcalP_1(mathcalY))$.
The proposed models are also shown to be capable of generically expressing the aleatoric uncertainty present in most randomized machine learning models.
arXiv Detail & Related papers (2021-05-17T11:34:09Z) - The Sample Complexity of Robust Covariance Testing [56.98280399449707]
We are given i.i.d. samples from a distribution of the form $Z = (1-epsilon) X + epsilon B$, where $X$ is a zero-mean and unknown covariance Gaussian $mathcalN(0, Sigma)$.
In the absence of contamination, prior work gave a simple tester for this hypothesis testing task that uses $O(d)$ samples.
We prove a sample complexity lower bound of $Omega(d2)$ for $epsilon$ an arbitrarily small constant and $gamma
arXiv Detail & Related papers (2020-12-31T18:24:41Z) - Optimal Mean Estimation without a Variance [103.26777953032537]
We study the problem of heavy-tailed mean estimation in settings where the variance of the data-generating distribution does not exist.
We design an estimator which attains the smallest possible confidence interval as a function of $n,d,delta$.
arXiv Detail & Related papers (2020-11-24T22:39:21Z) - Analysis of KNN Density Estimation [56.29748742084386]
kNN density estimation is minimax optimal under both $ell_infty$ and $ell_infty$ criteria, if the support set is known.
The $ell_infty$ error does not reach the minimax lower bound, but is better than kernel density estimation.
arXiv Detail & Related papers (2020-09-30T03:33:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.