Improving Nonparametric Density Estimation with Tensor Decompositions

• URL: http://arxiv.org/abs/2010.02425v1
• Date: Tue, 6 Oct 2020 01:39:09 GMT
• Title: Improving Nonparametric Density Estimation with Tensor Decompositions
• Authors: Robert A. Vandermeulen
• Abstract summary: Nonparametric density estimators often perform well on low dimensional data, but suffer when applied to higher dimensional data. This paper investigates whether these improvements can be extended to other simplified dependence assumptions. We prove that restricting estimation to low-rank nonnegative PARAFAC or Tucker decompositions removes the dimensionality exponent on bin width rates for multidimensional histograms.
• Score: 14.917420021212912
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While nonparametric density estimators often perform well on low dimensional data, their performance can suffer when applied to higher dimensional data, owing presumably to the curse of dimensionality. One technique for avoiding this is to assume no dependence between features and that the data are sampled from a separable density. This allows one to estimate each marginal distribution independently thereby avoiding the slow rates associated with estimating the full joint density. This is a strategy employed in naive Bayes models and is analogous to estimating a rank-one tensor. In this paper we investigate whether these improvements can be extended to other simplified dependence assumptions which we model via nonnegative tensor decompositions. In our central theoretical results we prove that restricting estimation to low-rank nonnegative PARAFAC or Tucker decompositions removes the dimensionality exponent on bin width rates for multidimensional histograms. These results are validated experimentally with high statistical significance via direct application of existing nonnegative tensor factorization to histogram estimators.

Related papers

• High-Dimensional Tensor Discriminant Analysis with Incomplete Tensors [5.745276598549783]
  We introduce a novel approach to tensor classification with incomplete data, framed within high-dimensional linear discriminant analysis. Our method demonstrates excellent performance in simulations and real data analysis, even with significant proportions of missing data.
  arXiv  Detail & Related papers  (2024-10-18T18:00:16Z)
• Double Debiased Covariate Shift Adaptation Robust to Density-Ratio Estimation [7.8856737627153874]
  We propose a doubly robust estimator for covariate shift adaptation via importance weighting. Our estimator reduces the bias arising from the density ratio estimation errors. Notably, our estimator remains consistent if either the density ratio estimator or the regression function is consistent.
  arXiv  Detail & Related papers  (2023-10-25T13:38:29Z)
• Anomaly Detection with Variance Stabilized Density Estimation [49.46356430493534]
  We present a variance-stabilized density estimation problem for maximizing the likelihood of the observed samples. To obtain a reliable anomaly detector, we introduce a spectral ensemble of autoregressive models for learning the variance-stabilized distribution. We have conducted an extensive benchmark with 52 datasets, demonstrating that our method leads to state-of-the-art results.
  arXiv  Detail & Related papers  (2023-06-01T11:52:58Z)
• Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data [68.62134204367668]
  This paper studies score approximation, estimation, and distribution recovery of diffusion models, when data are supported on an unknown low-dimensional linear subspace. We show that with a properly chosen neural network architecture, the score function can be both accurately approximated and efficiently estimated. The generated distribution based on the estimated score function captures the data geometric structures and converges to a close vicinity of the data distribution.
  arXiv  Detail & Related papers  (2023-02-14T17:02:35Z)
• Nonparametric Probabilistic Regression with Coarse Learners [1.8275108630751844]
  We show that we can compute precise conditional densities with minimal assumptions on the shape or form of the density. We demonstrate this approach on a variety of datasets and show competitive performance, particularly on larger datasets.
  arXiv  Detail & Related papers  (2022-10-28T16:25:26Z)
• Statistical Efficiency of Score Matching: The View from Isoperimetry [96.65637602827942]
  We show a tight connection between statistical efficiency of score matching and the isoperimetric properties of the distribution being estimated. We formalize these results both in the sample regime and in the finite regime.
  arXiv  Detail & Related papers  (2022-10-03T06:09:01Z)
• Error Analysis of Tensor-Train Cross Approximation [88.83467216606778]
  We provide accuracy guarantees in terms of the entire tensor for both exact and noisy measurements. Results are verified by numerical experiments, and may have important implications for the usefulness of cross approximations for high-order tensors.
  arXiv  Detail & Related papers  (2022-07-09T19:33:59Z)
• Learning to Estimate Without Bias [57.82628598276623]
  Gauss theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non linear settings via deep learning with bias constraints. A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance.
  arXiv  Detail & Related papers  (2021-10-24T10:23:51Z)
• Low-rank Characteristic Tensor Density Estimation Part II: Compression and Latent Density Estimation [31.631861197477185]
  Learning generative probabilistic models is a core problem in machine learning. This paper proposes a joint dimensionality reduction and non-parametric density estimation framework. We demonstrate that the proposed model achieves very promising results on regression tasks, sampling, and anomaly detection.
  arXiv  Detail & Related papers  (2021-06-20T00:38:56Z)
• Low-rank Characteristic Tensor Density Estimation Part I: Foundations [38.05393186002834]
  We propose a novel approach that builds upon tensor factorization tools. In order to circumvent the curse of dimensionality, we introduce a low-rank model of this characteristic tensor. We demonstrate the very promising performance of the proposed method using several measured datasets.
  arXiv  Detail & Related papers  (2020-08-27T18:06:19Z)
• TraDE: Transformers for Density Estimation [101.20137732920718]
  TraDE is a self-attention-based architecture for auto-regressive density estimation. We present a suite of tasks such as regression using generated samples, out-of-distribution detection, and robustness to noise in the training data.
  arXiv  Detail & Related papers  (2020-04-06T07:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.