Tensor-Train Density Estimation

• URL: http://arxiv.org/abs/2108.00089v1
• Date: Fri, 30 Jul 2021 21:51:12 GMT
• Title: Tensor-Train Density Estimation
• Authors: Georgii S. Novikov, Maxim E. Panov, Ivan V. Oseledets
• Abstract summary: We propose a new efficient tensor train-based model for density estimation (TTDE) Such density parametrization allows exact sampling, calculation of cumulative and marginal density functions, and partition function. We show that TTDE significantly outperforms competitors in training speed.
• Score: 16.414910030716555
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Estimation of probability density function from samples is one of the central problems in statistics and machine learning. Modern neural network-based models can learn high dimensional distributions but have problems with hyperparameter selection and are often prone to instabilities during training and inference. We propose a new efficient tensor train-based model for density estimation (TTDE). Such density parametrization allows exact sampling, calculation of cumulative and marginal density functions, and partition function. It also has very intuitive hyperparameters. We develop an efficient non-adversarial training procedure for TTDE based on the Riemannian optimization. Experimental results demonstrate the competitive performance of the proposed method in density estimation and sampling tasks, while TTDE significantly outperforms competitors in training speed.

Related papers

• Dynamical Measure Transport and Neural PDE Solvers for Sampling [77.38204731939273]
  We tackle the task of sampling from a probability density as transporting a tractable density function to the target. We employ physics-informed neural networks (PINNs) to approximate the respective partial differential equations (PDEs) solutions. PINNs allow for simulation- and discretization-free optimization and can be trained very efficiently.
  arXiv  Detail & Related papers  (2024-07-10T17:39:50Z)
• Equation Discovery with Bayesian Spike-and-Slab Priors and Efficient Kernels [57.46832672991433]
  We propose a novel equation discovery method based on Kernel learning and BAyesian Spike-and-Slab priors (KBASS) We use kernel regression to estimate the target function, which is flexible, expressive, and more robust to data sparsity and noises. We develop an expectation-propagation expectation-maximization algorithm for efficient posterior inference and function estimation.
  arXiv  Detail & Related papers  (2023-10-09T03:55:09Z)
• Adaptive learning of density ratios in RKHS [3.047411947074805]
  Estimating the ratio of two probability densities from finitely many observations is a central problem in machine learning and statistics. We analyze a large class of density ratio estimation methods that minimize a regularized Bregman divergence between the true density ratio and a model in a reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2023-07-30T08:18:39Z)
• Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation [59.45669299295436]
  We propose a Monte Carlo PDE solver for training unsupervised neural solvers. We use the PDEs' probabilistic representation, which regards macroscopic phenomena as ensembles of random particles. Our experiments on convection-diffusion, Allen-Cahn, and Navier-Stokes equations demonstrate significant improvements in accuracy and efficiency.
  arXiv  Detail & Related papers  (2023-02-10T08:05:19Z)
• Quantum Adaptive Fourier Features for Neural Density Estimation [0.0]
  This paper presents a method for neural density estimation that can be seen as a type of kernel density estimation. The method is based on density matrices, a formalism used in quantum mechanics, and adaptive Fourier features. The method was evaluated in different synthetic and real datasets, and its performance compared against state-of-the-art neural density estimation methods.
  arXiv  Detail & Related papers  (2022-08-01T01:39:11Z)
• TAKDE: Temporal Adaptive Kernel Density Estimator for Real-Time Dynamic Density Estimation [16.45003200150227]
  We name the temporal adaptive kernel density estimator (TAKDE) TAKDE is theoretically optimal in terms of the worst-case AMISE. We provide numerical experiments using synthetic and real-world datasets, showing that TAKDE outperforms other state-of-the-art dynamic density estimators.
  arXiv  Detail & Related papers  (2022-03-15T23:38:32Z)
• A Non-Classical Parameterization for Density Estimation Using Sample Moments [0.0]
  We propose a non-classical parametrization for density estimation using sample moments. The proposed estimator is the first one in the literature for which the power moments up to an arbitrary even order exactly match the sample moments.
  arXiv  Detail & Related papers  (2022-01-13T04:28:52Z)
• Density-Based Clustering with Kernel Diffusion [59.4179549482505]
  A naive density corresponding to the indicator function of a unit $d$-dimensional Euclidean ball is commonly used in density-based clustering algorithms. We propose a new kernel diffusion density function, which is adaptive to data of varying local distributional characteristics and smoothness.
  arXiv  Detail & Related papers  (2021-10-11T09:00:33Z)
• Efficient training of lightweight neural networks using Online Self-Acquired Knowledge Distillation [51.66271681532262]
  Online Self-Acquired Knowledge Distillation (OSAKD) is proposed, aiming to improve the performance of any deep neural model in an online manner. We utilize k-nn non-parametric density estimation technique for estimating the unknown probability distributions of the data samples in the output feature space.
  arXiv  Detail & Related papers  (2021-08-26T14:01:04Z)
• Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise [51.080620762639434]
  We propose a more efficient parameterization of the posterior approximation for sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks.
  arXiv  Detail & Related papers  (2021-03-15T16:16:18Z)
• Learning Generative Models using Denoising Density Estimators [29.068491722778827]
  We introduce a new generative model based on denoising density estimators (DDEs) Our main contribution is a novel technique to obtain generative models by minimizing the KL-divergence directly. Experimental results demonstrate substantial improvement in density estimation and competitive performance in generative model training.
  arXiv  Detail & Related papers  (2020-01-08T20:30:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.