A Unified MDL-based Binning and Tensor Factorization Framework for PDF Estimation

• URL: http://arxiv.org/abs/2504.18686v1
• Date: Fri, 25 Apr 2025 20:27:04 GMT
• Title: A Unified MDL-based Binning and Tensor Factorization Framework for PDF Estimation
• Authors: Mustafa Musab, Joseph K. Chege, Arie Yeredor, Martin Haardt,
• Abstract summary: We present a novel non-parametric approach for multivariate probability density function estimation (PDF)<n>Our approach builds upon tensor factorization techniques, leveraging the canonical polyadic decomposition (CPD) of a joint probability tensor.<n>We demonstrate the effectiveness of our method on synthetic data and a challenging real dry bean classification dataset.
• Score: 16.147973439788856
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reliable density estimation is fundamental for numerous applications in statistics and machine learning. In many practical scenarios, data are best modeled as mixtures of component densities that capture complex and multimodal patterns. However, conventional density estimators based on uniform histograms often fail to capture local variations, especially when the underlying distribution is highly nonuniform. Furthermore, the inherent discontinuity of histograms poses challenges for tasks requiring smooth derivatives, such as gradient-based optimization, clustering, and nonparametric discriminant analysis. In this work, we present a novel non-parametric approach for multivariate probability density function (PDF) estimation that utilizes minimum description length (MDL)-based binning with quantile cuts. Our approach builds upon tensor factorization techniques, leveraging the canonical polyadic decomposition (CPD) of a joint probability tensor. We demonstrate the effectiveness of our method on synthetic data and a challenging real dry bean classification dataset.

Related papers

• Learning Controlled Stochastic Differential Equations [61.82896036131116]
  This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled differential equations with non-uniform diffusion. We provide strong theoretical guarantees, including finite-sample bounds for (L2), (Linfty), and risk metrics, with learning rates adaptive to coefficients' regularity. Our method is available as an open-source Python library.
  arXiv  Detail & Related papers  (2024-11-04T11:09:58Z)
• Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
  We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data. Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
  arXiv  Detail & Related papers  (2024-04-24T09:04:36Z)
• Empirical Density Estimation based on Spline Quasi-Interpolation with applications to Copulas clustering modeling [0.0]
  Density estimation is a fundamental technique employed in various fields to model and to understand the underlying distribution of data. In this paper we propose the mono-variate approximation of the density using quasi-interpolation. The presented algorithm is validated on artificial and real datasets.
  arXiv  Detail & Related papers  (2024-02-18T11:49:38Z)
• Partially factorized variational inference for high-dimensional mixed models [0.0]
  Variational inference is a popular way to perform such computations, especially in the Bayesian context.<n>We show that standard mean-field variational inference dramatically underestimates posterior uncertainty in high-dimensions.<n>We then show how appropriately relaxing the mean-field assumption leads to methods whose uncertainty quantification does not deteriorate in high-dimensions.
  arXiv  Detail & Related papers  (2023-12-20T16:12:37Z)
• Compound Batch Normalization for Long-tailed Image Classification [77.42829178064807]
  We propose a compound batch normalization method based on a Gaussian mixture. It can model the feature space more comprehensively and reduce the dominance of head classes. The proposed method outperforms existing methods on long-tailed image classification.
  arXiv  Detail & Related papers  (2022-12-02T07:31:39Z)
• Moment Estimation for Nonparametric Mixture Models Through Implicit Tensor Decomposition [7.139680863764187]
  We present an alternating least squares type numerical optimization scheme to estimate conditionally-independent mixture models in $mathbbRn$. We compute the cumulative distribution functions, higher moments and other statistics of the component distributions through linear solves. Numerical experiments demonstrate the competitive performance of the algorithm, and its applicability to many models and applications.
  arXiv  Detail & Related papers  (2022-10-25T23:31:33Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• A Unified Framework for Multi-distribution Density Ratio Estimation [101.67420298343512]
  Binary density ratio estimation (DRE) provides the foundation for many state-of-the-art machine learning algorithms. We develop a general framework from the perspective of Bregman minimization divergence. We show that our framework leads to methods that strictly generalize their counterparts in binary DRE.
  arXiv  Detail & Related papers  (2021-12-07T01:23:20Z)
• Low-rank statistical finite elements for scalable model-data synthesis [0.8602553195689513]
  statFEM acknowledges a priori model misspecification, by embedding forcing within the governing equations. The method reconstructs the observed data-generating processes with minimal loss of information. This article overcomes this hurdle by embedding a low-rank approximation of the underlying dense covariance matrix.
  arXiv  Detail & Related papers  (2021-09-10T09:51:43Z)
• Marginalizable Density Models [14.50261153230204]
  We present a novel deep network architecture which provides closed form expressions for the probabilities, marginals and conditionals of any subset of the variables. The model also allows for parallelized sampling with only a logarithmic dependence of the time complexity on the number of variables.
  arXiv  Detail & Related papers  (2021-06-08T23:54:48Z)
• Learning with Density Matrices and Random Features [44.98964870180375]
  A density matrix describes the statistical state of a quantum system. It is a powerful formalism to represent both the quantum and classical uncertainty of quantum systems. This paper explores how density matrices can be used as a building block for machine learning models.
  arXiv  Detail & Related papers  (2021-02-08T17:54:59Z)
• Instability, Computational Efficiency and Statistical Accuracy [101.32305022521024]
  We develop a framework that yields statistical accuracy based on interplay between the deterministic convergence rate of the algorithm at the population level, and its degree of (instability) when applied to an empirical object based on $n$ samples. We provide applications of our general results to several concrete classes of models, including Gaussian mixture estimation, non-linear regression models, and informative non-response models.
  arXiv  Detail & Related papers  (2020-05-22T22:30:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.