Robust Multi-Modal Density Estimation

• URL: http://arxiv.org/abs/2401.10566v2
• Date: Mon, 6 May 2024 11:59:53 GMT
• Title: Robust Multi-Modal Density Estimation
• Authors: Anna Mészáros, Julian F. Schumann, Javier Alonso-Mora, Arkady Zgonnikov, Jens Kober,
• Abstract summary: ROME (RObust Multi-modal Estimator) is a non-parametric approach for density estimation. We show that ROME can overcome the issues of over-fitting and over-smoothing exhibited by other estimators.
• Score: 14.643918024937758
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The estimation of probability density functions is a fundamental problem in science and engineering. However, common methods such as kernel density estimation (KDE) have been demonstrated to lack robustness, while more complex methods have not been evaluated in multi-modal estimation problems. In this paper, we present ROME (RObust Multi-modal Estimator), a non-parametric approach for density estimation which addresses the challenge of estimating multi-modal, non-normal, and highly correlated distributions. ROME utilizes clustering to segment a multi-modal set of samples into multiple uni-modal ones and then combines simple KDE estimates obtained for individual clusters in a single multi-modal estimate. We compared our approach to state-of-the-art methods for density estimation as well as ablations of ROME, showing that it not only outperforms established methods but is also more robust to a variety of distributions. Our results demonstrate that ROME can overcome the issues of over-fitting and over-smoothing exhibited by other estimators.

Related papers

• Estimation of multiple mean vectors in high dimension [4.2466572124753]
  We endeavour to estimate numerous multi-dimensional means of various probability distributions on a common space based on independent samples. Our approach involves forming estimators through convex combinations of empirical means derived from these samples.
  arXiv  Detail & Related papers  (2024-03-22T08:42:41Z)
• Kernel Density Estimation for Multiclass Quantification [52.419589623702336]
  Quantification is the supervised machine learning task concerned with obtaining accurate predictors of class prevalence. The distribution-matching (DM) approaches represent one of the most important families among the quantification methods that have been proposed in the literature so far. We propose a new representation mechanism based on multivariate densities that we model via kernel density estimation (KDE)
  arXiv  Detail & Related papers  (2023-12-31T13:19:27Z)
• A Multivariate Unimodality Test Harnessing the Dip Statistic of Mahalanobis Distances Over Random Projections [0.18416014644193066]
  We extend one-dimensional unimodality principles to multi-dimensional spaces through linear random projections and point-to-point distancing. Our method, rooted in $alpha$-unimodality assumptions, presents a novel unimodality test named mud-pod. Both theoretical and empirical studies confirm the efficacy of our method in unimodality assessment of multidimensional datasets.
  arXiv  Detail & Related papers  (2023-11-28T09:11:02Z)
• A Unified Framework for Multi-distribution Density Ratio Estimation [101.67420298343512]
  Binary density ratio estimation (DRE) provides the foundation for many state-of-the-art machine learning algorithms. We develop a general framework from the perspective of Bregman minimization divergence. We show that our framework leads to methods that strictly generalize their counterparts in binary DRE.
  arXiv  Detail & Related papers  (2021-12-07T01:23:20Z)
• Density Ratio Estimation via Infinitesimal Classification [85.08255198145304]
  We propose DRE-infty, a divide-and-conquer approach to reduce Density ratio estimation (DRE) to a series of easier subproblems. Inspired by Monte Carlo methods, we smoothly interpolate between the two distributions via an infinite continuum of intermediate bridge distributions. We show that our approach performs well on downstream tasks such as mutual information estimation and energy-based modeling on complex, high-dimensional datasets.
  arXiv  Detail & Related papers  (2021-11-22T06:26:29Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)
• Meta-Learning for Relative Density-Ratio Estimation [59.75321498170363]
  Existing methods for (relative) density-ratio estimation (DRE) require many instances from both densities. We propose a meta-learning method for relative DRE, which estimates the relative density-ratio from a few instances by using knowledge in related datasets. We empirically demonstrate the effectiveness of the proposed method by using three problems: relative DRE, dataset comparison, and outlier detection.
  arXiv  Detail & Related papers  (2021-07-02T02:13:45Z)
• Multivariate Density Estimation with Deep Neural Mixture Models [0.0]
  Deep neural networks (DNNs) have seldom been applied to density estimation. This paper extends our previous work on Neural Mixture Densities (NMMs) A maximum-likelihood (ML) algorithm for estimating Deep NMMs (DNMMs) is handed out. The class of probability density functions that can be modeled to any degree of precision via DNMMs is formally defined.
  arXiv  Detail & Related papers  (2020-12-06T23:03:48Z)
• Nonparametric Density Estimation from Markov Chains [68.8204255655161]
  We introduce a new nonparametric density estimator inspired by Markov Chains, and generalizing the well-known Kernel Density Estor. Our estimator presents several benefits with respect to the usual ones and can be used straightforwardly as a foundation in all density-based algorithms.
  arXiv  Detail & Related papers  (2020-09-08T18:33:42Z)
• Robust Kernel Density Estimation with Median-of-Means principle [0.0]
  We introduce a robust nonparametric density estimator combining the popular Kernel Density Estimation method and the Median-of-Means principle (MoM-KDE) This estimator is shown to achieve robustness to any kind of anomalous data, even in the case of adversarial contamination.
  arXiv  Detail & Related papers  (2020-06-30T08:01:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.