Variational Weighting for Kernel Density Ratios

• URL: http://arxiv.org/abs/2311.03001v1
• Date: Mon, 6 Nov 2023 10:12:19 GMT
• Title: Variational Weighting for Kernel Density Ratios
• Authors: Sangwoong Yoon, Frank C. Park, Gunsu S Yun, Iljung Kim, Yung-Kyun Noh
• Abstract summary: Kernel density estimation (KDE) is integral to a range of generative and discriminative tasks in machine learning. We derive an optimal weight function that reduces bias in standard kernel density estimates for density ratios, leading to improved estimates of prediction posteriors and information-theoretic measures.
• Score: 11.555375654882525
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Kernel density estimation (KDE) is integral to a range of generative and discriminative tasks in machine learning. Drawing upon tools from the multidimensional calculus of variations, we derive an optimal weight function that reduces bias in standard kernel density estimates for density ratios, leading to improved estimates of prediction posteriors and information-theoretic measures. In the process, we shed light on some fundamental aspects of density estimation, particularly from the perspective of algorithms that employ KDEs as their main building blocks.

Related papers

• Density Estimation via Binless Multidimensional Integration [45.21975243399607]
  We introduce the Binless Multidimensional Thermodynamic Integration (BMTI) method for nonparametric, robust, and data-efficient density estimation. BMTI estimates the logarithm of the density by initially computing log-density differences between neighbouring data points. The method is tested on a variety of complex synthetic high-dimensional datasets, and is benchmarked on realistic datasets from the chemical physics literature.
  arXiv  Detail & Related papers  (2024-07-10T23:45:20Z)
• Tracking Dynamic Gaussian Density with a Theoretically Optimal Sliding Window Approach [16.45003200150227]
  We study the exact mean integrated squared error (MISE) of "sliding window" Kernel Density Estimators for evolving densities. We present empirical evidence with synthetic datasets to show that our weighting scheme improves the tracking performance.
  arXiv  Detail & Related papers  (2024-03-11T23:21:26Z)
• Towards Efficient Vision-Language Tuning: More Information Density, More Generalizability [73.34532767873785]
  We propose the concept of Information Density'' (ID) to indicate whether a matrix strongly belongs to certain feature spaces. We introduce the Dense Information Prompt (DIP) to enhance information density to improve generalization. DIP significantly reduces the number of tunable parameters and the requisite storage space, making it particularly advantageous in resource-constrained settings.
  arXiv  Detail & Related papers  (2023-12-17T20:42:43Z)
• Anomaly Detection with Variance Stabilized Density Estimation [49.46356430493534]
  We present a variance-stabilized density estimation problem for maximizing the likelihood of the observed samples. To obtain a reliable anomaly detector, we introduce a spectral ensemble of autoregressive models for learning the variance-stabilized distribution. We have conducted an extensive benchmark with 52 datasets, demonstrating that our method leads to state-of-the-art results.
  arXiv  Detail & Related papers  (2023-06-01T11:52:58Z)
• Fast Kernel Density Estimation with Density Matrices and Random Fourier Features [0.0]
  kernels density estimation (KDE) is one of the most widely used nonparametric density estimation methods. DMKDE uses density matrices, a quantum mechanical formalism, and random Fourier features, an explicit kernel approximation, to produce density estimates. DMKDE is on par with its competitors for computing density estimates and advantages are shown when performed on high-dimensional data.
  arXiv  Detail & Related papers  (2022-08-02T02:11:10Z)
• Learning Transfer Operators by Kernel Density Estimation [0.0]
  We recast the problem within the framework of statistical density estimation. We demonstrate the validity and effectiveness of this approach in estimating the eigenvectors of the Frobenius-Perron operator. We suggest the possibility of incorporating other density estimation methods into this field.
  arXiv  Detail & Related papers  (2022-08-01T14:28:10Z)
• TAKDE: Temporal Adaptive Kernel Density Estimator for Real-Time Dynamic Density Estimation [16.45003200150227]
  We name the temporal adaptive kernel density estimator (TAKDE) TAKDE is theoretically optimal in terms of the worst-case AMISE. We provide numerical experiments using synthetic and real-world datasets, showing that TAKDE outperforms other state-of-the-art dynamic density estimators.
  arXiv  Detail & Related papers  (2022-03-15T23:38:32Z)
• Density-Based Clustering with Kernel Diffusion [59.4179549482505]
  A naive density corresponding to the indicator function of a unit $d$-dimensional Euclidean ball is commonly used in density-based clustering algorithms. We propose a new kernel diffusion density function, which is adaptive to data of varying local distributional characteristics and smoothness.
  arXiv  Detail & Related papers  (2021-10-11T09:00:33Z)
• Featurized Density Ratio Estimation [82.40706152910292]
  In our work, we propose to leverage an invertible generative model to map the two distributions into a common feature space prior to estimation. This featurization brings the densities closer together in latent space, sidestepping pathological scenarios where the learned density ratios in input space can be arbitrarily inaccurate. At the same time, the invertibility of our feature map guarantees that the ratios computed in feature space are equivalent to those in input space.
  arXiv  Detail & Related papers  (2021-07-05T18:30:26Z)
• Nonparametric Density Estimation from Markov Chains [68.8204255655161]
  We introduce a new nonparametric density estimator inspired by Markov Chains, and generalizing the well-known Kernel Density Estor. Our estimator presents several benefits with respect to the usual ones and can be used straightforwardly as a foundation in all density-based algorithms.
  arXiv  Detail & Related papers  (2020-09-08T18:33:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.