SD-KDE: Score-Debiased Kernel Density Estimation

• URL: http://arxiv.org/abs/2504.19084v2
• Date: Sun, 22 Jun 2025 17:35:36 GMT
• Title: SD-KDE: Score-Debiased Kernel Density Estimation
• Abstract summary: We propose a novel method for density estimation that leverages an estimated score function to debias kernel density estimation.<n>Our experiments on synthetic tasks in 1D, 2D and on MNIST, demonstrate that our proposed SD-KDE method significantly reduces the mean integrated squared error.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel method for density estimation that leverages an estimated score function to debias kernel density estimation (SD-KDE). In our approach, each data point is adjusted by taking a single step along the score function with a specific choice of step size, followed by standard KDE with a modified bandwidth. The step size and modified bandwidth are chosen to remove the leading order bias in the KDE. Our experiments on synthetic tasks in 1D, 2D and on MNIST, demonstrate that our proposed SD-KDE method significantly reduces the mean integrated squared error compared to the standard Silverman KDE, even with noisy estimates in the score function. These results underscore the potential of integrating score-based corrections into nonparametric density estimation.

Related papers

• Geometry-Aware Bayesian Quantification via Compositional Data Analysis [46.938595111911845]
  We introduce a geometry-aware model for multiclass quantification based on log-ratio representations and Aitchison geometry.<n>We derive both point-estimation and Bayesian inference procedures for class prevalences.
  arXiv  Detail & Related papers  (2026-07-06T12:12:16Z)
• Flash-SD-KDE: Accelerating SD-KDE with Tensor Cores [2.421459418045937]
  We show that by re-ordering the SD-KDE to expose matrix-multiplication structure, Cores can be used to accelerate the GPU implementation.<n>On a 1M-sample 16-dimensional task evaluated on 131k queries, Flash-SD-KDE completes in $ s on a single GPU, making score-debiased density estimation practical at previously infeasible scales.
  arXiv  Detail & Related papers  (2026-02-10T23:56:03Z)
• From Kernels to Attention: A Transformer Framework for Density and Score Estimation [0.47745223151611654]
  We introduce a unified attention-based framework for joint score and density estimation.<n>We develop a permutation- and affine-equivariant transformer that estimates both the probability density $f(x)$ and its score $nabla_x log f(x)$ directly from i.i.d. samples.
  arXiv  Detail & Related papers  (2025-11-08T08:38:37Z)
• Differentially Private Gradient Flow based on the Sliced Wasserstein Distance [59.1056830438845]
  We introduce a novel differentially private generative modeling approach based on a gradient flow in the space of probability measures. Experiments show that our proposed model can generate higher-fidelity data at a low privacy budget.
  arXiv  Detail & Related papers  (2023-12-13T15:47:30Z)
• Deep Equilibrium Based Neural Operators for Steady-State PDEs [100.88355782126098]
  We study the benefits of weight-tied neural network architectures for steady-state PDEs. We propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE.
  arXiv  Detail & Related papers  (2023-11-30T22:34:57Z)
• Variational Weighting for Kernel Density Ratios [11.555375654882525]
  Kernel density estimation (KDE) is integral to a range of generative and discriminative tasks in machine learning. We derive an optimal weight function that reduces bias in standard kernel density estimates for density ratios, leading to improved estimates of prediction posteriors and information-theoretic measures.
  arXiv  Detail & Related papers  (2023-11-06T10:12:19Z)
• Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction [82.72686460985297]
  We tackle the problem of estimating a Manhattan frame. We derive two new 2-line solvers, one of which does not suffer from singularities affecting existing solvers. We also design a new non-minimal method, running on an arbitrary number of lines, to boost the performance in local optimization.
  arXiv  Detail & Related papers  (2023-08-21T13:03:25Z)
• Sobolev Space Regularised Pre Density Models [51.558848491038916]
  We propose a new approach to non-parametric density estimation that is based on regularizing a Sobolev norm of the density. This method is statistically consistent, and makes the inductive validation model clear and consistent.
  arXiv  Detail & Related papers  (2023-07-25T18:47:53Z)
• Fast Private Kernel Density Estimation via Locality Sensitive Quantization [10.227538355037554]
  We study efficient mechanisms for differentially private kernel density estimation (DP-KDE) We show how the kernel can privately be approximated in time linear in $d$, making it feasible for high-dimensional data.
  arXiv  Detail & Related papers  (2023-07-04T18:48:04Z)
• Concrete Score Matching: Generalized Score Matching for Discrete Data [109.12439278055213]
  "Concrete score" is a generalization of the (Stein) score for discrete settings. "Concrete Score Matching" is a framework to learn such scores from samples.
  arXiv  Detail & Related papers  (2022-11-02T00:41:37Z)
• Fast Kernel Density Estimation with Density Matrices and Random Fourier Features [0.0]
  kernels density estimation (KDE) is one of the most widely used nonparametric density estimation methods. DMKDE uses density matrices, a quantum mechanical formalism, and random Fourier features, an explicit kernel approximation, to produce density estimates. DMKDE is on par with its competitors for computing density estimates and advantages are shown when performed on high-dimensional data.
  arXiv  Detail & Related papers  (2022-08-02T02:11:10Z)
• Density Ratio Estimation via Infinitesimal Classification [85.08255198145304]
  We propose DRE-infty, a divide-and-conquer approach to reduce Density ratio estimation (DRE) to a series of easier subproblems. Inspired by Monte Carlo methods, we smoothly interpolate between the two distributions via an infinite continuum of intermediate bridge distributions. We show that our approach performs well on downstream tasks such as mutual information estimation and energy-based modeling on complex, high-dimensional datasets.
  arXiv  Detail & Related papers  (2021-11-22T06:26:29Z)
• Generative Adversarial Learning via Kernel Density Discrimination [32.91091065436645]
  We introduce Kernel Density Discrimination GAN (KDD GAN), a novel method for generative adversarial learning. We define the Kernel Density Estimates directly in feature space and forgo the requirement of invertibility of the kernel feature mappings. We show a boost in the quality of generated samples with respect to FID from 10% to 40% compared to the baseline.
  arXiv  Detail & Related papers  (2021-07-13T15:52:10Z)
• Projected Wasserstein gradient descent for high-dimensional Bayesian inference [8.750791391857264]
  We propose a projected Wasserstein gradient descent method (pWGD) for high-dimensional Bayesian inference problems. We overcome this challenge by exploiting the intrinsic low-rank structure in the difference between the posterior and prior distributions.
  arXiv  Detail & Related papers  (2021-02-12T05:12:05Z)
• Nonparametric Density Estimation from Markov Chains [68.8204255655161]
  We introduce a new nonparametric density estimator inspired by Markov Chains, and generalizing the well-known Kernel Density Estor. Our estimator presents several benefits with respect to the usual ones and can be used straightforwardly as a foundation in all density-based algorithms.
  arXiv  Detail & Related papers  (2020-09-08T18:33:42Z)
• Kernel Stein Generative Modeling [68.03537693810972]
  Gradient Langevin Dynamics (SGLD) demonstrates impressive results with energy-based models on high-dimensional and complex data distributions. Stein Variational Gradient Descent (SVGD) is a deterministic sampling algorithm that iteratively transports a set of particles to approximate a given distribution. We propose noise conditional kernel SVGD (NCK-SVGD), that works in tandem with the recently introduced Noise Conditional Score Network estimator.
  arXiv  Detail & Related papers  (2020-07-06T21:26:04Z)
• GradientDICE: Rethinking Generalized Offline Estimation of Stationary Values [75.17074235764757]
  We present GradientDICE for estimating the density ratio between the state distribution of the target policy and the sampling distribution. GenDICE is the state-of-the-art for estimating such density ratios.
  arXiv  Detail & Related papers  (2020-01-29T22:10:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.