Score-Based Density Estimation from Pairwise Comparisons

• URL: http://arxiv.org/abs/2510.09146v1
• Date: Fri, 10 Oct 2025 08:49:24 GMT
• Title: Score-Based Density Estimation from Pairwise Comparisons
• Authors: Petrus Mikkola, Luigi Acerbi, Arto Klami,
• Abstract summary: We study density estimation from pairwise comparisons, motivated by expert knowledge elicitation and learning from human feedback.<n>We relate the unobserved target density to a tempered winner density, learning the winner's score via score-matching.<n>We prove that the score vectors of the belief and the winner density are collinear, linked by a position-dependent tempering field.
• Score: 13.996217500923414
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study density estimation from pairwise comparisons, motivated by expert knowledge elicitation and learning from human feedback. We relate the unobserved target density to a tempered winner density (marginal density of preferred choices), learning the winner's score via score-matching. This allows estimating the target by `de-tempering' the estimated winner density's score. We prove that the score vectors of the belief and the winner density are collinear, linked by a position-dependent tempering field. We give analytical formulas for this field and propose an estimator for it under the Bradley-Terry model. Using a diffusion model trained on tempered samples generated via score-scaled annealed Langevin dynamics, we can learn complex multivariate belief densities of simulated experts, from only hundreds to thousands of pairwise comparisons.

Related papers

• Flow-Based Density Ratio Estimation for Intractable Distributions with Applications in Genomics [80.05951561886123]
  We leverage condition-aware flow matching to derive a single dynamical formulation for tracking density ratios along generative trajectories.<n>We demonstrate competitive performance on simulated benchmarks for closed-form ratio estimation, and show that our method supports versatile tasks in single-cell genomics data analysis.
  arXiv  Detail & Related papers  (2026-02-27T17:27:55Z)
• Density Ratio Estimation with Conditional Probability Paths [14.251729168309067]
  We introduce a novel framework for time score estimation, based on a conditioning variable.<n>We demonstrate that, compared to previous approaches, our approach results in faster learning of the time score and competitive or better estimation accuracies of the density ratio.
  arXiv  Detail & Related papers  (2025-02-04T13:13:35Z)
• Your copula is a classifier in disguise: classification-based copula density estimation [2.5261465733373965]
  We propose reinterpreting copula density estimation as a discriminative task.<n>We derive equivalences between well-known copula classes and classification problems naturally arising in our interpretation.<n>We show our estimator achieves theoretical guarantees akin to maximum likelihood estimation.
  arXiv  Detail & Related papers  (2024-11-05T11:25:34Z)
• Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
  We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data. Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
  arXiv  Detail & Related papers  (2024-04-24T09:04:36Z)
• Investigating the Adversarial Robustness of Density Estimation Using the Probability Flow ODE [4.7818621660181595]
  We introduce and evaluate six gradient-based log-likelihood attacks, including a novel reverse integration attack. Our experimental evaluations on CIFAR-10 show that density estimation using the PF ODE is robust against high-complexity, high-likelihood attacks, and that in some cases adversarial samples are semantically meaningful, as expected from a robust estimator.
  arXiv  Detail & Related papers  (2023-10-10T23:58:53Z)
• $CrowdDiff$: Multi-hypothesis Crowd Density Estimation using Diffusion Models [26.55769846846542]
  Crowd counting is a fundamental problem in crowd analysis which is typically accomplished by estimating a crowd density map and summing over the density values. We present $CrowdDiff$ that generates the crowd density map as a reverse diffusion process. In addition, owing to the nature of the diffusion model, we introduce producing multiple density maps to improve the counting performance.
  arXiv  Detail & Related papers  (2023-03-22T17:58:01Z)
• Learning Transfer Operators by Kernel Density Estimation [0.0]
  We recast the problem within the framework of statistical density estimation. We demonstrate the validity and effectiveness of this approach in estimating the eigenvectors of the Frobenius-Perron operator. We suggest the possibility of incorporating other density estimation methods into this field.
  arXiv  Detail & Related papers  (2022-08-01T14:28:10Z)
• Density Ratio Estimation via Infinitesimal Classification [85.08255198145304]
  We propose DRE-infty, a divide-and-conquer approach to reduce Density ratio estimation (DRE) to a series of easier subproblems. Inspired by Monte Carlo methods, we smoothly interpolate between the two distributions via an infinite continuum of intermediate bridge distributions. We show that our approach performs well on downstream tasks such as mutual information estimation and energy-based modeling on complex, high-dimensional datasets.
  arXiv  Detail & Related papers  (2021-11-22T06:26:29Z)
• Featurized Density Ratio Estimation [82.40706152910292]
  In our work, we propose to leverage an invertible generative model to map the two distributions into a common feature space prior to estimation. This featurization brings the densities closer together in latent space, sidestepping pathological scenarios where the learned density ratios in input space can be arbitrarily inaccurate. At the same time, the invertibility of our feature map guarantees that the ratios computed in feature space are equivalent to those in input space.
  arXiv  Detail & Related papers  (2021-07-05T18:30:26Z)
• Imitation with Neural Density Models [98.34503611309256]
  We propose a new framework for Imitation Learning (IL) via density estimation of the expert's occupancy measure followed by Imitation Occupancy Entropy Reinforcement Learning (RL) using the density as a reward. Our approach maximizes a non-adversarial model-free RL objective that provably lower bounds reverse Kullback-Leibler divergence between occupancy measures of the expert and imitator.
  arXiv  Detail & Related papers  (2020-10-19T19:38:36Z)
• Nonparametric Density Estimation from Markov Chains [68.8204255655161]
  We introduce a new nonparametric density estimator inspired by Markov Chains, and generalizing the well-known Kernel Density Estor. Our estimator presents several benefits with respect to the usual ones and can be used straightforwardly as a foundation in all density-based algorithms.
  arXiv  Detail & Related papers  (2020-09-08T18:33:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.