Featurized Density Ratio Estimation

• URL: http://arxiv.org/abs/2107.02212v1
• Date: Mon, 5 Jul 2021 18:30:26 GMT
• Title: Featurized Density Ratio Estimation
• Authors: Kristy Choi, Madeline Liao, Stefano Ermon
• Abstract summary: In our work, we propose to leverage an invertible generative model to map the two distributions into a common feature space prior to estimation. This featurization brings the densities closer together in latent space, sidestepping pathological scenarios where the learned density ratios in input space can be arbitrarily inaccurate. At the same time, the invertibility of our feature map guarantees that the ratios computed in feature space are equivalent to those in input space.
• Score: 82.40706152910292
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Density ratio estimation serves as an important technique in the unsupervised machine learning toolbox. However, such ratios are difficult to estimate for complex, high-dimensional data, particularly when the densities of interest are sufficiently different. In our work, we propose to leverage an invertible generative model to map the two distributions into a common feature space prior to estimation. This featurization brings the densities closer together in latent space, sidestepping pathological scenarios where the learned density ratios in input space can be arbitrarily inaccurate. At the same time, the invertibility of our feature map guarantees that the ratios computed in feature space are equivalent to those in input space. Empirically, we demonstrate the efficacy of our approach in a variety of downstream tasks that require access to accurate density ratios such as mutual information estimation, targeted sampling in deep generative models, and classification with data augmentation.

Related papers

• Your copula is a classifier in disguise: classification-based copula density estimation [2.5261465733373965]
  We propose reinterpreting copula density estimation as a discriminative task. We derive equivalences between well-known copula classes and classification problems naturally arising in our interpretation. We show our estimator achieves theoretical guarantees akin to maximum likelihood estimation.
  arXiv  Detail & Related papers  (2024-11-05T11:25:34Z)
• Towards the Uncharted: Density-Descending Feature Perturbation for Semi-supervised Semantic Segmentation [51.66997548477913]
  We propose a novel feature-level consistency learning framework named Density-Descending Feature Perturbation (DDFP) Inspired by the low-density separation assumption in semi-supervised learning, our key insight is that feature density can shed a light on the most promising direction for the segmentation classifier to explore. The proposed DDFP outperforms other designs on feature-level perturbations and shows state of the art performances on both Pascal VOC and Cityscapes dataset.
  arXiv  Detail & Related papers  (2024-03-11T06:59:05Z)
• Nonparametric Probabilistic Regression with Coarse Learners [1.8275108630751844]
  We show that we can compute precise conditional densities with minimal assumptions on the shape or form of the density. We demonstrate this approach on a variety of datasets and show competitive performance, particularly on larger datasets.
  arXiv  Detail & Related papers  (2022-10-28T16:25:26Z)
• Estimating Divergences in High Dimensions [6.172809837529207]
  We propose the use of decomposable models for estimating divergences in high dimensional data. These allow us to factorize the estimated density of the high-dimensional distribution into a product of lower dimensional functions. We show empirically that estimating the Kullback-Leibler divergence using decomposable models from a maximum likelihood estimator outperforms existing methods for divergence estimation.
  arXiv  Detail & Related papers  (2021-12-08T20:37:28Z)
• Density Ratio Estimation via Infinitesimal Classification [85.08255198145304]
  We propose DRE-infty, a divide-and-conquer approach to reduce Density ratio estimation (DRE) to a series of easier subproblems. Inspired by Monte Carlo methods, we smoothly interpolate between the two distributions via an infinite continuum of intermediate bridge distributions. We show that our approach performs well on downstream tasks such as mutual information estimation and energy-based modeling on complex, high-dimensional datasets.
  arXiv  Detail & Related papers  (2021-11-22T06:26:29Z)
• Density-Based Clustering with Kernel Diffusion [59.4179549482505]
  A naive density corresponding to the indicator function of a unit $d$-dimensional Euclidean ball is commonly used in density-based clustering algorithms. We propose a new kernel diffusion density function, which is adaptive to data of varying local distributional characteristics and smoothness.
  arXiv  Detail & Related papers  (2021-10-11T09:00:33Z)
• Meta-Learning for Relative Density-Ratio Estimation [59.75321498170363]
  Existing methods for (relative) density-ratio estimation (DRE) require many instances from both densities. We propose a meta-learning method for relative DRE, which estimates the relative density-ratio from a few instances by using knowledge in related datasets. We empirically demonstrate the effectiveness of the proposed method by using three problems: relative DRE, dataset comparison, and outlier detection.
  arXiv  Detail & Related papers  (2021-07-02T02:13:45Z)
• Learning Optical Flow from a Few Matches [67.83633948984954]
  We show that the dense correlation volume representation is redundant and accurate flow estimation can be achieved with only a fraction of elements in it. Experiments show that our method can reduce computational cost and memory use significantly, while maintaining high accuracy.
  arXiv  Detail & Related papers  (2021-04-05T21:44:00Z)
• High-Dimensional Non-Parametric Density Estimation in Mixed Smooth Sobolev Spaces [31.663702435594825]
  Density estimation plays a key role in many tasks in machine learning, statistical inference, and visualization. Main bottleneck in high-dimensional density estimation is the prohibitive computational cost and the slow convergence rate. We propose novel estimators for high-dimensional non-parametric density estimation called the adaptive hyperbolic cross density estimators.
  arXiv  Detail & Related papers  (2020-06-05T21:27:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.