Sequential transport maps using SoS density estimation and
$\alpha$-divergences
- URL: http://arxiv.org/abs/2402.17943v1
- Date: Tue, 27 Feb 2024 23:52:58 GMT
- Title: Sequential transport maps using SoS density estimation and
$\alpha$-divergences
- Authors: Benjamin Zanger, Tiangang Cui, Martin Schreiber, Olivier Zahm
- Abstract summary: Transport-based density estimation methods are receiving growing interest because of their ability to efficiently generate samples from the approximated density.
We build on a sequence of composed Knothe-Rosenblatt (KR) maps and explore the use of Sum-of-Squareimats (SoS) densities and $alpha$-divergences for approxing the intermediate densities.
We numerically demonstrate our methods on several benchmarks, including Bayesian inference problems and unsupervised learning task.
- Score: 0.6554326244334866
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Transport-based density estimation methods are receiving growing interest
because of their ability to efficiently generate samples from the approximated
density. We further invertigate the sequential transport maps framework
proposed from arXiv:2106.04170 arXiv:2303.02554, which builds on a sequence of
composed Knothe-Rosenblatt (KR) maps. Each of those maps are built by first
estimating an intermediate density of moderate complexity, and then by
computing the exact KR map from a reference density to the precomputed
approximate density. In our work, we explore the use of Sum-of-Squares (SoS)
densities and $\alpha$-divergences for approximating the intermediate
densities. Combining SoS densities with $\alpha$-divergence interestingly
yields convex optimization problems which can be efficiently solved using
semidefinite programming. The main advantage of $\alpha$-divergences is to
enable working with unnormalized densities, which provides benefits both
numerically and theoretically. In particular, we provide two new convergence
analyses of the sequential transport maps: one based on a triangle-like
inequality and the second on information geometric properties of
$\alpha$-divergences for unnormalizied densities. The choice of intermediate
densities is also crucial for the efficiency of the method. While tempered (or
annealed) densities are the state-of-the-art, we introduce diffusion-based
intermediate densities which permits to approximate densities known from
samples only. Such intermediate densities are well-established in machine
learning for generative modeling. Finally we propose and try different
low-dimensional maps (or lazy maps) for dealing with high-dimensional problems
and numerically demonstrate our methods on several benchmarks, including
Bayesian inference problems and unsupervised learning task.
Related papers
- Alignment of Density Maps in Wasserstein Distance [8.140400570642438]
We propose an algorithm for aligning three-dimensional objects when represented as density maps, motivated by applications in cryogenic electron microscopy.
The algorithm is based on minimizing the 1-Wasserstein distance between the density maps after a rigid transformation.
arXiv Detail & Related papers (2023-05-21T01:13:43Z) - Linearized Wasserstein dimensionality reduction with approximation
guarantees [65.16758672591365]
LOT Wassmap is a computationally feasible algorithm to uncover low-dimensional structures in the Wasserstein space.
We show that LOT Wassmap attains correct embeddings and that the quality improves with increased sample size.
We also show how LOT Wassmap significantly reduces the computational cost when compared to algorithms that depend on pairwise distance computations.
arXiv Detail & Related papers (2023-02-14T22:12:16Z) - Gaussian process regression and conditional Karhunen-Lo\'{e}ve models
for data assimilation in inverse problems [68.8204255655161]
We present a model inversion algorithm, CKLEMAP, for data assimilation and parameter estimation in partial differential equation models.
The CKLEMAP method provides better scalability compared to the standard MAP method.
arXiv Detail & Related papers (2023-01-26T18:14:12Z) - Rethinking Spatial Invariance of Convolutional Networks for Object
Counting [119.83017534355842]
We try to use locally connected Gaussian kernels to replace the original convolution filter to estimate the spatial position in the density map.
Inspired by previous work, we propose a low-rank approximation accompanied with translation invariance to favorably implement the approximation of massive Gaussian convolution.
Our methods significantly outperform other state-of-the-art methods and achieve promising learning of the spatial position of objects.
arXiv Detail & Related papers (2022-06-10T17:51:25Z) - Near-optimal estimation of smooth transport maps with kernel
sums-of-squares [81.02564078640275]
Under smoothness conditions, the squared Wasserstein distance between two distributions could be efficiently computed with appealing statistical error upper bounds.
The object of interest for applications such as generative modeling is the underlying optimal transport map.
We propose the first tractable algorithm for which the statistical $L2$ error on the maps nearly matches the existing minimax lower-bounds for smooth map estimation.
arXiv Detail & Related papers (2021-12-03T13:45:36Z) - Density Ratio Estimation via Infinitesimal Classification [85.08255198145304]
We propose DRE-infty, a divide-and-conquer approach to reduce Density ratio estimation (DRE) to a series of easier subproblems.
Inspired by Monte Carlo methods, we smoothly interpolate between the two distributions via an infinite continuum of intermediate bridge distributions.
We show that our approach performs well on downstream tasks such as mutual information estimation and energy-based modeling on complex, high-dimensional datasets.
arXiv Detail & Related papers (2021-11-22T06:26:29Z) - Density-Based Clustering with Kernel Diffusion [59.4179549482505]
A naive density corresponding to the indicator function of a unit $d$-dimensional Euclidean ball is commonly used in density-based clustering algorithms.
We propose a new kernel diffusion density function, which is adaptive to data of varying local distributional characteristics and smoothness.
arXiv Detail & Related papers (2021-10-11T09:00:33Z) - Generative Learning With Euler Particle Transport [14.557451744544592]
We propose an Euler particle transport (EPT) approach for generative learning.
The proposed approach is motivated by the problem of finding an optimal transport map from a reference distribution to a target distribution.
We show that the proposed density-ratio (difference) estimators do not suffer from the "curse of dimensionality" if data is supported on a lower-dimensional manifold.
arXiv Detail & Related papers (2020-12-11T03:10:53Z) - Nearest Neighbor Dirichlet Mixtures [3.3194866396158]
We propose a class of nearest neighbor-Dirichlet mixtures to maintain most of the strengths of Bayesian approaches without the computational disadvantages.
A simple and embarrassingly parallel Monte Carlo algorithm is proposed to sample from the resulting pseudo-posterior for the unknown density.
arXiv Detail & Related papers (2020-03-17T21:39:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.