On a Relation Between the Rate-Distortion Function and Optimal Transport
- URL: http://arxiv.org/abs/2307.00246v1
- Date: Sat, 1 Jul 2023 06:20:23 GMT
- Title: On a Relation Between the Rate-Distortion Function and Optimal Transport
- Authors: Eric Lei, Hamed Hassani, Shirin Saeedi Bidokhti
- Abstract summary: We show that a function defined via an extremal entropic OT distance is equivalent to the rate-distortion function.
We numerically verify this result as well as previous results that connect the Monge and Kantorovich problems to optimal scalar quantization.
- Score: 25.59334941818991
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We discuss a relationship between rate-distortion and optimal transport (OT)
theory, even though they seem to be unrelated at first glance. In particular,
we show that a function defined via an extremal entropic OT distance is
equivalent to the rate-distortion function. We numerically verify this result
as well as previous results that connect the Monge and Kantorovich problems to
optimal scalar quantization. Thus, we unify solving scalar quantization and
rate-distortion functions in an alternative fashion by using their respective
optimal transport solvers.
Related papers
- Variance-Reducing Couplings for Random Features: Perspectives from Optimal Transport [57.73648780299374]
Random features (RFs) are a popular technique to scale up kernel methods in machine learning, replacing exact kernel evaluations with Monte Carlo estimates.
We tackle this through the unifying framework of optimal transport, using theoretical insights and numerical algorithms to develop novel, high-performing RF couplings for kernels defined on Euclidean and discrete input spaces.
We reach surprising conclusions about the benefits and limitations of variance reduction as a paradigm.
arXiv Detail & Related papers (2024-05-26T12:25:09Z) - Conditional Optimal Transport on Function Spaces [53.9025059364831]
We develop a theory of constrained optimal transport problems that describe block-triangular Monge maps.
This generalizes the theory of optimal triangular transport to separable infinite-dimensional function spaces with general cost functions.
We present numerical experiments that demonstrate the computational applicability of our theoretical results for amortized and likelihood-free inference of functional parameters.
arXiv Detail & Related papers (2023-11-09T18:44:42Z) - Learning Unnormalized Statistical Models via Compositional Optimization [73.30514599338407]
Noise-contrastive estimation(NCE) has been proposed by formulating the objective as the logistic loss of the real data and the artificial noise.
In this paper, we study it a direct approach for optimizing the negative log-likelihood of unnormalized models.
arXiv Detail & Related papers (2023-06-13T01:18:16Z) - On the Convergence of Coordinate Ascent Variational Inference [11.166959724276337]
We consider the common coordinate ascent variational inference (CAVI) algorithm for implementing the mean-field (MF) VI.
We provide general conditions for certifying global or local exponential convergence of CAVI.
New notion of generalized correlation for characterizing the interaction between the constituting blocks in influencing the VI objective functional is introduced.
arXiv Detail & Related papers (2023-06-01T20:19:30Z) - Data-Driven Influence Functions for Optimization-Based Causal Inference [105.5385525290466]
We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing.
We study the case where probability distributions are not known a priori but need to be estimated from data.
arXiv Detail & Related papers (2022-08-29T16:16:22Z) - Error Analysis of Tensor-Train Cross Approximation [88.83467216606778]
We provide accuracy guarantees in terms of the entire tensor for both exact and noisy measurements.
Results are verified by numerical experiments, and may have important implications for the usefulness of cross approximations for high-order tensors.
arXiv Detail & Related papers (2022-07-09T19:33:59Z) - An improved central limit theorem and fast convergence rates for
entropic transportation costs [13.9170193921377]
We prove a central limit theorem for the entropic transportation cost between subgaussian probability measures.
We complement these results with new, faster, convergence rates for the expected entropic transportation cost between empirical measures.
arXiv Detail & Related papers (2022-04-19T19:26:59Z) - Nearly Tight Convergence Bounds for Semi-discrete Entropic Optimal
Transport [0.483420384410068]
We derive nearly tight and non-asymptotic convergence bounds for solutions of entropic semi-discrete optimal transport.
Our results also entail a non-asymptotic and tight expansion of the difference between the entropic and the unregularized costs.
arXiv Detail & Related papers (2021-10-25T06:52:45Z) - Variational Transport: A Convergent Particle-BasedAlgorithm for Distributional Optimization [106.70006655990176]
A distributional optimization problem arises widely in machine learning and statistics.
We propose a novel particle-based algorithm, dubbed as variational transport, which approximately performs Wasserstein gradient descent.
We prove that when the objective function satisfies a functional version of the Polyak-Lojasiewicz (PL) (Polyak, 1963) and smoothness conditions, variational transport converges linearly.
arXiv Detail & Related papers (2020-12-21T18:33:13Z) - Inverses of Matern Covariances on Grids [0.0]
We study the properties of a popular approximation based on partial differential equations on a regular grid of points.
We find that it assigns too much power at high frequencies and does not provide increasingly accurate approximations to the inverse as the grid spacing goes to zero.
In a simulation study, we investigate the implications for parameter estimation, finding that the SPDE approximation tends to overestimate spatial range parameters.
arXiv Detail & Related papers (2019-12-26T18:36:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.