OT-Net: A Reusable Neural Optimal Transport Solver
- URL: http://arxiv.org/abs/2306.08233v2
- Date: Sun, 12 Nov 2023 06:24:25 GMT
- Title: OT-Net: A Reusable Neural Optimal Transport Solver
- Authors: Zezeng Li, Shenghao Li, Lianbao Jin, Na Lei, Zhongxuan Luo
- Abstract summary: A novel reusable neural OT solver OT-Net is presented.
OT-Net learns Brenier's height representation via the neural network to obtain its potential.
It then gained the OT map by computing the gradient of the potential.
- Score: 26.153287448650126
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: With the widespread application of optimal transport (OT), its calculation
becomes essential, and various algorithms have emerged. However, the existing
methods either have low efficiency or cannot represent discontinuous maps. A
novel reusable neural OT solver OT-Net is thus presented, which first learns
Brenier's height representation via the neural network to obtain its potential,
and then gained the OT map by computing the gradient of the potential. The
algorithm has two merits, 1) it can easily represent discontinuous maps, which
allows it to match any target distribution with discontinuous supports and
achieve sharp boundaries. This can well eliminate mode collapse in the
generated models. 2) The OT map can be calculated straightly by the proposed
algorithm when new target samples are added, which greatly improves the
efficiency and reusability of the map. Moreover, the theoretical error bound of
the algorithm is analyzed, and we have demonstrated the empirical success of
our approach in image generation, color transfer, and domain adaptation.
Related papers
- Efficient Neural Network Approaches for Conditional Optimal Transport with Applications in Bayesian Inference [1.740133468405535]
We present two neural network approaches that approximate the solutions of static and conditional optimal transport (COT) problems.
We demonstrate both algorithms, comparing them with competing state-the-art approaches, using benchmark datasets and simulation-based inverse problems.
arXiv Detail & Related papers (2023-10-25T20:20:09Z) - Unfolded proximal neural networks for robust image Gaussian denoising [7.018591019975253]
We propose a unified framework to build PNNs for the Gaussian denoising task, based on both the dual-FB and the primal-dual Chambolle-Pock algorithms.
We also show that accelerated versions of these algorithms enable skip connections in the associated NN layers.
arXiv Detail & Related papers (2023-08-06T15:32:16Z) - Globally Optimal Training of Neural Networks with Threshold Activation
Functions [63.03759813952481]
We study weight decay regularized training problems of deep neural networks with threshold activations.
We derive a simplified convex optimization formulation when the dataset can be shattered at a certain layer of the network.
arXiv Detail & Related papers (2023-03-06T18:59:13Z) - Sparsity-Constrained Optimal Transport [27.76137474217754]
Regularized optimal transport is now increasingly used as a loss or as a matching layer in neural networks.
We propose a new approach for OT with explicit cardinality constraints on the transportation plan.
Our method can be thought as a middle ground between unregularized OT (recovered in the case $k$) and quadratically-regularized OT (recovered when $k$ is large enough)
arXiv Detail & Related papers (2022-09-30T13:39:47Z) - Multi-Robot Active Mapping via Neural Bipartite Graph Matching [49.72892929603187]
We study the problem of multi-robot active mapping, which aims for complete scene map construction in minimum time steps.
The key to this problem lies in the goal position estimation to enable more efficient robot movements.
We propose a novel algorithm, namely NeuralCoMapping, which takes advantage of both approaches.
arXiv Detail & Related papers (2022-03-30T14:03:17Z) - Algorithms for Efficiently Learning Low-Rank Neural Networks [12.916132936159713]
We study algorithms for learning low-rank neural networks.
We present a provably efficient algorithm which learns an optimal low-rank approximation to a single-hidden-layer ReLU network.
We propose a novel low-rank framework for training low-rank $textitdeep$ networks.
arXiv Detail & Related papers (2022-02-02T01:08:29Z) - Near-optimal estimation of smooth transport maps with kernel
sums-of-squares [81.02564078640275]
Under smoothness conditions, the squared Wasserstein distance between two distributions could be efficiently computed with appealing statistical error upper bounds.
The object of interest for applications such as generative modeling is the underlying optimal transport map.
We propose the first tractable algorithm for which the statistical $L2$ error on the maps nearly matches the existing minimax lower-bounds for smooth map estimation.
arXiv Detail & Related papers (2021-12-03T13:45:36Z) - Generative Modeling with Optimal Transport Maps [83.59805931374197]
Optimal Transport (OT) has become a powerful tool for large-scale generative modeling tasks.
We show that the OT map itself can be used as a generative model, providing comparable performance.
arXiv Detail & Related papers (2021-10-06T18:17:02Z) - Unfolding Projection-free SDP Relaxation of Binary Graph Classifier via
GDPA Linearization [59.87663954467815]
Algorithm unfolding creates an interpretable and parsimonious neural network architecture by implementing each iteration of a model-based algorithm as a neural layer.
In this paper, leveraging a recent linear algebraic theorem called Gershgorin disc perfect alignment (GDPA), we unroll a projection-free algorithm for semi-definite programming relaxation (SDR) of a binary graph.
Experimental results show that our unrolled network outperformed pure model-based graph classifiers, and achieved comparable performance to pure data-driven networks but using far fewer parameters.
arXiv Detail & Related papers (2021-09-10T07:01:15Z) - Lagrangian Decomposition for Neural Network Verification [148.0448557991349]
A fundamental component of neural network verification is the computation of bounds on the values their outputs can take.
We propose a novel approach based on Lagrangian Decomposition.
We show that we obtain bounds comparable with off-the-shelf solvers in a fraction of their running time.
arXiv Detail & Related papers (2020-02-24T17:55:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.