An Optimal Transport Perspective on Unpaired Image Super-Resolution

• URL: http://arxiv.org/abs/2202.01116v3
• Date: Tue, 08 Jul 2025 14:01:59 GMT
• Title: An Optimal Transport Perspective on Unpaired Image Super-Resolution
• Authors: Milena Gazdieva, Petr Mokrov, Litu Rout, Alexander Korotin, Andrey Kravchenko, Alexander Filippov, Evgeny Burnaev,
• Abstract summary: Real-world image super-resolution (SR) tasks often do not have paired datasets, which limits the application of supervised techniques.<n>We investigate optimization problems which arise in such models and find two surprising observations.<n>We prove and empirically show that the learned map is biased, i.e., it does not actually transform the distribution of low-resolution images to high-resolution ones.
• Score: 97.24140709634203
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world image super-resolution (SR) tasks often do not have paired datasets, which limits the application of supervised techniques. As a result, the tasks are usually approached by unpaired techniques based on Generative Adversarial Networks (GANs), which yield complex training losses with several regularization terms, e.g., content or identity losses. While GANs usually provide good practical performance, they are used heuristically, i.e., theoretical understanding of their behaviour is yet rather limited. We theoretically investigate optimization problems which arise in such models and find two surprising observations. First, the learned SR map is always an optimal transport (OT) map. Second, we theoretically prove and empirically show that the learned map is biased, i.e., it does not actually transform the distribution of low-resolution images to high-resolution ones. Inspired by these findings, we investigate recent advances in neural OT field to resolve the bias issue. We establish an intriguing connection between regularized GANs and neural OT approaches. We show that unlike the existing GAN-based alternatives, these algorithms aim to learn an unbiased OT map. We empirically demonstrate our findings via a series of synthetic and real-world unpaired SR experiments. Our source code is publicly available at https://github.com/milenagazdieva/OT-Super-Resolution.

Related papers

• Efficient Differentiable Approximation of Generalized Low-rank Regularization [64.73416824444328]
  Low-rank regularization (LRR) has been widely applied in various machine learning tasks.<n>In this paper, we propose an efficient differentiable approximation of LRR.
  arXiv  Detail & Related papers  (2025-05-21T11:49:17Z)
• A Statistical Learning Perspective on Semi-dual Adversarial Neural Optimal Transport Solvers [65.28989155951132]
  In this paper, we establish upper bounds on the generalization error of an approximate OT map recovered by the minimax quadratic OT solver.<n>While our analysis focuses on the quadratic OT, we believe that similar bounds could be derived for general OT case, paving the promising direction for future research.
  arXiv  Detail & Related papers  (2025-02-03T12:37:20Z)
• Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution [10.074968164380314]
  Implicit Neural Representation (INR) has been successfully employed for Arbitrary-scale Super-Resolution (ASR) We develop two novel techniques to generalize GS for ASR. We implement an efficient differentiable 2D GPU/CUDA-based scale-awareization to render super-aware images.
  arXiv  Detail & Related papers  (2025-01-12T15:14:58Z)
• Improving Neural Optimal Transport via Displacement Interpolation [16.474572112062535]
  Optimal Transport (OT) theory investigates the cost-minimizing transport map that moves a source distribution to a target distribution. We propose a novel method to improve stability and achieve a better approximation of the OT Map by exploiting displacement. We demonstrate that DIOTM outperforms existing OT-based models on image-to-image translation tasks.
  arXiv  Detail & Related papers  (2024-10-03T16:42:23Z)
• Self-supervised Feature Adaptation for 3D Industrial Anomaly Detection [59.41026558455904]
  We focus on multi-modal anomaly detection. Specifically, we investigate early multi-modal approaches that attempted to utilize models pre-trained on large-scale visual datasets. We propose a Local-to-global Self-supervised Feature Adaptation (LSFA) method to finetune the adaptors and learn task-oriented representation toward anomaly detection.
  arXiv  Detail & Related papers  (2024-01-06T07:30:41Z)
• Generalized Schrödinger Bridge Matching [54.171931505066]
  Generalized Schr"odinger Bridge (GSB) problem setup is prevalent in many scientific areas both within and without machine learning. We propose Generalized Schr"odinger Bridge Matching (GSBM), a new matching algorithm inspired by recent advances. We show that such a generalization can be cast as solving conditional optimal control, for which variational approximations can be used.
  arXiv  Detail & Related papers  (2023-10-03T17:42:11Z)
• OT-Net: A Reusable Neural Optimal Transport Solver [26.153287448650126]
  A novel reusable neural OT solver OT-Net is presented. OT-Net learns Brenier's height representation via the neural network to obtain its potential. It then gained the OT map by computing the gradient of the potential.
  arXiv  Detail & Related papers  (2023-06-14T04:11:38Z)
• Universal Neural Optimal Transport [0.0]
  UNOT (Universal Neural Optimal Transport) is a novel framework capable of accurately predicting (entropic) OT distances and plans between discrete measures for a given cost function.<n>We show that our network can be used as a state-of-the-art initialization for the Sinkhorn algorithm with speedups of up to $7.4times$.
  arXiv  Detail & Related papers  (2022-11-30T21:56:09Z)
• ATASI-Net: An Efficient Sparse Reconstruction Network for Tomographic SAR Imaging with Adaptive Threshold [13.379416816598873]
  This paper proposes a novel efficient sparse unfolding network based on the analytic learned iterative shrinkage thresholding algorithm (ALISTA) The weight matrix in each layer of ATASI-Net is pre-computed as the solution of an off-line optimization problem. In addition, adaptive threshold is introduced for each azimuth-range pixel, enabling the threshold shrinkage to be not only layer-varied but also element-wise.
  arXiv  Detail & Related papers  (2022-11-30T09:55:45Z)
• Perception-Oriented Single Image Super-Resolution using Optimal Objective Estimation [11.830754741007029]
  We propose a new SISR framework that applies optimal objectives for each region to generate plausible results in overall areas of high-resolution outputs. The framework comprises two models: a predictive model that infers an optimal objective map for a given low-resolution (LR) input and a generative model that applies a target objective map to produce the corresponding SR output.
  arXiv  Detail & Related papers  (2022-11-24T15:45:03Z)
• Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution [64.15915577164894]
  A hierarchical image super-resolution network (HSRNet) is proposed to suppress the influence of aliasing. HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.
  arXiv  Detail & Related papers  (2022-06-07T14:55:32Z)
• Generative Modeling with Optimal Transport Maps [83.59805931374197]
  Optimal Transport (OT) has become a powerful tool for large-scale generative modeling tasks. We show that the OT map itself can be used as a generative model, providing comparable performance.
  arXiv  Detail & Related papers  (2021-10-06T18:17:02Z)
• Towards Measuring Bias in Image Classification [61.802949761385]
  Convolutional Neural Networks (CNN) have become state-of-the-art for the main computer vision tasks. However, due to the complex structure their decisions are hard to understand which limits their use in some context of the industrial world. We present a systematic approach to uncover data bias by means of attribution maps.
  arXiv  Detail & Related papers  (2021-07-01T10:50:39Z)
• Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark [133.46066694893318]
  We evaluate the performance of neural network-based solvers for optimal transport. We find that existing solvers do not recover optimal transport maps even though they perform well in downstream tasks.
  arXiv  Detail & Related papers  (2021-06-03T15:59:28Z)
• Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Prior [0.0]
  We address the problem of solving ill-posed inverse problems in imaging where the prior is a JPal autoencoder (VAE) We show that our technique is quite sufficient that it satisfies the proposed objective function. Results also show the robustness of our approach to provide more robust estimates.
  arXiv  Detail & Related papers  (2021-03-02T11:18:34Z)
• AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection [76.7063732501752]
  We provide a new identity swapping algorithm with large differences in appearance for face forgery detection. The appearance gaps mainly arise from the large discrepancies in illuminations and skin colors. A discriminator is introduced to distinguish the fake parts from a mix of real and fake image patches.
  arXiv  Detail & Related papers  (2020-11-05T06:17:04Z)
• PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models [77.32079593577821]
  PULSE (Photo Upsampling via Latent Space Exploration) generates high-resolution, realistic images at resolutions previously unseen in the literature. Our method outperforms state-of-the-art methods in perceptual quality at higher resolutions and scale factors than previously possible.
  arXiv  Detail & Related papers  (2020-03-08T16:44:31Z)
• AE-OT-GAN: Training GANs from data specific latent distribution [21.48007565143911]
  generative adversarial networks (GANs) areprominent models to generate realistic and crisp images. GANs often encounter the mode collapse problems and arehard to train, which comes from approximating the intrinsicdiscontinuous distribution transform map with continuousDNNs. The recently proposed AE-OT model addresses thisproblem by explicitly computing the discontinuous distribu-tion transform map. In this paper, wepropose the AE-OT-GAN model to utilize the advantages ofthe both models: generate high quality images and at the same time overcome the mode collapse/mixture problems.
  arXiv  Detail & Related papers  (2020-01-11T01:18:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.