Extremal Domain Translation with Neural Optimal Transport

• URL: http://arxiv.org/abs/2301.12874v3
• Date: Thu, 2 Nov 2023 14:15:11 GMT
• Title: Extremal Domain Translation with Neural Optimal Transport
• Authors: Milena Gazdieva, Alexander Korotin, Daniil Selikhanovych, Evgeny Burnaev
• Abstract summary: We propose the extremal transport (ET) which is a formalization of the theoretically best possible unpaired translation between a pair of domains. Inspired by the recent advances in neural optimal transport (OT), we propose a scalable algorithm to approximate ET maps as a limit of partial OT maps. We test our algorithm on toy examples and on the unpaired image-to-image translation task.
• Score: 76.38747967445994
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In many unpaired image domain translation problems, e.g., style transfer or super-resolution, it is important to keep the translated image similar to its respective input image. We propose the extremal transport (ET) which is a mathematical formalization of the theoretically best possible unpaired translation between a pair of domains w.r.t. the given similarity function. Inspired by the recent advances in neural optimal transport (OT), we propose a scalable algorithm to approximate ET maps as a limit of partial OT maps. We test our algorithm on toy examples and on the unpaired image-to-image translation task. The code is publicly available at https://github.com/milenagazdieva/ExtremalNeuralOptimalTransport

Related papers

• Optimal Image Transport on Sparse Dictionaries [2.7855886538423182]
  We derive a novel optimal image transport algorithm over sparse dictionaries by taking advantage of Sparse Representation (SR) and Optimal Transport (OT) We demonstrate its versatility and many benefits to different image-to-image translation tasks, in particular image color transform and artistic style transfer, and show the plausible results for photo-realistic transferred effects.
  arXiv  Detail & Related papers  (2023-11-03T15:37:01Z)
• Image-to-Image Translation for Autonomous Driving from Coarsely-Aligned Image Pairs [57.33431586417377]
  A self-driving car must be able to handle adverse weather conditions to operate safely. In this paper, we investigate the idea of turning sensor inputs captured in an adverse condition into a benign one. We show that our coarsely-aligned training scheme leads to a better image translation quality and improved downstream tasks.
  arXiv  Detail & Related papers  (2022-09-23T16:03:18Z)
• Neural Style Transfer and Unpaired Image-to-Image Translation to deal with the Domain Shift Problem on Spheroid Segmentation [0.0]
  Domain shift is a generalisation problem of machine learning models that occurs when the data distribution of the training set is different to the data distribution encountered by the model when it is deployed. This is common in the context of biomedical image segmentation due to the variance of experimental conditions, equipment, and capturing settings. We have illustrated the domain shift problem in the context of spheroid segmentation with 4 deep learning segmentation models that achieved an IoU over 97% when tested with images following the training distribution, but whose performance decreased up to an 84% when applied to images captured under different conditions.
  arXiv  Detail & Related papers  (2021-12-16T17:34:45Z)
• Generative Modeling with Optimal Transport Maps [83.59805931374197]
  Optimal Transport (OT) has become a powerful tool for large-scale generative modeling tasks. We show that the OT map itself can be used as a generative model, providing comparable performance.
  arXiv  Detail & Related papers  (2021-10-06T18:17:02Z)
• Translating Images into Maps [43.81207458783278]
  We show how a novel form of transformer network can be used to map from images and video directly to an overhead map or bird's-eye-view (BEV) of the world. We assume a 1-1 correspondence between a vertical scanline in the image, and rays passing through the camera location in an overhead map. Posing the problem as translation allows the network to use the context of the image when interpreting the role of each pixel.
  arXiv  Detail & Related papers  (2021-10-03T09:52:46Z)
• The Spatially-Correlative Loss for Various Image Translation Tasks [69.62228639870114]
  We propose a novel spatially-correlative loss that is simple, efficient and yet effective for preserving scene structure consistency. Previous methods attempt this by using pixel-level cycle-consistency or feature-level matching losses. We show distinct improvement over baseline models in all three modes of unpaired I2I translation: single-modal, multi-modal, and even single-image translation.
  arXiv  Detail & Related papers  (2021-04-02T02:13:30Z)
• Unpaired Image-to-Image Translation via Latent Energy Transport [61.62293304236371]
  Image-to-image translation aims to preserve source contents while translating to discriminative target styles between two visual domains. In this paper, we propose to deploy an energy-based model (EBM) in the latent space of a pretrained autoencoder for this task. Our model is the first to be applicable to 1024$times$1024-resolution unpaired image translation.
  arXiv  Detail & Related papers  (2020-12-01T17:18:58Z)
• Zero-Pair Image to Image Translation using Domain Conditional Normalization [138.7878582237908]
  We propose an approach based on domain conditional normalization (DCN) for zero-pair image-to-image translation. We employ a single generator which has an encoder-decoder structure and analyze different implementations of domain conditional normalization to obtain the desired target domain output.
  arXiv  Detail & Related papers  (2020-11-11T10:20:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.