ParGAN: Learning Real Parametrizable Transformations

• URL: http://arxiv.org/abs/2211.04996v1
• Date: Wed, 9 Nov 2022 16:16:06 GMT
• Title: ParGAN: Learning Real Parametrizable Transformations
• Authors: Diego Martin Arroyo, Alessio Tonioni, Federico Tombari
• Abstract summary: We propose ParGAN, a generalization of the cycle-consistent GAN framework to learn image transformations. The proposed generator takes as input both an image and a parametrization of the transformation. We show how, with disjoint image domains with no annotated parametrization, our framework can create smooths as well as learn multiple transformations simultaneously.
• Score: 50.51405390150066
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current methods for image-to-image translation produce compelling results, however, the applied transformation is difficult to control, since existing mechanisms are often limited and non-intuitive. We propose ParGAN, a generalization of the cycle-consistent GAN framework to learn image transformations with simple and intuitive controls. The proposed generator takes as input both an image and a parametrization of the transformation. We train this network to preserve the content of the input image while ensuring that the result is consistent with the given parametrization. Our approach does not require paired data and can learn transformations across several tasks and datasets. We show how, with disjoint image domains with no annotated parametrization, our framework can create smooth interpolations as well as learn multiple transformations simultaneously.

Related papers

• DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation [56.514462874501675]
  We propose a dynamic sparse attention based Transformer model to achieve fine-level matching with favorable efficiency. The heart of our approach is a novel dynamic-attention unit, dedicated to covering the variation on the optimal number of tokens one position should focus on. Experiments on three applications, pose-guided person image generation, edge-based face synthesis, and undistorted image style transfer, demonstrate that DynaST achieves superior performance in local details.
  arXiv  Detail & Related papers  (2022-07-13T11:12:03Z)
• XCiT: Cross-Covariance Image Transformers [73.33400159139708]
  We propose a "transposed" version of self-attention that operates across feature channels rather than tokens. The resulting cross-covariance attention (XCA) has linear complexity in the number of tokens, and allows efficient processing of high-resolution images.
  arXiv  Detail & Related papers  (2021-06-17T17:33:35Z)
• Robust Training Using Natural Transformation [19.455666609149567]
  We present NaTra, an adversarial training scheme to improve robustness of image classification algorithms. We target attributes of the input images that are independent of the class identification, and manipulate those attributes to mimic real-world natural transformations. We demonstrate the efficacy of our scheme by utilizing the disentangled latent representations derived from well-trained GANs.
  arXiv  Detail & Related papers  (2021-05-10T01:56:03Z)
• FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning [64.32306537419498]
  We propose a novel learned feature-based refinement and augmentation method that produces a varied set of complex transformations. These transformations also use information from both within-class and across-class representations that we extract through clustering. We demonstrate that our method is comparable to current state of art for smaller datasets while being able to scale up to larger datasets.
  arXiv  Detail & Related papers  (2020-07-16T17:55:31Z)
• Image-to-image Mapping with Many Domains by Sparse Attribute Transfer [71.28847881318013]
  Unsupervised image-to-image translation consists of learning a pair of mappings between two domains without known pairwise correspondences between points. Current convention is to approach this task with cycle-consistent GANs. We propose an alternate approach that directly restricts the generator to performing a simple sparse transformation in a latent layer.
  arXiv  Detail & Related papers  (2020-06-23T19:52:23Z)
• Probabilistic Spatial Transformer Networks [0.6999740786886537]
  We propose a probabilistic extension that estimates a transformation rather than a deterministic one. We show that these two properties lead to improved classification performance, robustness and model calibration. We further demonstrate that the approach generalizes to non-visual domains by improving model performance on time-series data.
  arXiv  Detail & Related papers  (2020-04-07T18:22:02Z)
• Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks [11.4219428942199]
  We present a novel, efficient unsupervised symmetric image registration method. We evaluate our method on 3D image registration with a large scale brain image dataset.
  arXiv  Detail & Related papers  (2020-03-20T22:07:24Z)
• Fine-grained Image-to-Image Transformation towards Visual Recognition [102.51124181873101]
  We aim at transforming an image with a fine-grained category to synthesize new images that preserve the identity of the input image. We adopt a model based on generative adversarial networks to disentangle the identity related and unrelated factors of an image. Experiments on the CompCars and Multi-PIE datasets demonstrate that our model preserves the identity of the generated images much better than the state-of-the-art image-to-image transformation models.
  arXiv  Detail & Related papers  (2020-01-12T05:26:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.