Analogical Image Translation for Fog Generation

• URL: http://arxiv.org/abs/2006.15618v1
• Date: Sun, 28 Jun 2020 14:33:31 GMT
• Title: Analogical Image Translation for Fog Generation
• Authors: Rui Gong, Dengxin Dai, Yuhua Chen, Wen Li, Luc Van Gool
• Abstract summary: AIT learns with synthetic clear-weather images, synthetic foggy images and real clear-weather images to add fog effects onto real clear-weather images without seeing any real foggy images during training. AIT achieves this zero-shot image translation capability by coupling a supervised training scheme in the synthetic domain, a cycle consistency strategy in the real domain, an adversarial training scheme between the two domains, and a novel network design.
• Score: 114.39308837759329
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image-to-image translation is to map images from a given \emph{style} to another given \emph{style}. While exceptionally successful, current methods assume the availability of training images in both source and target domains, which does not always hold in practice. Inspired by humans' reasoning capability of analogy, we propose analogical image translation (AIT). Given images of two styles in the source domain: $\mathcal{A}$ and $\mathcal{A}^\prime$, along with images $\mathcal{B}$ of the first style in the target domain, learn a model to translate $\mathcal{B}$ to $\mathcal{B}^\prime$ in the target domain, such that $\mathcal{A}:\mathcal{A}^\prime ::\mathcal{B}:\mathcal{B}^\prime$. AIT is especially useful for translation scenarios in which training data of one style is hard to obtain but training data of the same two styles in another domain is available. For instance, in the case from normal conditions to extreme, rare conditions, obtaining real training images for the latter case is challenging but obtaining synthetic data for both cases is relatively easy. In this work, we are interested in adding adverse weather effects, more specifically fog effects, to images taken in clear weather. To circumvent the challenge of collecting real foggy images, AIT learns with synthetic clear-weather images, synthetic foggy images and real clear-weather images to add fog effects onto real clear-weather images without seeing any real foggy images during training. AIT achieves this zero-shot image translation capability by coupling a supervised training scheme in the synthetic domain, a cycle consistency strategy in the real domain, an adversarial training scheme between the two domains, and a novel network design. Experiments show the effectiveness of our method for zero-short image translation and its benefit for downstream tasks such as semantic foggy scene understanding.

Related papers

• CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulation [57.836686457542385]
  Diffusion models (DMs) have enabled breakthroughs in image synthesis tasks but lack an intuitive interface for consistent image-to-image (I2I) translation. This paper introduces Cyclenet, a novel but simple method that incorporates cycle consistency into DMs to regularize image manipulation.
  arXiv  Detail & Related papers  (2023-10-19T21:32:21Z)
• ACE: Zero-Shot Image to Image Translation via Pretrained Auto-Contrastive-Encoder [2.1874189959020427]
  We propose a new approach to extract image features by learning the similarities and differences of samples within the same data distribution. The design of ACE enables us to achieve zero-shot image-to-image translation with no training on image translation tasks for the first time. Our model achieves competitive results on multimodal image translation tasks with zero-shot learning as well.
  arXiv  Detail & Related papers  (2023-02-22T23:52:23Z)
• What is Where by Looking: Weakly-Supervised Open-World Phrase-Grounding without Text Inputs [82.93345261434943]
  Given an input image, and nothing else, our method returns the bounding boxes of objects in the image and phrases that describe the objects. This is achieved within an open world paradigm, in which the objects in the input image may not have been encountered during the training of the localization mechanism. Our work generalizes weakly supervised segmentation and phrase grounding and is shown empirically to outperform the state of the art in both domains.
  arXiv  Detail & Related papers  (2022-06-19T09:07:30Z)
• Synthetic-to-Real Domain Adaptation using Contrastive Unpaired Translation [28.19031441659854]
  We propose a multi-step method to obtain training data without manual annotation effort. From 3D object meshes, we generate images using a modern synthesis pipeline. We utilize a state-of-the-art image-to-image translation method to adapt the synthetic images to the real domain.
  arXiv  Detail & Related papers  (2022-03-17T17:13:23Z)
• ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translation [55.47515538020578]
  This work proposes an implicit style function (ISF) to straightforwardly achieve multi-modal and multi-domain image-to-image translation. Our results in human face and animal manipulations show significantly improved results over the baselines. Our model enables cost-effective multi-modal unsupervised image-to-image translations at high resolution using pre-trained unconditional GANs.
  arXiv  Detail & Related papers  (2021-09-26T04:51:39Z)
• Domain Adaptation for Image Dehazing [72.15994735131835]
  Most existing methods train a dehazing model on synthetic hazy images, which are less able to generalize well to real hazy images due to domain shift. We propose a domain adaptation paradigm, which consists of an image translation module and two image dehazing modules. Experimental results on both synthetic and real-world images demonstrate that our model performs favorably against the state-of-the-art dehazing algorithms.
  arXiv  Detail & Related papers  (2020-05-10T13:54:56Z)
• TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images [102.4003329297039]
  An unsupervised image-to-image translation (UI2I) task deals with learning a mapping between two domains without paired images. We propose TuiGAN, a generative model that is trained on only two unpaired images and amounts to one-shot unsupervised learning.
  arXiv  Detail & Related papers  (2020-04-09T16:23:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.