Optimizing Generative Adversarial Networks for Image Super Resolution via Latent Space Regularization

• URL: http://arxiv.org/abs/2001.08126v2
• Date: Sat, 9 Jan 2021 04:40:13 GMT
• Title: Optimizing Generative Adversarial Networks for Image Super Resolution via Latent Space Regularization
• Authors: Sheng Zhong and Shifu Zhou (Agora.io)
• Abstract summary: Generative Adversarial Networks (GANs) try to learn the distribution of the real images in the manifold to generate samples that look real. We probe for ways to alleviate these problems for supervised GANs in this paper.
• Score: 4.529132742139768
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural images can be regarded as residing in a manifold that is embedded in a higher dimensional Euclidean space. Generative Adversarial Networks (GANs) try to learn the distribution of the real images in the manifold to generate samples that look real. But the results of existing methods still exhibit many unpleasant artifacts and distortions even for the cases where the desired ground truth target images are available for supervised learning such as in single image super resolution (SISR). We probe for ways to alleviate these problems for supervised GANs in this paper. We explicitly apply the Lipschitz Continuity Condition (LCC) to regularize the GAN. An encoding network that maps the image space to a new optimal latent space is derived from the LCC, and it is used to augment the GAN as a coupling component. The LCC is also converted to new regularization terms in the generator loss function to enforce local invariance. The GAN is optimized together with the encoding network in an attempt to make the generator converge to a more ideal and disentangled mapping that can generate samples more faithful to the target images. When the proposed models are applied to the single image super resolution problem, the results outperform the state of the art.

Related papers

• In-Domain GAN Inversion for Faithful Reconstruction and Editability [132.68255553099834]
  We propose in-domain GAN inversion, which consists of a domain-guided domain-regularized and a encoder to regularize the inverted code in the native latent space of the pre-trained GAN model. We make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property.
  arXiv  Detail & Related papers  (2023-09-25T08:42:06Z)
• LD-GAN: Low-Dimensional Generative Adversarial Network for Spectral Image Generation with Variance Regularization [72.4394510913927]
  Deep learning methods are state-of-the-art for spectral image (SI) computational tasks. GANs enable diverse augmentation by learning and sampling from the data distribution. GAN-based SI generation is challenging since the high-dimensionality nature of this kind of data hinders the convergence of the GAN training yielding to suboptimal generation. We propose a statistical regularization to control the low-dimensional representation variance for the autoencoder training and to achieve high diversity of samples generated with the GAN.
  arXiv  Detail & Related papers  (2023-04-29T00:25:02Z)
• Latent Space is Feature Space: Regularization Term for GANs Training on Limited Dataset [1.8634083978855898]
  I proposed an additional structure and loss function for GANs called LFM, trained to maximize the feature diversity between the different dimensions of the latent space. In experiments, this system has been built upon DCGAN and proved to have improvement on Frechet Inception Distance (FID) training from scratch on CelebA dataset.
  arXiv  Detail & Related papers  (2022-10-28T16:34:48Z)
• One-Shot Adaptation of GAN in Just One CLIP [51.188396199083336]
  We present a novel single-shot GAN adaptation method through unified CLIP space manipulations. Specifically, our model employs a two-step training strategy: reference image search in the source generator using a CLIP-guided latent optimization. We show that our model generates diverse outputs with the target texture and outperforms the baseline models both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-03-17T13:03:06Z)
• Unpaired Image Super-Resolution with Optimal Transport Maps [128.1189695209663]
  Real-world image super-resolution (SR) tasks often do not have paired datasets limiting the application of supervised techniques. We propose an algorithm for unpaired SR which learns an unbiased OT map for the perceptual transport cost. Our algorithm provides nearly state-of-the-art performance on the large-scale unpaired AIM-19 dataset.
  arXiv  Detail & Related papers  (2022-02-02T16:21:20Z)
• Low-Rank Subspaces in GANs [101.48350547067628]
  This work introduces low-rank subspaces that enable more precise control of GAN generation. LowRankGAN is able to find the low-dimensional representation of attribute manifold. Experiments on state-of-the-art GAN models (including StyleGAN2 and BigGAN) trained on various datasets demonstrate the effectiveness of our LowRankGAN.
  arXiv  Detail & Related papers  (2021-06-08T16:16:32Z)
• Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space [41.20193123974535]
  Modern Generative Adversarial Networks are capable of creating artificial, photorealistic images from latent vectors living in a low-dimensional learned latent space. We show that, under a simple nonlinear operation, the data distribution can be modeled as Gaussian and therefore expressed using sufficient statistics. The resulting projections lie in smoother and better behaved regions of the latent space, as shown using performance for both real and generated images.
  arXiv  Detail & Related papers  (2020-09-14T15:45:58Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)
• AE-OT-GAN: Training GANs from data specific latent distribution [21.48007565143911]
  generative adversarial networks (GANs) areprominent models to generate realistic and crisp images. GANs often encounter the mode collapse problems and arehard to train, which comes from approximating the intrinsicdiscontinuous distribution transform map with continuousDNNs. The recently proposed AE-OT model addresses thisproblem by explicitly computing the discontinuous distribu-tion transform map. In this paper, wepropose the AE-OT-GAN model to utilize the advantages ofthe both models: generate high quality images and at the same time overcome the mode collapse/mixture problems.
  arXiv  Detail & Related papers  (2020-01-11T01:18:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.