Compressing Image-to-Image Translation GANs Using Local Density Structures on Their Learned Manifold

• URL: http://arxiv.org/abs/2312.14776v1
• Date: Fri, 22 Dec 2023 15:43:12 GMT
• Title: Compressing Image-to-Image Translation GANs Using Local Density Structures on Their Learned Manifold
• Authors: Alireza Ganjdanesh, Shangqian Gao, Hirad Alipanah, Heng Huang
• Abstract summary: Generative Adversarial Networks (GANs) have shown remarkable success in modeling complex data distributions for image-to-image translation. Existing GAN compression methods mainly rely on knowledge distillation or convolutional classifiers' pruning techniques. We propose a new approach by explicitly encouraging the pruned model to preserve the density structure of the original parameter-heavy model on its learned manifold. Our experiments on image translation GAN models, Pix2Pix and CycleGAN, with various benchmark datasets and architectures demonstrate our method's effectiveness.
• Score: 69.33930972652594
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Generative Adversarial Networks (GANs) have shown remarkable success in modeling complex data distributions for image-to-image translation. Still, their high computational demands prohibit their deployment in practical scenarios like edge devices. Existing GAN compression methods mainly rely on knowledge distillation or convolutional classifiers' pruning techniques. Thus, they neglect the critical characteristic of GANs: their local density structure over their learned manifold. Accordingly, we approach GAN compression from a new perspective by explicitly encouraging the pruned model to preserve the density structure of the original parameter-heavy model on its learned manifold. We facilitate this objective for the pruned model by partitioning the learned manifold of the original generator into local neighborhoods around its generated samples. Then, we propose a novel pruning objective to regularize the pruned model to preserve the local density structure over each neighborhood, resembling the kernel density estimation method. Also, we develop a collaborative pruning scheme in which the discriminator and generator are pruned by two pruning agents. We design the agents to capture interactions between the generator and discriminator by exchanging their peer's feedback when determining corresponding models' architectures. Thanks to such a design, our pruning method can efficiently find performant sub-networks and can maintain the balance between the generator and discriminator more effectively compared to baselines during pruning, thereby showing more stable pruning dynamics. Our experiments on image translation GAN models, Pix2Pix and CycleGAN, with various benchmark datasets and architectures demonstrate our method's effectiveness.

Related papers

• One-Shot Adaptation of GAN in Just One CLIP [51.188396199083336]
  We present a novel single-shot GAN adaptation method through unified CLIP space manipulations. Specifically, our model employs a two-step training strategy: reference image search in the source generator using a CLIP-guided latent optimization. We show that our model generates diverse outputs with the target texture and outperforms the baseline models both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-03-17T13:03:06Z)
• Hilbert Flattening: a Locality-Preserving Matrix Unfolding Method for Visual Discrimination [51.432453379052724]
  We propose Hilbert curve flattening as an innovative method to preserve locality in flattened matrices. We also introduce the Localformer, a vision transformer architecture that incorporates token sampling with a token aggregator to enhance its locality bias.
  arXiv  Detail & Related papers  (2022-02-21T13:53:04Z)
• CSformer: Bridging Convolution and Transformer for Compressive Sensing [65.22377493627687]
  This paper proposes a hybrid framework that integrates the advantages of leveraging detailed spatial information from CNN and the global context provided by transformer for enhanced representation learning. The proposed approach is an end-to-end compressive image sensing method, composed of adaptive sampling and recovery. The experimental results demonstrate the effectiveness of the dedicated transformer-based architecture for compressive sensing.
  arXiv  Detail & Related papers  (2021-12-31T04:37:11Z)
• Top-Down Deep Clustering with Multi-generator GANs [0.0]
  Deep clustering (DC) learns embedding spaces that are optimal for cluster analysis. We propose HC-MGAN, a new technique based on GANs with multiple generators (MGANs) Our method is inspired by the observation that each generator of a MGAN tends to generate data that correlates with a sub-region of the real data distribution.
  arXiv  Detail & Related papers  (2021-12-06T22:53:12Z)
• Rethinking conditional GAN training: An approach using geometrically structured latent manifolds [58.07468272236356]
  Conditional GANs (cGAN) suffer from critical drawbacks such as the lack of diversity in generated outputs. We propose a novel training mechanism that increases both the diversity and the visual quality of a vanilla cGAN.
  arXiv  Detail & Related papers  (2020-11-25T22:54:11Z)
• Generative Model without Prior Distribution Matching [26.91643368299913]
  Variational Autoencoder (VAE) and its variations are classic generative models by learning a low-dimensional latent representation to satisfy some prior distribution. We propose to let the prior match the embedding distribution rather than imposing the latent variables to fit the prior.
  arXiv  Detail & Related papers  (2020-09-23T09:33:24Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)
• Optimizing Generative Adversarial Networks for Image Super Resolution via Latent Space Regularization [4.529132742139768]
  Generative Adversarial Networks (GANs) try to learn the distribution of the real images in the manifold to generate samples that look real. We probe for ways to alleviate these problems for supervised GANs in this paper.
  arXiv  Detail & Related papers  (2020-01-22T16:27:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.