Interpreting Generative Adversarial Networks for Interactive Image Generation

• URL: http://arxiv.org/abs/2108.04896v1
• Date: Tue, 10 Aug 2021 19:42:20 GMT
• Title: Interpreting Generative Adversarial Networks for Interactive Image Generation
• Abstract summary: This chapter will give a summary of recent works on interpreting deep generative models. We will see how the human-understandable concepts that emerge in the learned representation can be identified and used for interactive image generation and editing.
• Score: 36.539339220652046
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Great progress has been made by the advances in Generative Adversarial Networks (GANs) for image generation. However, there lacks enough understanding on how a realistic image can be generated by the deep representations of GANs from a random vector. This chapter will give a summary of recent works on interpreting deep generative models. We will see how the human-understandable concepts that emerge in the learned representation can be identified and used for interactive image generation and editing.

Related papers

• Self-Corrected Image Generation with Explainable Latent Rewards [55.29175717238288]
  We propose xLARD, a self-correcting framework that guides generation through Explainable LAtent RewarDs.<n>xLARD introduces a lightweight corrector that refines latent representations based on structured feedback from model-generated references.<n> Experiments show that xLARD improves semantic alignment and visual fidelity while maintaining generative priors.
  arXiv  Detail & Related papers  (2026-03-26T02:59:35Z)
• Understanding-in-Generation: Reinforcing Generative Capability of Unified Model via Infusing Understanding into Generation [43.98469957837991]
  We propose a novel reasoning framework for unified models, Understanding-in-Generation (UiG)<n>The core insight of our UiG is to integrate generative guidance by the strong understanding capabilities during the reasoning process.<n>Our UiG framework demonstrates a significant performance improvement in text-to-image generation over existing text-to-image reasoning methods.
  arXiv  Detail & Related papers  (2025-09-23T04:52:39Z)
• Iterative Flow Matching -- Path Correction and Gradual Refinement for Enhanced Generative Modeling [6.343872515377999]
  We explain and demonstrate why flow matching can generate hallucinations, and propose an iterative process to improve the generation process. Our iterative process can be integrated into virtually $textitany$ generative modeling technique, thereby enhancing the performance and robustness of synthesis image systems.
  arXiv  Detail & Related papers  (2025-02-23T05:08:06Z)
• RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model [93.8067369210696]
  Text-to-image generation (TTI) refers to the usage of models that could process text input and generate high fidelity images based on text descriptions. Diffusion models are one prominent type of generative model used for the generation of images through the systematic introduction of noises with repeating steps. In the era of large models, scaling up model size and the integration with large language models have further improved the performance of TTI models.
  arXiv  Detail & Related papers  (2023-09-02T03:27:20Z)
• Traditional Classification Neural Networks are Good Generators: They are Competitive with DDPMs and GANs [104.72108627191041]
  We show that conventional neural network classifiers can generate high-quality images comparable to state-of-the-art generative models. We propose a mask-based reconstruction module to make semantic gradients-aware to synthesize plausible images. We show that our method is also applicable to text-to-image generation by regarding image-text foundation models.
  arXiv  Detail & Related papers  (2022-11-27T11:25:35Z)
• Distilling Representations from GAN Generator via Squeeze and Span [55.76208869775715]
  We propose to distill knowledge from GAN generators by squeezing and spanning their representations. We span the distilled representation of the synthetic domain to the real domain by also using real training data to remedy the mode collapse of GANs.
  arXiv  Detail & Related papers  (2022-11-06T01:10:28Z)
• IR-GAN: Image Manipulation with Linguistic Instruction by Increment Reasoning [110.7118381246156]
  Increment Reasoning Generative Adversarial Network (IR-GAN) aims to reason consistency between visual increment in images and semantic increment in instructions. First, we introduce the word-level and instruction-level instruction encoders to learn user's intention from history-correlated instructions as semantic increment. Second, we embed the representation of semantic increment into that of source image for generating target image, where source image plays the role of referring auxiliary.
  arXiv  Detail & Related papers  (2022-04-02T07:48:39Z)
• Improved Image Generation via Sparse Modeling [27.66648389933265]
  We show that generators can be viewed as manifestations of the Convolutional Sparse Coding (CSC) and its Multi-Layered version (ML-CSC) synthesis processes. We leverage this observation by explicitly enforcing a sparsifying regularization on appropriately chosen activation layers in the generator.
  arXiv  Detail & Related papers  (2021-04-01T13:52:40Z)
• Unsupervised Discovery of Disentangled Manifolds in GANs [74.24771216154105]
  Interpretable generation process is beneficial to various image editing applications. We propose a framework to discover interpretable directions in the latent space given arbitrary pre-trained generative adversarial networks.
  arXiv  Detail & Related papers  (2020-11-24T02:18:08Z)
• Interpreting Spatially Infinite Generative Models [40.453301580034804]
  Recent work has shown that feeding spatial noise vectors into a fully convolutional neural network enables both generation of arbitrary resolution output images and training on arbitrary resolution training images. We provide a firm theoretical interpretation for infinite spatial generation, by drawing connections to spatial processes. Experiments on world map generation, panoramic images and texture synthesis verify the ability of $infty$-GAN to efficiently generate images of arbitrary size.
  arXiv  Detail & Related papers  (2020-07-24T09:00:41Z)
• Network Bending: Expressive Manipulation of Deep Generative Models [0.2062593640149624]
  We introduce a new framework for manipulating and interacting with deep generative models that we call network bending. We show how it allows for the direct manipulation of semantically meaningful aspects of the generative process as well as allowing for a broad range of expressive outcomes.
  arXiv  Detail & Related papers  (2020-05-25T21:48:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.