TAGE: Trustworthy Attribute Group Editing for Stable Few-shot Image Generation

• URL: http://arxiv.org/abs/2410.17855v1
• Date: Wed, 23 Oct 2024 13:26:19 GMT
• Title: TAGE: Trustworthy Attribute Group Editing for Stable Few-shot Image Generation
• Authors: Ruicheng Zhang, Guoheng Huang, Yejing Huo, Xiaochen Yuan, Zhizhen Zhou, Xuhang Chen, Guo Zhong,
• Abstract summary: TAGE is an innovative image generation network comprising three integral modules. The CPM module delves into the semantic dimensions of category-agnostic attributes, encapsulating them within a discrete codebook. The PSM module generates semantic cues that are seamlessly integrated into the Transformer architecture of the CPM.
• Score: 10.569380190029317
• License:
• Abstract: Generative Adversarial Networks (GANs) have emerged as a prominent research focus for image editing tasks, leveraging the powerful image generation capabilities of the GAN framework to produce remarkable results.However, prevailing approaches are contingent upon extensive training datasets and explicit supervision, presenting a significant challenge in manipulating the diverse attributes of new image classes with limited sample availability. To surmount this hurdle, we introduce TAGE, an innovative image generation network comprising three integral modules: the Codebook Learning Module (CLM), the Code Prediction Module (CPM) and the Prompt-driven Semantic Module (PSM). The CPM module delves into the semantic dimensions of category-agnostic attributes, encapsulating them within a discrete codebook. This module is predicated on the concept that images are assemblages of attributes, and thus, by editing these category-independent attributes, it is theoretically possible to generate images from unseen categories. Subsequently, the CPM module facilitates naturalistic image editing by predicting indices of category-independent attribute vectors within the codebook. Additionally, the PSM module generates semantic cues that are seamlessly integrated into the Transformer architecture of the CPM, enhancing the model's comprehension of the targeted attributes for editing. With these semantic cues, the model can generate images that accentuate desired attributes more prominently while maintaining the integrity of the original category, even with a limited number of samples. We have conducted extensive experiments utilizing the Animal Faces, Flowers, and VGGFaces datasets. The results of these experiments demonstrate that our proposed method not only achieves superior performance but also exhibits a high degree of stability when compared to other few-shot image generation techniques.

Related papers

• Divide and Conquer: Language Models can Plan and Self-Correct for Compositional Text-to-Image Generation [72.6168579583414]
  CompAgent is a training-free approach for compositional text-to-image generation with a large language model (LLM) agent as its core. Our approach achieves more than 10% improvement on T2I-CompBench, a comprehensive benchmark for open-world compositional T2I generation.
  arXiv  Detail & Related papers  (2024-01-28T16:18:39Z)
• R-Cut: Enhancing Explainability in Vision Transformers with Relationship Weighted Out and Cut [14.382326829600283]
  We introduce two modules: the Relationship Weighted Out" and the Cut" modules. The Cut" module performs fine-grained feature decomposition, taking into account factors such as position, texture, and color. We validate our method with extensive qualitative and quantitative experiments on the ImageNet dataset.
  arXiv  Detail & Related papers  (2023-07-18T08:03:51Z)
• DiffuseGAE: Controllable and High-fidelity Image Manipulation from Disentangled Representation [14.725538019917625]
  Diffusion probabilistic models (DPMs) have shown remarkable results on various image synthesis tasks. DPMs lack a low-dimensional, interpretable, and well-decoupled latent code. We propose Diff-AE to explore the potential of DPMs for representation learning via autoencoding.
  arXiv  Detail & Related papers  (2023-07-12T04:11:08Z)
• Learning Structured Output Representations from Attributes using Deep Conditional Generative Models [0.0]
  This paper recreates the Conditional Variational Auto-encoder architecture and trains it on images conditioned on attributes. We attempt to generate new faces with distinct attributes such as hair color and glasses, as well as different bird species samples.
  arXiv  Detail & Related papers  (2023-04-30T17:25:31Z)
• Stable Attribute Group Editing for Reliable Few-shot Image Generation [88.59350889410794]
  We present an editing-based'' framework Attribute Group Editing (AGE) for reliable few-shot image generation. We find that class inconsistency is a common problem in GAN-generated images for downstream classification. We propose to boost the downstream classification performance of SAGE by enhancing the pixel and frequency components.
  arXiv  Detail & Related papers  (2023-02-01T01:51:47Z)
• Semantic Image Synthesis via Diffusion Models [159.4285444680301]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks. Recent work on semantic image synthesis mainly follows the emphde facto Generative Adversarial Nets (GANs)
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)
• Attribute Group Editing for Reliable Few-shot Image Generation [85.52840521454411]
  We propose a new editing-based method, i.e., Attribute Group Editing (AGE), for few-shot image generation. AGE examines the internal representation learned in GANs and identifies semantically meaningful directions.
  arXiv  Detail & Related papers  (2022-03-16T06:54:09Z)
• Meta Internal Learning [88.68276505511922]
  Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image. We propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively. Our results show that the models obtained are as suitable as single-image GANs for many common image applications.
  arXiv  Detail & Related papers  (2021-10-06T16:27:38Z)
• Generative Hierarchical Features from Synthesizing Images [65.66756821069124]
  We show that learning to synthesize images can bring remarkable hierarchical visual features that are generalizable across a wide range of applications. The visual feature produced by our encoder, termed as Generative Hierarchical Feature (GH-Feat), has strong transferability to both generative and discriminative tasks.
  arXiv  Detail & Related papers  (2020-07-20T18:04:14Z)
• Generating Annotated High-Fidelity Images Containing Multiple Coherent Objects [10.783993190686132]
  We propose a multi-object generation framework that can synthesize images with multiple objects without explicitly requiring contextual information. We demonstrate how coherency and fidelity are preserved with our method through experiments on the Multi-MNIST and CLEVR datasets.
  arXiv  Detail & Related papers  (2020-06-22T11:33:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.