MU-GAN: Facial Attribute Editing based on Multi-attention Mechanism

• URL: http://arxiv.org/abs/2009.04177v1
• Date: Wed, 9 Sep 2020 09:25:04 GMT
• Title: MU-GAN: Facial Attribute Editing based on Multi-attention Mechanism
• Authors: Ke Zhang, Yukun Su, Xiwang Guo, Liang Qi, and Zhenbing Zhao
• Abstract summary: We propose a Multi-attention U-Net-based Generative Adversarial Network (MU-GAN) First, we replace a classic convolutional encoder-decoder with a symmetric U-Net-like structure in a generator. Second, a self-attention mechanism is incorporated into convolutional layers for modeling long-range and multi-level dependencies.
• Score: 12.762892831902349
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Facial attribute editing has mainly two objectives: 1) translating image from a source domain to a target one, and 2) only changing the facial regions related to a target attribute and preserving the attribute-excluding details. In this work, we propose a Multi-attention U-Net-based Generative Adversarial Network (MU-GAN). First, we replace a classic convolutional encoder-decoder with a symmetric U-Net-like structure in a generator, and then apply an additive attention mechanism to build attention-based U-Net connections for adaptively transferring encoder representations to complement a decoder with attribute-excluding detail and enhance attribute editing ability. Second, a self-attention mechanism is incorporated into convolutional layers for modeling long-range and multi-level dependencies across image regions. experimental results indicate that our method is capable of balancing attribute editing ability and details preservation ability, and can decouple the correlation among attributes. It outperforms the state-of-the-art methods in terms of attribute manipulation accuracy and image quality.

Related papers

• Attribute-Aware Deep Hashing with Self-Consistency for Large-Scale Fine-Grained Image Retrieval [65.43522019468976]
  We propose attribute-aware hashing networks with self-consistency for generating attribute-aware hash codes. We develop an encoder-decoder structure network of a reconstruction task to unsupervisedly distill high-level attribute-specific vectors. Our models are equipped with a feature decorrelation constraint upon these attribute vectors to strengthen their representative abilities.
  arXiv  Detail & Related papers  (2023-11-21T08:20:38Z)
• Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning [52.506434446439776]
  Compositional zero-shot learning (CZSL) aims to recognize compositions with prior knowledge of known primitives (attribute and object) We propose a simple and scalable framework called Composition Transformer (CoT) to address these issues. Our method achieves SoTA performance on several benchmarks, including MIT-States, C-GQA, and VAW-CZSL.
  arXiv  Detail & Related papers  (2023-08-08T03:24:21Z)
• Leveraging Off-the-shelf Diffusion Model for Multi-attribute Fashion Image Manipulation [27.587905673112473]
  Fashion attribute editing is a task that aims to convert the semantic attributes of a given fashion image while preserving the irrelevant regions. Previous works typically employ conditional GANs where the generator explicitly learns the target attributes and directly execute the conversion. We explore the classifier-guided diffusion that leverages the off-the-shelf diffusion model pretrained on general visual semantics such as Imagenet.
  arXiv  Detail & Related papers  (2022-10-12T02:21:18Z)
• Supervised Attribute Information Removal and Reconstruction for Image Manipulation [15.559224431459551]
  We propose an Attribute Information Removal and Reconstruction (AIRR) network that prevents such information hiding. We evaluate our approach on four diverse datasets with a variety of attributes including DeepFashion Synthesis, DeepFashion Fine-grained Attribute, CelebA and CelebA-HQ.
  arXiv  Detail & Related papers  (2022-07-13T23:30:44Z)
• Context-Aware Layout to Image Generation with Enhanced Object Appearance [123.62597976732948]
  A layout to image (L2I) generation model aims to generate a complicated image containing multiple objects (things) against natural background (stuff) Existing L2I models have made great progress, but object-to-object and object-to-stuff relations are often broken. We argue that these are caused by the lack of context-aware object and stuff feature encoding in their generators, and location-sensitive appearance representation in their discriminators.
  arXiv  Detail & Related papers  (2021-03-22T14:43:25Z)
• Unpaired Image Enhancement with Quality-Attention Generative Adversarial Network [92.01145655155374]
  We propose a quality attention generative adversarial network (QAGAN) trained on unpaired data. Key novelty of the proposed QAGAN lies in the injected QAM for the generator. Our proposed method achieves better performance in both objective and subjective evaluations.
  arXiv  Detail & Related papers  (2020-12-30T05:57:20Z)
• CAFE-GAN: Arbitrary Face Attribute Editing with Complementary Attention Feature [31.425326840578098]
  We propose a novel GAN model which is designed to edit only the parts of a face pertinent to the target attributes. CAFE identifies the facial regions to be transformed by considering both target attributes as well as complementary attributes.
  arXiv  Detail & Related papers  (2020-11-24T05:21:03Z)
• SMILE: Semantically-guided Multi-attribute Image and Layout Editing [154.69452301122175]
  Attribute image manipulation has been a very active topic since the introduction of Generative Adversarial Networks (GANs) We present a multimodal representation that handles all attributes, be it guided by random noise or images, while only using the underlying domain information of the target domain. Our method is capable of adding, removing or changing either fine-grained or coarse attributes by using an image as a reference or by exploring the style distribution space.
  arXiv  Detail & Related papers  (2020-10-05T20:15:21Z)
• PA-GAN: Progressive Attention Generative Adversarial Network for Facial Attribute Editing [67.94255549416548]
  We propose a progressive attention GAN (PA-GAN) for facial attribute editing. Our approach achieves correct attribute editing with irrelevant details much better preserved compared with the state-of-the-arts.
  arXiv  Detail & Related papers  (2020-07-12T03:04:12Z)
• Prominent Attribute Modification using Attribute Dependent Generative Adversarial Network [4.654937118111992]
  The proposed approach is based on two generators and two discriminators that utilize the binary as well as the real representation of the attributes. Experiments on the CelebA dataset show that our method effectively performs the multiple attribute editing with preserving other facial details intactly.
  arXiv  Detail & Related papers  (2020-04-24T13:38:05Z)
• MulGAN: Facial Attribute Editing by Exemplar [2.272764591035106]
  Methods encode attribute-related information in images into the predefined region of the latent feature space by employing a pair of images with opposite attributes as input to train model. They suffer from three limitations: (1) the model must be trained using a pair of images with opposite attributes as input; (2) weak capability of editing multiple attributes by exemplars; and (3) poor quality of generating image.
  arXiv  Detail & Related papers  (2019-12-28T04:02:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.