Attribute-Specific Manipulation Based on Layer-Wise Channels

• URL: http://arxiv.org/abs/2302.09260v1
• Date: Sat, 18 Feb 2023 08:49:20 GMT
• Title: Attribute-Specific Manipulation Based on Layer-Wise Channels
• Authors: Yuanjie Yan, Jian Zhao, Furao Shen
• Abstract summary: Some studies have focused on detecting channels with specific properties to manipulate the latent code. We propose a novel detection method in the context of pre-trained classifiers. Our methods can accurately detect relevant channels for a large number of face attributes.
• Score: 11.063763802330142
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image manipulation on the latent space of the pre-trained StyleGAN can control the semantic attributes of the generated images. Recently, some studies have focused on detecting channels with specific properties to directly manipulate the latent code, which is limited by the entanglement of the latent space. To detect the attribute-specific channels, we propose a novel detection method in the context of pre-trained classifiers. We analyse the gradients layer by layer on the style space. The intensities of the gradients indicate the channel's responses to specific attributes. The latent style codes of channels control separate attributes in the layers. We choose channels with top-$k$ gradients to control specific attributes in the maximum response layer. We implement single-channel and multi-channel manipulations with a certain attribute. Our methods can accurately detect relevant channels for a large number of face attributes. Extensive qualitative and quantitative results demonstrate that the proposed methods outperform state-of-the-art methods in generalization and scalability.

Related papers

• Exploring Gradient-based Multi-directional Controls in GANs [19.950198707910587]
  We propose a novel approach that discovers nonlinear controls, which enables multi-directional manipulation as well as effective disentanglement. Our approach is able to gain fine-grained controls over a diverse set of bi-directional and multi-directional attributes, and we showcase its ability to achieve disentanglement significantly better than state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-09-01T19:10:26Z)
• Gradient-Based Adversarial and Out-of-Distribution Detection [15.510581400494207]
  We introduce confounding labels in gradient generation to probe the effective expressivity of neural networks. We show that our gradient-based approach allows for capturing the anomaly in inputs based on the effective expressivity of the models.
  arXiv  Detail & Related papers  (2022-06-16T15:50:41Z)
• Attribute-specific Control Units in StyleGAN for Fine-grained Image Manipulation [57.99007520795998]
  We discover attribute-specific control units, which consist of multiple channels of feature maps and modulation styles. Specifically, we collaboratively manipulate the modulation style channels and feature maps in control units to obtain the semantic and spatial disentangled controls. We move the modulation style along a specific sparse direction vector and replace the filter-wise styles used to compute the feature maps to manipulate these control units.
  arXiv  Detail & Related papers  (2021-11-25T10:42:10Z)
• Disentangled Face Attribute Editing via Instance-Aware Latent Space Search [30.17338705964925]
  A rich set of semantic directions exist in the latent space of Generative Adversarial Networks (GANs) Existing methods may suffer poor attribute variation disentanglement, leading to unwanted change of other attributes when altering the desired one. We propose a novel framework (IALS) that performs Instance-Aware Latent-Space Search to find semantic directions for disentangled attribute editing.
  arXiv  Detail & Related papers  (2021-05-26T16:19:08Z)
• Channel-wise Knowledge Distillation for Dense Prediction [73.99057249472735]
  We propose to align features channel-wise between the student and teacher networks. We consistently achieve superior performance on three benchmarks with various network structures.
  arXiv  Detail & Related papers  (2020-11-26T12:00:38Z)
• StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation [45.20783737095007]
  We explore and analyze the latent style space of StyleGAN2, a state-of-the-art architecture for image generation. StyleSpace is significantly more disentangled than the other intermediate latent spaces explored by previous works. Our findings pave the way to semantically meaningful and well-disentangled image manipulations via simple and intuitive interfaces.
  arXiv  Detail & Related papers  (2020-11-25T15:00:33Z)
• Channel-wise Alignment for Adaptive Object Detection [66.76486843397267]
  Generic object detection has been immensely promoted by the development of deep convolutional neural networks. Existing methods on this task usually draw attention on the high-level alignment based on the whole image or object of interest. In this paper, we realize adaptation from a thoroughly different perspective, i.e., channel-wise alignment.
  arXiv  Detail & Related papers  (2020-09-07T02:42:18Z)
• Operation-Aware Soft Channel Pruning using Differentiable Masks [51.04085547997066]
  We propose a data-driven algorithm, which compresses deep neural networks in a differentiable way by exploiting the characteristics of operations. We perform extensive experiments and achieve outstanding performance in terms of the accuracy of output networks.
  arXiv  Detail & Related papers  (2020-07-08T07:44:00Z)
• Attribute-based Regularization of Latent Spaces for Variational Auto-Encoders [79.68916470119743]
  We present a novel method to structure the latent space of a Variational Auto-Encoder (VAE) to encode different continuous-valued attributes explicitly. This is accomplished by using an attribute regularization loss which enforces a monotonic relationship between the attribute values and the latent code of the dimension along which the attribute is to be encoded.
  arXiv  Detail & Related papers  (2020-04-11T20:53:13Z)
• Attribute Mix: Semantic Data Augmentation for Fine Grained Recognition [102.45926816660665]
  We propose Attribute Mix, a data augmentation strategy at attribute level to expand the fine-grained samples. The principle lies in that attribute features are shared among fine-grained sub-categories, and can be seamlessly transferred among images.
  arXiv  Detail & Related papers  (2020-04-06T14:06:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.