Householder Projector for Unsupervised Latent Semantics Discovery

• URL: http://arxiv.org/abs/2307.08012v1
• Date: Sun, 16 Jul 2023 11:43:04 GMT
• Title: Householder Projector for Unsupervised Latent Semantics Discovery
• Authors: Yue Song, Jichao Zhang, Nicu Sebe, Wei Wang
• Abstract summary: Householder Projector helps StyleGANs to discover more disentangled and precise semantic attributes without sacrificing image fidelity. We integrate our projector into pre-trained StyleGAN2/StyleGAN3 and evaluate the models on several benchmarks.
• Score: 58.92485745195358
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Generative Adversarial Networks (GANs), especially the recent style-based generators (StyleGANs), have versatile semantics in the structured latent space. Latent semantics discovery methods emerge to move around the latent code such that only one factor varies during the traversal. Recently, an unsupervised method proposed a promising direction to directly use the eigenvectors of the projection matrix that maps latent codes to features as the interpretable directions. However, one overlooked fact is that the projection matrix is non-orthogonal and the number of eigenvectors is too large. The non-orthogonality would entangle semantic attributes in the top few eigenvectors, and the large dimensionality might result in meaningless variations among the directions even if the matrix is orthogonal. To avoid these issues, we propose Householder Projector, a flexible and general low-rank orthogonal matrix representation based on Householder transformations, to parameterize the projection matrix. The orthogonality guarantees that the eigenvectors correspond to disentangled interpretable semantics, while the low-rank property encourages that each identified direction has meaningful variations. We integrate our projector into pre-trained StyleGAN2/StyleGAN3 and evaluate the models on several benchmarks. Within only $1\%$ of the original training steps for fine-tuning, our projector helps StyleGANs to discover more disentangled and precise semantic attributes without sacrificing image fidelity.

Related papers

• Efficient Adaptation of Pre-trained Vision Transformer via Householder Transformation [53.88562288388169]
  A common strategy for. Efficient Fine-Tuning (PEFT) of pre-trained Vision Transformers (ViTs) involves adapting the model to downstream tasks. We propose a novel PEFT approach inspired by Singular Value Decomposition (SVD) for representing the adaptation matrix. SVD decomposes a matrix into the product of a left unitary matrix, a diagonal matrix of scaling values, and a right unitary matrix.
  arXiv  Detail & Related papers  (2024-10-30T12:08:30Z)
• Refined Risk Bounds for Unbounded Losses via Transductive Priors [58.967816314671296]
  We revisit the sequential variants of linear regression with the squared loss, classification problems with hinge loss, and logistic regression. Our key tools are based on the exponential weights algorithm with carefully chosen transductive priors.
  arXiv  Detail & Related papers  (2024-10-29T00:01:04Z)
• Optimal Projections for Discriminative Dictionary Learning using the JL-lemma [0.5461938536945723]
  Dimensionality reduction-based dictionary learning methods have often used iterative random projections. This paper proposes a constructive approach to derandomize the projection matrix using the Johnson-Lindenstrauss lemma.
  arXiv  Detail & Related papers  (2023-08-27T02:59:59Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• Controlling Directions Orthogonal to a Classifier [11.882219706353045]
  We propose to identify directions invariant to a given classifier so that these directions can be controlled in tasks such as style transfer. We present three use cases where controlling orthogonal variation is important: style transfer, domain adaptation, and fairness. The code is available at http://github.com/Newbeeer/orthogonal_classifier.
  arXiv  Detail & Related papers  (2022-01-27T01:23:08Z)
• Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation [64.92152574895111]
  We propose a simple Orthogonal Jacobian Regularization (OroJaR) to encourage deep generative model to learn disentangled representations. Our method is effective in disentangled and controllable image generation, and performs favorably against the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-17T15:01:46Z)
• EigenGAN: Layer-Wise Eigen-Learning for GANs [84.33920839885619]
  EigenGAN is able to unsupervisedly mine interpretable and controllable dimensions from different generator layers. By traversing the coefficient of a specific eigen-dimension, the generator can produce samples with continuous changes corresponding to a specific semantic attribute.
  arXiv  Detail & Related papers  (2021-04-26T11:14:37Z)
• Disentangled Representation Learning and Generation with Manifold Optimization [10.69910379275607]
  This work presents a representation learning framework that explicitly promotes disentanglement by encouraging directions of variations. Our theoretical discussion and various experiments show that the proposed model improves over many VAE variants in terms of both generation quality and disentangled representation learning.
  arXiv  Detail & Related papers  (2020-06-12T10:00:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.