Finding the global semantic representation in GAN through Frechet Mean

• URL: http://arxiv.org/abs/2210.05509v2
• Date: Sun, 23 Apr 2023 09:30:56 GMT
• Title: Finding the global semantic representation in GAN through Frechet Mean
• Authors: Jaewoong Choi, Geonho Hwang, Hyunsoo Cho, Myungjoo Kang
• Abstract summary: We propose an unsupervised method for finding this global semantic basis in the intermediate latent space in GANs. The proposed global basis, called Fr'echet basis, is derived by introducing Fr'echet mean to the local semantic perturbations in a latent space. Experimental results demonstrate that Fr'echet basis provides better semantic factorization and robustness compared to the previous methods.
• Score: 4.688163910878411
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ideally disentangled latent space in GAN involves the global representation of latent space with semantic attribute coordinates. In other words, considering that this disentangled latent space is a vector space, there exists the global semantic basis where each basis component describes one attribute of generated images. In this paper, we propose an unsupervised method for finding this global semantic basis in the intermediate latent space in GANs. This semantic basis represents sample-independent meaningful perturbations that change the same semantic attribute of an image on the entire latent space. The proposed global basis, called Fr\'echet basis, is derived by introducing Fr\'echet mean to the local semantic perturbations in a latent space. Fr\'echet basis is discovered in two stages. First, the global semantic subspace is discovered by the Fr\'echet mean in the Grassmannian manifold of the local semantic subspaces. Second, Fr\'echet basis is found by optimizing a basis of the semantic subspace via the Fr\'echet mean in the Special Orthogonal Group. Experimental results demonstrate that Fr\'echet basis provides better semantic factorization and robustness compared to the previous methods. Moreover, we suggest the basis refinement scheme for the previous methods. The quantitative experiments show that the refined basis achieves better semantic factorization while constrained on the same semantic subspace given by the previous method.

Related papers

• Spatial Semantic Recurrent Mining for Referring Image Segmentation [63.34997546393106]
  We propose Stextsuperscript2RM to achieve high-quality cross-modality fusion. It follows a working strategy of trilogy: distributing language feature, spatial semantic recurrent coparsing, and parsed-semantic balancing. Our proposed method performs favorably against other state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2024-05-15T00:17:48Z)
• Semantic-Rearrangement-Based Multi-Level Alignment for Domain Generalized Segmentation [11.105659621713855]
  We argue that different local semantic regions perform different visual characteristics from the source domain to the target domain. We propose the Semantic-Rearrangement-based Multi-Level Alignment (SRMA) to overcome this problem.
  arXiv  Detail & Related papers  (2024-04-21T16:05:38Z)
• Adaptive Betweenness Clustering for Semi-Supervised Domain Adaptation [108.40945109477886]
  We propose a novel SSDA approach named Graph-based Adaptive Betweenness Clustering (G-ABC) for achieving categorical domain alignment. Our method outperforms previous state-of-the-art SSDA approaches, demonstrating the superiority of the proposed G-ABC algorithm.
  arXiv  Detail & Related papers  (2024-01-21T09:57:56Z)
• Analyzing the Latent Space of GAN through Local Dimension Estimation [4.688163910878411]
  style-based GANs (StyleGANs) in high-fidelity image synthesis have motivated research to understand the semantic properties of their latent spaces. We propose a local dimension estimation algorithm for arbitrary intermediate layers in a pre-trained GAN model. Our proposed metric, called Distortion, measures an inconsistency of intrinsic space on the learned latent space.
  arXiv  Detail & Related papers  (2022-05-26T06:36:06Z)
• Region-Based Semantic Factorization in GANs [67.90498535507106]
  We present a highly efficient algorithm to factorize the latent semantics learned by Generative Adversarial Networks (GANs) concerning an arbitrary image region. Through an appropriately defined generalized Rayleigh quotient, we solve such a problem without any annotations or training. Experimental results on various state-of-the-art GAN models demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2022-02-19T17:46:02Z)
• HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning [74.76431541169342]
  Zero-shot learning (ZSL) tackles the unseen class recognition problem, transferring semantic knowledge from seen classes to unseen ones. We propose a novel hierarchical semantic-visual adaptation (HSVA) framework to align semantic and visual domains. Experiments on four benchmark datasets demonstrate HSVA achieves superior performance on both conventional and generalized ZSL.
  arXiv  Detail & Related papers  (2021-09-30T14:27:50Z)
• Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs [7.443321740418409]
  We propose a method to find local-geometry-aware traversal directions on the intermediate latent space of Generative Adversarial Networks (GANs) Motivated by the intrinsic sparsity of the latent space, the basis is discovered by solving the low-rank approximation problem of the differential of the partial network.
  arXiv  Detail & Related papers  (2021-06-13T10:29:42Z)
• Generalizable Representation Learning for Mixture Domain Face Anti-Spoofing [53.82826073959756]
  Face anti-spoofing approach based on domain generalization(DG) has drawn growing attention due to its robustness forunseen scenarios. We propose domain dy-namic adjustment meta-learning (D2AM) without using do-main labels. To overcome the limitation, we propose domain dy-namic adjustment meta-learning (D2AM) without using do-main labels.
  arXiv  Detail & Related papers  (2021-05-06T06:04:59Z)
• Alleviating Semantic-level Shift: A Semi-supervised Domain Adaptation Method for Semantic Segmentation [97.8552697905657]
  A key challenge of this task is how to alleviate the data distribution discrepancy between the source and target domains. We propose Alleviating Semantic-level Shift (ASS), which can successfully promote the distribution consistency from both global and local views. We apply our ASS to two domain adaptation tasks, from GTA5 to Cityscapes and from Synthia to Cityscapes.
  arXiv  Detail & Related papers  (2020-04-02T03:25:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.