LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions

• URL: http://arxiv.org/abs/2104.00820v1
• Date: Fri, 2 Apr 2021 00:11:22 GMT
• Title: LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions
• Authors: O\u{g}uz Kaan Y\"uksel, Enis Simsar, Ezgi G\"ulperi Er, Pinar Yanardag
• Abstract summary: We propose a contrastive-learning-based approach for discovering semantic directions in the latent space of pretrained GANs. Our approach finds semantically meaningful dimensions compatible with state-of-the-art methods.
• Score: 0.02294014185517203
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent research has shown great potential for finding interpretable directions in the latent spaces of pre-trained Generative Adversarial Networks (GANs). These directions provide controllable generation and support a wide range of semantic editing operations such as zoom or rotation. The discovery of such directions is often performed in a supervised or semi-supervised fashion and requires manual annotations, limiting their applications in practice. In comparison, unsupervised discovery enables finding subtle directions a priori hard to recognize. In this work, we propose a contrastive-learning-based approach for discovering semantic directions in the latent space of pretrained GANs in a self-supervised manner. Our approach finds semantically meaningful dimensions compatible with state-of-the-art methods.

Related papers

• Decoding Diffusion: A Scalable Framework for Unsupervised Analysis of Latent Space Biases and Representations Using Natural Language Prompts [68.48103545146127]
  This paper proposes a novel framework for unsupervised exploration of diffusion latent spaces. We directly leverage natural language prompts and image captions to map latent directions. Our method provides a more scalable and interpretable understanding of the semantic knowledge encoded within diffusion models.
  arXiv  Detail & Related papers  (2024-10-25T21:44:51Z)
• Detecting Adversarial Directions in Deep Reinforcement Learning to Make Robust Decisions [8.173034693197351]
  We propose a novel method to detect the presence of non-robust directions in MDPs. Our method provides a theoretical basis for the fundamental cut-off between safe observations and adversarial observations. Most significantly, we demonstrate the effectiveness of our approach even in the setting where non-robust directions are explicitly optimized to circumvent our proposed method.
  arXiv  Detail & Related papers  (2023-06-09T13:11:05Z)
• Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models [21.173910627285338]
  Denoising Diffusion Models (DDMs) have emerged as a strong competitor to Generative Adversarial Networks (GANs) In this paper, we explore the properties of h-space and propose several novel methods for finding meaningful semantic directions within it. Our approaches are applicable without requiring architectural modifications, text-based guidance, CLIP-based optimization, or model fine-tuning.
  arXiv  Detail & Related papers  (2023-03-20T12:59:32Z)
• Unsupervised Interpretable Basis Extraction for Concept-Based Visual Explanations [53.973055975918655]
  We show that, intermediate layer representations become more interpretable when transformed to the bases extracted with our method. We compare the bases extracted with our method with the bases derived with a supervised approach and find that, in one aspect, the proposed unsupervised approach has a strength that constitutes a limitation of the supervised one and give potential directions for future research.
  arXiv  Detail & Related papers  (2023-03-19T00:37:19Z)
• Discovering Class-Specific GAN Controls for Semantic Image Synthesis [73.91655061467988]
  We propose a novel method for finding spatially disentangled class-specific directions in the latent space of pretrained SIS models. We show that the latent directions found by our method can effectively control the local appearance of semantic classes.
  arXiv  Detail & Related papers  (2022-12-02T21:39:26Z)
• Fantastic Style Channels and Where to Find Them: A Submodular Framework for Discovering Diverse Directions in GANs [0.0]
  StyleGAN2 has enabled various image generation and manipulation tasks due to its rich and disentangled latent spaces. We design a novel submodular framework that finds the most representative and diverse subset of directions in the latent space of StyleGAN2. Our framework promotes diversity by using the notion of clusters and can be efficiently solved with a greedy optimization scheme.
  arXiv  Detail & Related papers  (2022-03-16T10:35:41Z)
• Self-supervised Enhancement of Latent Discovery in GANs [2.277447144331876]
  We propose Scale Ranking Estimator (SRE) which is trained using self-supervision. SRE enhances the disentanglement in directions obtained by existing unsupervised disentanglement techniques. We also show that the learned SRE can be used to perform Attribute-based image retrieval task without further training.
  arXiv  Detail & Related papers  (2021-12-16T12:36:40Z)
• Contrastive Instruction-Trajectory Learning for Vision-Language Navigation [66.16980504844233]
  A vision-language navigation (VLN) task requires an agent to reach a target with the guidance of natural language instruction. Previous works fail to discriminate the similarities and discrepancies across instruction-trajectory pairs and ignore the temporal continuity of sub-instructions. We propose a Contrastive Instruction-Trajectory Learning framework that explores invariance across similar data samples and variance across different ones to learn distinctive representations for robust navigation.
  arXiv  Detail & Related papers  (2021-12-08T06:32:52Z)
• Can Semantic Labels Assist Self-Supervised Visual Representation Learning? [194.1681088693248]
  We present a new algorithm named Supervised Contrastive Adjustment in Neighborhood (SCAN) In a series of downstream tasks, SCAN achieves superior performance compared to previous fully-supervised and self-supervised methods. Our study reveals that semantic labels are useful in assisting self-supervised methods, opening a new direction for the community.
  arXiv  Detail & Related papers  (2020-11-17T13:25:00Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)
• Unsupervised Discovery of Interpretable Directions in the GAN Latent Space [39.54530450932134]
  latent spaces of GAN models often have semantically meaningful directions. We introduce an unsupervised method to identify interpretable directions in the latent space of a pretrained GAN model. We show how to exploit this finding to achieve competitive performance for weakly-supervised saliency detection.
  arXiv  Detail & Related papers  (2020-02-10T13:57:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.