Locality-preserving Directions for Interpreting the Latent Space of Satellite Image GANs

• URL: http://arxiv.org/abs/2309.14883v1
• Date: Tue, 26 Sep 2023 12:29:36 GMT
• Title: Locality-preserving Directions for Interpreting the Latent Space of Satellite Image GANs
• Authors: Georgia Kourmouli, Nikos Kostagiolas, Yannis Panagakis, Mihalis A. Nicolaou
• Abstract summary: We present a locality-aware method for interpreting the latent space of wavelet-based Generative Adversarial Networks (GANs) By focusing on preserving locality, the proposed method is able to decompose the weight-space of pre-trained GANs and recover interpretable directions.
• Score: 20.010911311234718
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a locality-aware method for interpreting the latent space of wavelet-based Generative Adversarial Networks (GANs), that can well capture the large spatial and spectral variability that is characteristic to satellite imagery. By focusing on preserving locality, the proposed method is able to decompose the weight-space of pre-trained GANs and recover interpretable directions that correspond to high-level semantic concepts (such as urbanization, structure density, flora presence) - that can subsequently be used for guided synthesis of satellite imagery. In contrast to typically used approaches that focus on capturing the variability of the weight-space in a reduced dimensionality space (i.e., based on Principal Component Analysis, PCA), we show that preserving locality leads to vectors with different angles, that are more robust to artifacts and can better preserve class information. Via a set of quantitative and qualitative examples, we further show that the proposed approach can outperform both baseline geometric augmentations, as well as global, PCA-based approaches for data synthesis in the context of data augmentation for satellite scene classification.

Related papers

• Leveraging edge detection and neural networks for better UAV localization [5.781342655426309]
  We propose a novel method for geolocalizing Unmanned Aerial Vehicles (UAVs) in environments lacking Global Navigation Satellite Systems (GNSS) Current state-of-the-art techniques employ an offline-trained encoder to generate a vector representation (embedding) of the UAV's current view. We demonstrate that the performance of these methods can be significantly enhanced by preprocessing the images to extract their edges.
  arXiv  Detail & Related papers  (2024-04-09T10:56:46Z)
• View Consistent Purification for Accurate Cross-View Localization [59.48131378244399]
  This paper proposes a fine-grained self-localization method for outdoor robotics. The proposed method addresses limitations in existing cross-view localization methods. It is the first sparse visual-only method that enhances perception in dynamic environments.
  arXiv  Detail & Related papers  (2023-08-16T02:51:52Z)
• Discovering Class-Specific GAN Controls for Semantic Image Synthesis [73.91655061467988]
  We propose a novel method for finding spatially disentangled class-specific directions in the latent space of pretrained SIS models. We show that the latent directions found by our method can effectively control the local appearance of semantic classes.
  arXiv  Detail & Related papers  (2022-12-02T21:39:26Z)
• Unsupervised Discovery of Semantic Concepts in Satellite Imagery with Style-based Wavelet-driven Generative Models [27.62417543307831]
  We present the first pre-trained style- and wavelet-based GAN model that can synthesize a wide gamut of realistic satellite images. We show that by analyzing the intermediate activations of our network, one can discover a multitude of interpretable semantic directions.
  arXiv  Detail & Related papers  (2022-08-03T14:19:24Z)
• Incorporating Texture Information into Dimensionality Reduction for High-Dimensional Images [65.74185962364211]
  We present a method for incorporating neighborhood information into distance-based dimensionality reduction methods. Based on a classification of different methods for comparing image patches, we explore a number of different approaches.
  arXiv  Detail & Related papers  (2022-02-18T13:17:43Z)
• IGAN: Inferent and Generative Adversarial Networks [0.0]
  IGAN learns both a generative and an inference model on a complex high dimensional data distribution. It extends the traditional GAN framework with inference by rewriting the adversarial strategy in both the image and the latent space.
  arXiv  Detail & Related papers  (2021-09-27T21:48:35Z)
• Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs [7.443321740418409]
  We propose a method to find local-geometry-aware traversal directions on the intermediate latent space of Generative Adversarial Networks (GANs) Motivated by the intrinsic sparsity of the latent space, the basis is discovered by solving the low-rank approximation problem of the differential of the partial network.
  arXiv  Detail & Related papers  (2021-06-13T10:29:42Z)
• Low-Rank Subspaces in GANs [101.48350547067628]
  This work introduces low-rank subspaces that enable more precise control of GAN generation. LowRankGAN is able to find the low-dimensional representation of attribute manifold. Experiments on state-of-the-art GAN models (including StyleGAN2 and BigGAN) trained on various datasets demonstrate the effectiveness of our LowRankGAN.
  arXiv  Detail & Related papers  (2021-06-08T16:16:32Z)
• Joint Characterization of Multiscale Information in High Dimensional Data [0.0]
  We propose a multiscale joint characterization approach designed to exploit synergies between global and local approaches to dimensionality reduction. We show that joint characterization is capable of detecting and isolating signals which are not evident from either PCA or t-sne alone.
  arXiv  Detail & Related papers  (2021-02-18T23:33:00Z)
• Joint and Progressive Subspace Analysis (JPSA) with Spatial-Spectral Manifold Alignment for Semi-Supervised Hyperspectral Dimensionality Reduction [48.73525876467408]
  We propose a novel technique for hyperspectral subspace analysis. The technique is called joint and progressive subspace analysis (JPSA) Experiments are conducted to demonstrate the superiority and effectiveness of the proposed JPSA on two widely-used hyperspectral datasets.
  arXiv  Detail & Related papers  (2020-09-21T16:29:59Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.