Abstract: The latent space of a Generative Adversarial Network (GAN) has been shown to
encode rich semantics within some subspaces. To identify these subspaces,
researchers typically analyze the statistical information from a collection of
synthesized data, and the identified subspaces tend to control image attributes
globally (i.e., manipulating an attribute causes the change of an entire
image). By contrast, this work introduces low-rank subspaces that enable more
precise control of GAN generation. Concretely, given an arbitrary image and a
region of interest (e.g., eyes of face images), we manage to relate the latent
space to the image region with the Jacobian matrix and then use low-rank
factorization to discover steerable latent subspaces. There are three
distinguishable strengths of our approach that can be aptly called LowRankGAN.
First, compared to analytic algorithms in prior work, our low-rank
factorization of Jacobians is able to find the low-dimensional representation
of attribute manifold, making image editing more precise and controllable.
Second, low-rank factorization naturally yields a null space of attributes such
that moving the latent code within it only affects the outer region of
interest. Therefore, local image editing can be simply achieved by projecting
an attribute vector into the null space without relying on a spatial mask as
existing methods do. Third, our method can robustly work with a local region
from one image for analysis yet well generalize to other images, making it much
easy to use in practice. Extensive experiments on state-of-the-art GAN models
(including StyleGAN2 and BigGAN) trained on various datasets demonstrate the
effectiveness of our LowRankGAN.