Abstract: We present an alternative perspective on the training of generative
adversarial networks (GANs), showing that the training step for a GAN generator
decomposes into two implicit sub-problems. In the first, the discriminator
provides new target data to the generator in the form of "inverse examples"
produced by approximately inverting classifier labels. In the second, these
examples are used as targets to update the generator via least-squares
regression, regardless of the main loss specified to train the network. We
experimentally validate our main theoretical result and discuss implications
for alternative training methods that are made possible by making these
sub-problems explicit. We also introduce a simple representation of inductive
bias in networks, which we apply to describing the generator's output relative
to its regression targets.