GAN Inversion
GAN inversion aims to invert a given image back into the latent space of a pre-trained GAN model so that the image can be faithfully reconstructed from the inverted code by the generator. As an emerging technique to bridge the real and fake image domains, GAN inversion is essential in enabling pre-trained GAN models, such as StyleGAN and BigGAN, for applications of real image editing. Moreover, GAN inversion interprets GAN’s latent space and examines how realistic images can be generated.
In this excellent article by Weihao Xia et al., the authors provide a survey of GAN inversion, focusing on its representative algorithms and applications in image restoration and manipulation.
(a) Given a well-trained GAN model G, photo-realistic images x can be generated from randomly sampled latent vectors z. GAN inversion aims to obtain the latent code z for a given image x. A learning-based inversion method aims to learn an encoder network to map an image into the latent space such that the reconstructed image based on the latent code look as similar to the original one as possible.
An optimization-based inversion approach directly solves the objective function through back-propagation to find a latent code that minimizes pixel-wise reconstruction loss.
A hybrid approach first uses an encoder to generate initial latent code and then refines it with an optimization algorithm. Depicted by the dotted E, the well-trained encoder is included as a regularizer for optimization. Blue blocks represent trainable or iterative modules, and red dashed arrows indicate the supervision.