??#83: GAN is back
TuringPost
Newsletter about AI and ML. ?? Sign up for free to get your list of essential AI resources ??
?? Turing Post has been invited to join ?? Hugging Face as a resident -> click to follow!
Now, to the main topic:
Last week’s headlines were dominated by timid CEOs and burning events in LA, with CES coverage flooding every feed. In was intense, and we found ourselves yearning for the comfort of good old machine learning. So today, we’re revisiting a classic: GANs. Are they still worthy of their title as one of the most captivating ideas in ML?
This overview is inspired by the recent paper “The GAN Is Dead; Long Live the GAN!”. As always, let’s begin with our favorite starting point – a refreshing dive into history.
The Birth of GANs: A Game of Two Networks
The paper “Generative Adversarial Nets” was introduced in 2014 by Ian Goodfellow and his team. The concept was simple yet revolutionary: two neural networks, a generator and a discriminator, compete in a zero-sum game.
This adversarial training forces both networks to improve, eventually producing synthetic data that’s indistinguishable from the real thing.
This approach turned to be very effective. To the point that in 2016 Yann LeCun said that “it’s the best idea we had in a bit”.
Compared to earlier generative models like Variational Autoencoders (VAEs) and Restricted Boltzmann Machines (RBMs), GANs generated sharper images, learned more intricate patterns, and opened up new possibilities.
The excitement around GANs was palpable but training challenges, like instability and mode collapse, were also real.
The Shift to Diffusion Models
As the years passed, those training difficulties became harder to ignore. Around 2022, a new challenger emerged: diffusion models. These models approached data generation as a gradual refinement process, which made them more stable and easier to train.
Diffusion models quickly stole the spotlight, offering high-quality, diverse outputs and fewer headaches for researchers. GANs, once the star of generative modeling, began to fade from the conversation.
The GAN Is Dead; Long Live the GAN!
Not from the conversations of the true believers! Just a few days ago, in this brand new 2025, a paper with the bold title “The GAN Is Dead; Long Live the GAN!” reignited interest in GANs. Written by Yiwen Huang, Aaron Gokaslan, Volodymyr Kuleshov, and James Tompkin, the paper argued that GANs’ challenges were more about outdated architectures and techniques than inherent flaws.
领英推荐
At the heart of this idea is a better loss function – think of it as a smarter way for the GAN to measure how well it’s learning. They call it the relativistic GAN loss. It makes the GAN training process smoother and less prone to common problems like weird artifacts or getting stuck generating only a small set of images.
The researchers also modernized the GAN architecture. They started with StyleGAN2 (a popular model known for generating photorealistic faces) and stripped out all the extra stuff that’s no longer necessary thanks to recent advancements in AI design. They added better building blocks, like ResNets and grouped convolutions, to create a leaner, meaner GAN called R3GAN.
This new approach not only works better but is also simpler. On standard benchmarks like FFHQ (a dataset of human faces) and CIFAR-10 (smaller images of everyday objects), R3GAN beats existing models, including some diffusion models. Plus, it’s faster to train and uses less computing power.
If you’ve avoided GANs because they seemed too fiddly or outdated, this might be the perfect time to give them another shot. R3GAN makes the whole process way more accessible. It’s time to rethink what GANs can do.
Iterative nature of ML innovation
The revival of GANs is a reminder of the iterative nature of machine learning innovation. GANs remain relevant because they solve real problems efficiently. Their ability to generate high-quality synthetic data is even more critical now, as the demand for genAI data overwhelms available sources. This is especially important in industries like healthcare, where privacy concerns make sharing real-world data challenging.
Curated Collections (ex Twitter library)
Inspired by Agent Laboratory: Using LLM Agents as Research Assistants by AMD and Johns Hopkins University and LLM4SR: A Survey on LLMs for Scientific Research by University of Texas at Dallas, we put together this collection:
Do you like Turing Post? –> Click 'Follow'! And subscribe to receive it straight into your inbox -> https://www.turingpost.com/subscribe
We are reading
The freshest research papers, categorized for your convenience: