Exploring Imagen: Google's Research-Driven Approach to Photorealistic AI Art

While conducting research for yesterday's article about the German open-source image model "FLUX," I discovered Google's research model, Imagen (also known as ImageFX in Test Labs).

I've been fascinated by the work of Google's Test Labs ever since I was first sent an invitation to start trying out early features. Ummm... play around with early versions of fun and nerdy things? Yes, please! If you're interested in participating in Google Labs, you can learn more about it and how to sign up through this Google Labs blog post.

What is Imagen?

Google's research paper, "Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding" describes the challenges behind creating images from a prompt that look just like real photos and perfectly match the description. (Note: this paper is from May 2022, but based upon the quality of the images I was able to generate from the model, I suspect work on ImageFX has been ongoing.)

Developed by Google’s Brain Team, Imagen sets a new standard for generating high-quality, realistic images that deeply understand and align with text. Or, to put it more simply, the model does a better job of reading and interpreting the prompt.

Imagen In Action

I first noticed this difference yesterday when I was doing the model testing for my FLUX article.

I entered my test prompt: An AI robot is spray painting a wall in a futuristic city. Gorgeous. Beautifully rendered. Colorful. Cyberpunk. He is spray painting the words: "Flux." "DALL-E." "Midjourney." "Stable Diffusion."

ImageFX started highlighting words like a college student trying to make sense of a physics textbook.

ImageFX pulled out the key words: "AI robot," "spray painting," "futuristic city," "spray painting," and then two of the model names. Although, interestingly enough, in the final render, the ImageFX generated three of the model names, and possibly the fourth. (See the second image below. It is possible the word "DALL-E" is blocked by the robot itself.)

This adherence to text sets Imagen apart from other models. Language models often follow prompts like a distracted teenager. They hear bits in the beginning and bits in the end, but tend to tune out everything you say in the middle.

Case in point. The image below by StableDiffusion XL did a great job demonstrating what happens when a model only reads a portion of the prompt. The model tuned out about a third of the prompt, and did not spray paint the words on the wall, and only named one of the models requested.

The Strengths of Imagen

Imagen uses advanced AI models called Transformers to understand and generate text-based content. One of these models is T5, short for "Text-To-Text Transfer Transformer." T5 is specifically designed to understand complex language and convert it into something else—like turning a description into an image.

In Imagen, T5 plays a crucial role by deeply understanding the text prompt and ensuring that the images generated match the description as accurately as possible.

The key insight in this research is that when we make T5 larger—meaning it has more capacity to understand and process language—the quality of the generated images improves significantly. This improvement is even greater than what you would get by just making the image generation part of the model bigger. Essentially, the better T5 understands the text, the more lifelike and accurate the images Imagen creates.

This approach highlights how important it is to focus on the text understanding part of the model to achieve the best possible results in AI-driven image generation.

The Core Idea (A "Frozen" Model)

Imagen leverages the Transformer architecture’s powerful text understanding to generate highly detailed images. The model uses a diffusion process, starting with random noise and gradually refining it into a coherent image based on the text input. '

Imagen uses a large, frozen language model. This means the language model has been pre-trained on a vast amount of text data and is then "frozen," or kept unchanged, during image generation. By using this frozen model, Imagen can accurately understand and translate the prompt into an image, ensuring high quality and alignment with the text.

Key Findings:

• State-of-the-Art Performance: Imagen achieves a record-breaking FID (Fréchet Inception Distance) score of 7.27 on the COCO dataset, surpassing other models like DALL-E 2.
• Human-Like Language Understanding: By using a large, frozen language model, Imagen captures nuances in text that other models miss, resulting in images that are not just visually appealing but also contextually accurate.
• Dynamic Thresholding: This new sampling technique allows Imagen to use high guidance weights without degrading image quality, leading to better photorealism and alignment with the text.

Challenges and Future Outlook

Despite its achievements, Imagen isn't without limitations. The reliance on large datasets for training raises concerns about bias and the ethical use of generated content. Additionally, while the model excels at generating non-human subjects, the research paper suggests it struggles with photorealism in images depicting people. (I did not find this to be the case in my own testing which further confirms my theory DeepBrain is continuing to refine this model. See Appendix A.)

The paper also highlights the need for future work to address biases and improve the model's ability to generate diverse and fair representations.

Future research may focus on refining the model's ability to handle more complex and nuanced text inputs, potentially integrating ethical frameworks to mitigate biases in generated content.

Final Thoughts

In a world overflowing with image generation models, Imagen stands out by not just creating pictures but by truly understanding the story behind them. What makes Imagen different is its deep alignment between text and image—a result of Google’s innovative use of large, frozen language models. This isn’t just about generating visually appealing images; it’s about crafting visuals that are intimately connected to the words that inspire them.

Imagen pushes the boundaries of what we expect from AI by delivering photorealism that feels genuinely rooted in the text, capturing nuances that other models might overlook. But it’s not just the technical prowess that sets Imagen apart—it’s the thoughtful consideration of ethics, bias, and the societal impact of such a powerful tool.

It's obvious the team at Google #DeepBrain have put a lot of thought in this model and they are not rushing it out to market. In a crowded field, Imagen is a model that dares to be different by understanding what truly matters.

Read the research paper here.

Try out the model here. (You may have to sign up for Google Labs first.)

I am a freelance writer and a retired educator. I love to read white papers and write about AI. Hey, we all need a hobby.

Additional Resources for Inquisitive Minds:

Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., Salimans, T., Fleet, D. J., & Norouzi, M. (2022). Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding. arXiv. https://doi.org/10.48550/arXiv.2205.11487

• Generalization and Fine-Tuning (Throughout the Paper) "Imagen demonstrates strong generalization capabilities, allowing it to adapt across a range of text prompts (Saharia et al., 2022)."
• Benchmark Comparisons (Page 8) "Imagen demonstrates strong generalization capabilities, allowing it to adapt across a range of text prompts (Saharia et al., 2022)."
• Ethical Considerations and Societal Impact (Page 9) "Imagen demonstrates strong generalization capabilities, allowing it to adapt across a range of text prompts (Saharia et al., 2022)."
• User Interaction and Real-World Applications (Page 9) "Imagen demonstrates strong generalization capabilities, allowing it to adapt across a range of text prompts (Saharia et al., 2022)."
• Limitations in Terms of Computational Resources (Page 9) "Imagen demonstrates strong generalization capabilities, allowing it to adapt across a range of text prompts (Saharia et al., 2022)."
• Comparisons to Other Models (Throughout the Paper)"Imagen demonstrates strong generalization capabilities, allowing it to adapt across a range of text prompts (Saharia et al., 2022)."

Appendix A:

While the Imagen paper lists generation of human subjects as a limitation, I ran it through several stress tests to generate images of humans. It passed them all with flying... eh... limbs.

I asked for the help of my DeepLearningDaily GPT in coming up with a torture test for the Imagen model.

Here are the results of the bustling street market prompt:

Ethical and Bias Testing

Ambiguous Prompts:

This prompt threw the model off and it could not create it.

Realism vs. Surrealism

This is a difficult prompt and I thought ImageFX did an excellent job on these images, even generating the veins on the hands.

#AIArt #DeepLearning #TextToImage #Imagen #GoogleAI #MachineLearning #Photorealism #AIResearch #EthicalAI #LanguageModels