Interview on the Use of AI in Creative Professions. Question 3.

Recently, I was invited by a master's student to share my perspectives and experiences with AI text-to-image generators like DALL-E 2, Midjourney, Stable Diffusion, and others. In a thorough interview, I answered ten poignant questions on this exciting topic. If you're curious about these questions and my views on them, I warmly invite you to take a look. And if you have additional thoughts or questions about it, I would love to hear from you.What is your current personal opinion on this?

Question 3.

How have you engaged with text-to-image generators in terms of extent and intensity?

(Workflow, Tools, Approach...)

Initial Steps: Image Editing with AI

Initially, I focused primarily on refining prompts and generating images. These were later post-processed and reassembled using image editing programs like Adobe Photoshop or Serif Affinity Photo.

?

Fascination with Stable Diffusion:

Local AI Integration

My interest soon turned to the AI model Stable Diffusion, which can be installed locally on a computer and utilizes the AI capabilities of the graphics card. I found experimenting with the Control Net (Image 1.) particularly exciting. As I create 3D illustrations and animations, I wanted to find out how the workflow could be optimized through the integration of AI.

?

3D Experiments:

From Geometry to Painterly Scenes with Control-Net

My approach was to work with three prototype images.

For instance, I used simple boxes or other geometric shapes to symbolize buildings. From a certain camera angle, I extracted the depth information (Z-buffer) with 3D software or the AI-Control-Net component "Depth" (Image 1).

This served as a basis for generating images with the following prompt and the "control-net" function: "A cute, funny, stylized, low-poly 3D illustration of an Italian village street with flowers and plants on wooden windows on a sunny day with blue sky." (Image 2.)

?

Two other variants depict a cyberpunk scene (Image 3.) and a Wild West town in the rain (Image 4.).

Innovation in 3D Software:

3dsMax Meets AI

Currently, I am working on a tool for the 3D software 3dsMax that enables the same process, which AI coding enthusiasts have already implemented in 3D programs like Blender or Houdini, to be possible in 3dsMax as well. I am in the midst of programming and looking forward to the results!

?

Another project was a spaceship in a desert.

I created the image using a 3D program (Image 6.) and then used it as a basis for various prompts.

I interpreted these with varying noise settings in the 'Image2Image' function of 'Stable Diffusion'.

?

You can clearly see how the higher noise factor interprets the shapes more "creatively" and adds details.