An endless world of images, done by the machine

For some years now I've been loosely following the AI space with quite some interest and how some professional fields are being pushed into a corner by improved AI that, in time, will take over work tasks from people. Some things are already using AI in some capacity and for sure it will be more. Soon. One of the most talked about fields has been self driving cars. And they are well on their way. It's enough to watch the Youtube channel 'AI Driver' to see that.

Previously I took some comfort in the notion that the creative fields would be among the last to be affected by AI. That was before DALL-E showed up, which is a text to image generator from 'Open AI'. Some of you may have seen images created by DALL-E 2 which is staggeringly correct and beatiful. But off course, DALL-E 2 is just one system out of several. This area is worked on by many. Google has it's 'Imagen' system recently announced.

'Midjourney'

So just to inform myself I decided to try this thing out. How should I as a professional designer approach this? Is there a need for worry or will this be just one of many tools I use in a few years time?

The header image for this article was created using 'Midjourney', and the text to create it was: "eyeball with insect legs, realistic". Then there are some 'modifiers' that determine the style of it. In 60 seconds Midjourney spits out 4 variations. And that's more or less all I have to do. An important thing to remember here is that this is Beta. Non of these systems are widely available yet. Which means they will improve fast in the coming years.

A few examples of some of the images I generated just after getting access to Midjourney:

So, as you can see some prompts works better than others. Or, rather some interpretations of the AI matches our own expectations better than others. Abstract images works pretty well and turn out great, while humans and animals work less well, as you can see in this example it made of the Devil.

Modifers

Many known artist have now become a 'modifier' in these systems. I can write: "Skateboarding worm in the style of Ruben, Michealangelo, Chriss Foss, Giger etc and it will generate in that style. So then the question about who owns that 'modifier' or style starts? After all, some artists have spent an entire career perfecting a certain style. And even brands like Unreal Engine and Artstation has become modifiers. Just as some words are banned for sensitive content reasons, maybe well known names and brands will come with a fee to use in these systems? (After commenting this I felt this should be in here as it's a really important question.)

Controls and direction

Also, it seems to favor a straight on angle to the subject and this is where the lack of controls starts to become apparent. I tried to write in 'three quarter view' as a modifier, but that didn't really do much.

This exmaple also shows a similar problem. I couldn't get this tank to be full view or a 'wide shot' no matter what prompts I tried. It always cropped it by rendering it too close. It's easy to see improvements being made here to gain some more control or, you should just look at it like you're directing someone else. On the other hand I'm a total noob at this, so there's that.

Another thing it struggles with is relatonships. It doesn't really understand 'on', 'inside', 'beside' and things like that, which currently also makes it harder to direct it.

While trying prompts for the above image I encountered censorship for the first time. Certain words are banned from the system. In this case it was 'Flesh'. This is to prevent obscene and upsetting content to be created which opens up a whole new can of worms. The authors of these upcoming systems will, in some form or another, dictate what you can create and what not. Obviously this is an important topic as you don't want people to be able to create images of well known people doing criminal, acts for instance.

Some things it just can't get right though. Like this one of a supposed Yeti.

As you can see the AI creates variations of the prompt you enter and from there you can iterate on it indefinitely.

Diffusion

Most people's initial reaction to this tech is that this is just photobashing from millions of images across the internet to create new images; following the prompt the user wrote. This is not what is happening here. The system uses something called 'Diffusion', which is a rendering model that starts of with an image of noise and iterates on that until it meets the prompts decription. It's like image tagging in reverse, so there's a languade model in the works here too called GTP-3. It's better explained in this video by Cold Fusion TV.

While we're on the techy stuff I can mention that this doesn't require a monster machine to use. I can use it on my phone if I want to, as the test prompts are sent to a server and executed there. It then spits back the images to me to review.

Reflections

I must say though that it's really fun to just play around with this and see what comes out. I sat well into the night the first try I had with this and each image sparks new ideas. As for concepting ideas goes this is unbeatable, I would say, when it's fully ready. The AI almost always creates the unexpected, which can both be frustrating and fun, depending how you use it.

Another thing I noticed was the lack of ownership I felt about these images. I didn't really care about loosing them or saving them because I didn't really put any effort into them. Had I made the above asteroid skulle in ZBrush for instance and rendered it in Keyshot, I would have been pretty pleased with it and perhaps even put it into my portfolio. Now it's really not that important to me, which I find interesting.

Naturally this is already able to create animations as well. And with Nvidias latest announcement about their tech that can create 3d scenes from images, it will soon be able to do 3d content too.