Use of AI for Concept Art

AI will undoubtedly shape our work life in the upcoming future. A recent increase in quality by AI-generated pictures is making headlines as an AI recently won an art prize (1). This begs the question if there is potential for the use of artificial intelligence for tasks, which require visualization, such as finding a creative direction in designing.

As a driving idea factory AI could potentially serve as a concept art generator for first iterations. The market for AI art is arguably being lead by the Midjourney AI (2) and the second version of DALLE - DALLE2 (3). Most AI interfaces use the so-called prompt text as input: Users simply describe the imagined picture and press on create to finalize their idea. However, some interfaces also offer the option to upload an existing picture as foundation, which then can be changed via the prompt text. It should be noted that using existing pictures and changing them slightly might violate copy rights or underlying terms and conditions of AI This is ethically questionable if used for commercial reasons. Thus, this guide will focus on the use of AI as a shapeshifter for directional creative decision making.

It is notable that most of the generated pictures miss the direction of the text input. However, designers can and will improve their results by changing specific keywords such as alternative descriptions, perspectives or common buzzwords. An in-depth guide on how to talk to AI on DALLE2 is presented by the DALLE2 prompt book - a highly recommended read (4).

While most AI Art generators offer free trials, an extensive usage is required for improving created content through failing and learning. A great learning resource on how to talk to AI can be found on the MidJourney Discord, because it publishes and displays user prompts and the finalized picture.

After discussing the opportunities of AI as a creative tool this part of the guide will display a practical walkthrough on how to get better results and improve underlying pictures. In theory existing AIs can create landscapes, characters, animals, objects and so on. However, most AIs still have problems with symmetrical face design and very specific prompts.

I have chosen Nightcafe as AI platform (5), because it features different AI picture generators and has the option to create pictures in bulk. It also has multiple payment solutions and is relatively cost efficient. As a first step after signing up at Nightcafe and pressing on create I recommend to enable advanced options. This offers better customization.

In the advanced settings we find multiple options. As a general guideline when creating new prompts, I recommend to start with a bulk of pictures and then refining the results. This will generate better pictures in the long run and prevents your credits from getting used up to fast. The slider for prompt weight configures the "freedom" of the AI and decreasing it might result in more random results. From experience the best results are generated in a range from 40% - 60%.

Finally in step 3 we decide on how much GPU power is used on the generation of the picture. While adding runtime will often greatly increase the output quality it will also greatly increase the creation costs. I highly recommend to choose a short run time with a low output resolution to generate a first prototype batch of pictures and only increasing the runtime for evolving and refining pictures.

After creating our first batch of images we can either create more batches if we are unhappy with the batch-quality or we can improve a prototype of the first batch. In the following example I created a rhinoceros walking on ice. As you can see in this example, I used different keywords such as "full polish" or "cinematic vfx" to further shape my desired result. The linked textbook of DALLE at the end of this guide can help you improve your prompt texts (4). You can evolve a chosen picture of your first prototype by selecting it from the generated batch, pressing the eye button and then choosing the evolve option. I highly recommend to choose a longer runtime now but still the thumbnail resolution.

Finally, once we are happy with our creation, we can simply upscale it by choosing the up-scaling option in the creation overview. The following picture displays an up-scaled result of our first prompt.

I hope this guide offered a new perspective on a potential new tool which might shape the future of creative work in game development and beyond.

(1) https://www.nytimes.com/2022/09/02/technology/ai-artificial-intelligence-artists.html

(2) https://www.midjourney.com/

(3) https://openai.com/dall-e-2/

(4) https://dallery.gallery/the-dalle-2-prompt-book/

(5) https://creator.nightcafe.studio/