How to teach AI to draw

Prequel’s first two AI effects, Cartoon and Cartoon+, went viral and became a basis for a variety of social media challenges: #cartoonme, #charactertrope, and many others. All the Prequel users that have already tried the AI effects (haven’t all of them though?) have seen the caption? “Generating AI magic…” while their pic is being edited with our neural network. To be honest, artificial intelligence remains but a pure magic for the majority of the Prequel Team members. But those who have actually created and trained the neural networks in question, see not magic, but a clear process, albeit complicated.?

For some time, our Tech Department was playing around with one that generated realistic human faces of not-existent people. We called it Model One. After that, there was a process of “educating” it. As long as it knew how to generate random faces, it could be trained to generate transformed ones as well. Transformed in whatever selected style, like a certain type of comic art. We called this newer one Model Two.

A bit on how these models work. Every one of them has a lot of so-called layers that are responsible for the resolution of the image we get as a result. The first layer has the 4x4 pixels resolution so that you can hardly guess the pose of the person while the last layer has a resolution of 1024x1024 pixels so that you can clearly see every single detail. It can be said that earlier layers are responsible for defining some general things—such as pose, for instance–and later layers are responsible for color and texture. In other words, the first layers give us the shape of the final generated image and later layers make the image look more authentic.

Then we went on to create one more neural network that actually combined the two already existing—the one that generates random human faces and the one that generates stylized images. It worked as follows: taking the first layers from the Model One (to get information about the pose) and the later layers from the Model Two (that would add texture and stylize the final image). If you have ever cooked without a recipe, using your intuition, then you’ll probably understand the feelings behind what followed. The point was to “mix” two models in the right proportion. Here, our developers did their best to find appropriate coefficients for each part, so that the characteristics of the original image were still seen and recognized in the final, stylized one. Call this Model Three.

Thing is, Model One (and, subsequently, Model Three as well) generates random faces from random numbers—don’t ask how but it just works. And a model usable in our interactive app should have an input image as a prompt. So there had to be another round of training but our Tech Department proved it was a piece of cake. With input from our Art Department that had been behind the concept for eventual image styling, we finally had our “image (user-sourced, original) to image (generated, stylized)” pipeline.?

The technology was packaged as two effects on our app, called Cartoon and Cartoon+. The results they demonstrated upon release were really impressive (let’s say viral) and reception was warm across the board. There has to be a special shoutout to the cosplay community, who we now regard as the target audience for this tool, but it felt like almost everyone enjoyed it. Then, who wouldn’t fall in love with becoming a piece of art!

Seems that artificial intelligence has so much to add to the industry of photo and video editing that its further possibilities should be definitely tested. What Prequel can actually do with it? We’re thrilled as well to find it out.