The art of my AI algorithm became an exhibit at a digital art exhibition

Hello, my name is Oleh and I am interested in AI image generation. Recently, my algorithm XFutuRestyle attracted the attention of OpenAI, and its results were presented at an international digital art exhibition. Now I will tell you how it happened.

How XFutuRestyle came to be

I started experimenting with new visual effects on Bing Image Creator when it was on DALL-E 2, and with the release of DALL-E 3, I started to discover even more interesting possibilities.

The turning point came when OpenAI released GPTs - that's when I found a new way to create images from multiple photos.

However, the implementation of such a technology turned out to be quite complicated, and I conducted tests every day, gradually adjusting the algorithm and minimizing the number of errors. Each iteration of the setup included three stages:

1. Specific requirements for image analysis. Each existing image must be analyzed separately before creating a new one. This requires an accurate and thorough study of the image properties to ensure that the future image meets these requirements.
2. Exactly following the instructions. The algorithm must follow the instructions provided exactly, which makes the development and testing process longer and more complicated, since every little detail is important.
3. Continuous testing and customization. Since the algorithm is complex and has many nuances, it is necessary to conduct numerous tests to ensure its stability and compliance with expectations. And because the photos may not be compatible at all, the result can be very unpredictable

Eventually, I developed the XFutuRestyle algorithm, which can automatically combine up to four photos or images.

Here's how it works:

• several images are uploaded;
• each image is analyzed and a very complex clue is created based on the input data;
• then, based on this clue, an image is created with a combination of all the elements. This combination is quite complex, so the DALL-E 3 cannot always reproduce it.

GPT developer interface and an example of a successful test

The developer interface looks like this, divided into two windows. In the first window, you can set instructions for GPT control, model training, and debugging, and in the second window, you can test the changes made.

At the bottom, there is an option to upload your database and also involve the use of web search, DALL-E and a code interpreter, which also makes it possible to perform data analysis with computer vision.

Here is one of the successful tests of the algorithm:

This is the result of the transfer and transformation of objects from three photos:

The point is that a multi-layered complex scene is created from a minimum amount of input data.

The process of building a scene can even resemble Sora in some ways (I implemented this method when Sora was not even announced yet) in creating images for an example like this one. That is, all elements fit optimally into the scene.

The advantages of this method are: obtaining temporary datasets based on input data, imitating creative thinking by creating an incredible number of combinations during analysis.

Promt

Speaking of building a promt, here's how complex a query we managed to create from just three photos:

A wide-angle abstract futuristic cityscape at dusk in high resolution, combining the industrial essence of a brewery with organic elements of nature and a majestic sunset. Visualize the large metal tanks and pipelines, permeated with natural, fresh greenery, and the bright, warm hues of the sunset sky reflected on their surfaces. Integrate industrial and organic components to create a neon-lit scene that embodies the fusion of technology and nature. The entire scene is lit with high-contrast light that emphasizes textures and materials, casting long, dramatic shadows and glowing highlights that create a sense of depth and complexity. Add a touch of abstraction to the details, blurring the lines between natural and artificial, with skies that transition from clear sunset to starry night, suggesting the passage of time in a single frame.

Art On LOOP exhibition and OpenAI reaction

The idea arose when I was trying to combine images and I suddenly wondered: what if I tried to combine elements from four photos?

The sun + shark + oriental architecture + a train crossing the bridge = a result that exceeded all expectations.

One day I came across a video titled "Artworks Created by Artificial Intelligence Showcased in Hong Kong" - that's when I started working very hard to turn my idea into reality.

I found an exhibition that offered an innovative format - the world's first presentation of an image on a screen in two countries at once - and I was interested. I sent the image to the organizers and received a reply: "Your exceptional talent has impressed us, and we are happy to showcase your work." That's how I ended up on the walls of the exhibition Art On LOOP. London - Athens.

And here is the response I received a month after the exhibition from OpenAI regarding my appeal:

It is truly inspiring to hear about Ukraine's achievement in GPT-4 image creation, which was recognized at the International Digital Art Exhibition in London. Recognizing such innovative applications of artificial intelligence technology not only highlights the creative potential of these tools, but also demonstrates the talent and resilience of communities around the world, including Ukraine's significant contribution.

At OpenAI, we are always happy to see diverse and effective ways to use our technologies. Recognizing such achievements not only supports the communities involved, but also aligns with our mission to ensure that artificial intelligence benefits all of humanity. While we cannot commit to specific promotions, we deeply appreciate your suggestion and will certainly consider how we can support and recognize the achievements of users from around the world, including Ukraine.

We are eager to explore ways to amplify such stories, recognizing the value they bring to the community and the world at large. Thank you again for your proposal and for highlighting Ukraine's achievements. We are honored to play a role in the creative and innovative endeavors of the global community.

Conclusion

Why did I take on such a complex project? Because I was curious about how to get interesting images from simple free photos. In the process, I realized that I had the opportunity to demonstrate that Ukraine can really reach a new quality level in the field of AI according to international standards, even during the war.

I was constantly studying the capabilities and limitations of DALL-E and GPT-4 to be able to realize this, even if it was not completely stable. In fact, I tried to create a very powerful benchmark for DALL-E 3. I tried to implement all this on my own after a significant life shock. That's right, it turns out that when I lost a lot of things, this was the reason for creating an innovative solution that could attract the attention of OpenAI to Ukraine.