AI image generators: ‘ethically trained’ Firefly VS ‘copyright infringing’ Midjourney - which is better?

Need an AI image generator? There’s plenty to choose from, some popular ones include Midjourney, Dall-E 2, and Stable Diffusion. Canva even has one built into its tools. And now Adobe has announced their own tool, Firefly.?

So which one should you use?

I’ve been using Midjourney since last year. It produces stunning images, and has a creative flair to it that I haven’t found in other tools that I’ve tested.

But Midjourney has come under criticism for creating an “an ethical and copyright black hole. ” In fact, Midjourney is even being sued by artists for copyright infringement, because it “violated the rights of millions of artists” by training the AI on images without obtaining consent.?

We’ll just have to wait and see if their claim is upheld - and what the impact of the ruling might be.??

In the meantime, Adobe released a Beta of Firefly saying that it was designed to create images safe for commercial use . They say that it won’t infringe on the IP of other creators because it was trained on Adobe Stock images, openly licensed content, and public domain content.

I don’t think the ‘ethical training’ situation is as clear-cut as Adobe are making out, but I’m a lifetime Adobe user, so using an image generator that’s built into the tools I use everyday seems like a no-brainer.?

Is it time to switch from Midjourney? I got access to the Firefly Beta to find out… and was underwhelmed. But maybe I was biased toward Midjourney’s output since I was a regular user?

So I put a survey together and pitted Firefly and Midjourney against each other for the same prompts. The survey was a ‘blind taste test’ and had no way for respondents to tell which tool created which image, and the questions and options were randomised.?

71 people voted in the survey. Let’s take a look at the results…

Two robots

Prompt: Two robots sitting facing one another, a fashion magazine style photograph with moody lighting?

Result: 79% in favour of Midjourney.?

Firefly made a valiant effort here, and I get why some people would choose the Firefly image, it’s an impressive image and has a lightness to it that’s appealing. But Midjourney did a better job here of looking like something from a fashion magazine.?

A factory in a coffee cup

Prompt: Cutaway illustration of a coffee cup, with a busy factory inside the cup, Hand drawn, Line drawing, Isometric, Outlining everything, Thin lines, one line weight, Black lines, Colourful, Vibrant, Flat colours

Result: 79% in favour of Midjourney.

I think Midjourney absolutely nailed this one. The only place I would dock Midjourney points on this is “vibrant”. It seems to have chosen a duller palette than I would have liked. But otherwise the detail is excellent compared to Firefly.?

Man holding up hands

Prompt: A man holding up his hands

Result: 67% in favour of Midjourney.

Hands. AI image generator’s kryptonite! Quite surprisingly, both tools did well with the hands. But while Firefly’s effort is impressive, it looks like a stock photo. Whereas Midjourney’s looks like it’s from a photo exhibition. Also worth noting here that the prompt did not specify ethnicity and both AIs have generated a person of colour, not so long ago Midjourney almost always defaulted to a white person. Good to see progress there.?

Logo with milk and wheat

Prompt: A logo of a bottle of milk and wheat, circular, sticker

Result: 79% in favour of Midjourney.

There’s an argument to be made here that Firefly did a better job of creating a logo, it’s simple, pleasing, and would scale to small sizes reasonably well. But if you’re choosing the more beautiful image there’s no doubt Midjourney wins hands down.?

Happy beagle

Prompt: A happy beagle with love hearts

Result: 81% in favour of Midjourney.

Midjourney managed to capture something really joyful in the beagle’s face, which gave it an edge. Firefly seemed to be off to a flying start, and then for some reason decided to use a really shallow depth of field in this image and the beagle’s nose is out of focus. I mean… why??

The letter A

Prompt: Black typography of the letter "A", sans serif, modern, Expressive and simple line drawing, stark black and white outline, ink on plain white background, centered, simple, elegant, drawn by hand, hand-drawn, with imperfections

Result: 50.7% in favour of Firefly.

Midjourney generated a beautiful image. But I asked for ‘sans-serif’ and Midjourney gave me a serifed font. And are all those pretty ink splatters really simple or elegant?? Firefly’s output is a better match to the prompt, if we’re going by the letter of the law. So while I might prefer Midjourney’s image, Firefly’s is closer to what I asked for, so although this was almost a tie, it’s only fair that Firefly won.

Geometric superheroes

Prompt: Simple geometric illustration of 5 comic book superheroes, male and female, female superhero in foreground, group of superheroes in background, full length, full body, primary colours, isolated image on a solid colour background, centered, geometric with detail

Result: 69% in favour of Midjourney

This was an actual prompt I used when working on ideas for the cover of ‘The Most Amazing Marketing Book Ever’. I think this is a fair win by Midjourney - it’s come up with a beautiful interpretation of this prompt. I might have been more impressed with Firefly’s image if it hadn’t absolutely mangled the faces of these superheroes.?

A circle, triangle, and diamond

Prompt: A circle, a triangle, and a diamond, watercolour, Expressive and simple line drawing, stark outline, ink on plain white background, centered, simple, elegant, drawn by hand, hand-drawn, with imperfections

Result: 77% in favour of Midjourney.

One of my favourite pastimes is getting Midjourney to create artworks for me. Just for fun. These are the kinds of prompts I use. So I was really interested to see what Firefly would do with a similar prompt. Eh… I’m not going to be hanging Firefly’s effort on my wall anytime soon.?

Digital city

Prompt: a digital city, lots of neon, futuristic, science fiction, cinematic

Result: 59% in favour of Midjourney.

Pretty solid results from both AIs here, I like the brightness of Firefly’s output, and it’s not just in this image, Midjourney does seem to have a bias toward darker, moody colours if you don’t specify otherwise. I reckon Midjourney wins by a nose here because of the detail though, Firefly’s image is a bit more broad strokes.?

Elephant riding a bike

Prompt: oil painting of an elephant riding a bicycle holding flowers in his trunk

Result: 71% in favour of Midjourney.

This is a prompt I like to test AI image generators with. Midjourney does a stellar job of creating an artistic style I love - but that elephant is not holding those flowers in its trunk. Firefly’s artistic style on this one is a little less sophisticated, but you could argue that elephant is holding a basket of flowers with its trunk.

Bonus: out of ethical reach

I didn't include this in the survey, I wanted the survey to be a level playing field so I didn’t include any prompts that Firefly wouldn’t be able to interpret. But here’s the thing, if you only train an AI on images you have licence to use, obviously it won’t know who Spider-Man is.?

Which means Firefly can’t do much with a prompt like ‘Spider-Man in the style of Degas’

So Firefly is not infringing on Marvel’s IP with this image, and doesn’t seem familiar with Degas either, even though his work is in the public domain.?

Meanwhile Midjourney is having a ball infringing on IP and copying the style of a french impressionist.?

Clearly I could never use an image of Spider-Man like this commercially, but it’s a lot of fun to create and admire. So where should the ethical responsibility lie? With Midjourney in the creation of images, or with me in the usage?

The Final Results

Midjourney won 9 out of 10 of the battles. When you put all the prompt battles together, Midjourney won the overall competition with 71% of the total votes.?

Did Midjourney infringe on the copyright of artists? Is its training unethical? We will have to see what the courts say. In the meantime, in terms of quality of images, Midjourney is clearly streets ahead of the ‘ethically trained’ Firefly.?

Do you believe Midjourney’s approach is unethical? And if so, are you willing to sacrifice image quality for your ethics?

-

This article was originally published on the Frank and Marci blog.