I tested the 3 most popular AI tools for UX?so you don't have to

The state of AI in UX

I recently spoke to a lawyer acquaintance of mine who told me he had trained Claude to do something like 80% of his weekly work.

It's no secret that AI is impacting every industry at an unprecedented pace, automating a great deal of low-level knowledge work, especially the kind that follows well-known patterns and conventions.

UX is nothing if not convention-driven, so I wanted to see how far the new crop of AI tools had come in automating my own work.

I tested three apps that offer the same basic functionality: you enter a prompt and they will generate some screens.

1. Galileo
2. UIzard
3. UX Pilot

Like many of today's early AI experiments, the results were in some ways astonishing and in other ways lackluster.

The test prompts

I wanted to try these apps from my context as a professional designer. To make the test realistic, I gave each of apps a prompt based on a real client project:

Users:
Sales reps and receptionists at busy medical practices, usually for elective medicine like dermatology

Problem:
Prospective patients need to be guided to an initial consultation with a doctor so they can determine what treatment is right for them.

Actions:
? Communicate with patients via SMS or email
? Capture information about new patients
? Follow up with patients at regular intervals
? Schedule initial consultations based on doctors’ schedules
? Send marketing emails and text messages to patients
? Manage reviews on social media channels
? View analytics about conversion and sales rep performance

Integrations:
? Practice management system
? VOIP providers such as Twilio or RingCentral
? Social media channels like Facebook, Yelp and Google Places

Similar products:
Hubspot, Pipedrive, Nexhealth, NextPatient
        

Later, I decided to try a second prompt that might be more typical of a non-professional exploring an app idea:

An app that helps me plan trips with my friends — deciding on a destination and dates, group booking, and planning our itinerary together.        

Let’s take a look at how each tool performed.

Galileo

Galileo uses a chat style UI similar to ChatGPT. I entered my prompt and let it rip.

Before generating anything, Galileo first demonstrated that it understood my request by describing a set of screens to address most of my requirements.

This response accurately restated my requests in a parrot-like fashion, but didn't envision an overall information architecture for the app. Galileo seems to be optimized to design individual screens, not to think about user experience as a whole.

After confirming the proposal and a bit more prompting, I got it to produce six screens to illustrate some key functional areas of the app.

The screen design itself was pretty good, using a nice and clean style typical of Apple or Google UIs.

I copied one of the screens and brought it into Figma. Although the visual appearance translated perfectly, the output was heavily nested in unlabeled auto-layout frames, making it unusable for further design work.

It seems that Galileo's AI generates its designs in HTML, and then traverses the DOM to create corresponding Figma layers as some HTML-to-Figma plugins do.

The HTML output itself looked great, so it might be possible to string together these screens into a respectable clickthrough mockup. But that assumes that these screens are good enough in their present state, which they are not – all the AI output needs further refinement.

Finally, I tried my simple mobile app prompt. Again Galileo produced results according to my specs, this time adding a few of its own ideas.

The screens it created were nice looking but low on functionality. I was pleasantly surprised to see voting features on destinations and an understanding that users would need to input broad ranges of availability.

I think these more simple use cases are where Galileo and the other AI apps really shine. When the requirements are less complex, the instant design, complete with images, really feels like magic.

UIzard

UIzard uses a chat UI like Galileo to capture initial input, but proceeds directly from the prompt to screen generation without pausing to confirm that it actually understands the request.

This was evident in the generated screens, which showed a lack of understanding that the app was basically a CRM. The "dashboard" was essentially just a set of buttons repeating back my instructions.

The Patient Profile looked more like a fitness tracker than a contact record.

I did appreciate the fact that UIzard included a screen navigator to make it easier to see the overall app flow.

UIzard's biggest weakness is its lack of Figma export. They have decided to build their own quasi-editor – a serious error in my opinion. They're not going to out-Figma Figma, and they are diverting resources from improving the AI generation capabilities.

The only way to export is as a flat file or with code. However, the code inspector only exists at the component level – another error. This assumes the design is ready for production and that developers are here to pull specs off individual components. Obviously none of these AI tools is close to delivering that level of polish just yet, so this feature is superfluous.

When I tried the travel app prompt, UIzard produced a lot more screens than I expected.

However, I again felt UIzard didn't really understand the assignment. The screens were typical of travel apps, but the group coordination aspect seems to be lost except for the presence of a messaging screen.

UX Pilot

Like Galileo, UX Pilot took my prompt and proposed a set of screens before designing anything.

The responses were similar to Galileo's, addressing my specific requests one by one rather than thinking about the overall structure of the app.

I found UX Pilot's own UX to be rather clunky overall. Although Galileo's chat-style interface is a bit of an AI cliche, I never got stuck while using it. With UX Pilot, there were several moments where things were not where I expected, or interactions didn't work as intended. I've been getting many updates from them so I know the product is in active development – hopefully some of these issues will be resolved soon.

I proceeded to generate screens in wireframe mode, since I wanted the focus to be on overall UX rather than the details of visual design.

I felt that UX Pilot did a pretty good job of interpreting the functions of each screen into a well-organized UI. For example, it understood that the patient communications UI was actually a kind of to-do list, with the ability to perform certain actions for each incoming message.

Moving the designs into Figma proved a bit clumsy, as I had to first select the screens I wanted and add them to Favorites, then import them via a weirdly-designed plugin.

In the end, I was able to successfully import the designs all at once, rather than having to use Galileo's manual cut-and-paste approach.

Like Galileo, the output was heavily nested and unusable. The HTML tags are clearly visible here, validating my theory about how these screens are generated.

With the travel app prompt, UX Pilot did an OK job of creating relevant screens but, like UIzard, I felt that it missed the spirit of the assignment. Also, the images it generated were quite odd, even though I selected the hi-fi design option.

Discussion

At a high level, my impression of these apps were that that they are cool toys but unsuitable for professional use. Based on my evaluation criteria of understanding an assignment and generating useful outputs, Galileo is the best, UX Pilot is somewhere in the middle, and UIzard is behind.

As a whole, the apps are optimized for screen-level UI design but they lack much capacity for information architecture, a core aspect of overall UX. I found this pretty amusing because I see the exact same problem with many human designers these days!

Critically, none of the outputs of these tools are usable for further refinement. Specifically, the Figma file generation is too nested and entirely without styles or components. Although it's impressive to get me 80% of the way there in minutes, I still need to finalize the remaining 20% somehow.

Right now, the only real use case I can envision for these tools is for first-time founders to visualize their ideas quickly. However, I am keenly aware of how fast things are evolving, so I may be eating my words within a few months.

I was very impressed by how well the apps were able to do UI design, and I can envision a future where screen-level design is almost fully automated.

Whether you're a human or an AI, it's not very hard to design individual UIs because there are usually well-established patterns for most kinds of screens and design patterns, as evidenced by the many thousands of design templates available for virtually any purpose.

The hard part is the abstract thought involved in contemplating how a user moves around an app as they seek to achieve their goals. This requires a holistic point of view and usually some additional context.