Can it (genAI) solve wordle?

For a very long time, the running theme for high end consumer pc's and cutting edge graphics cards was "Can it play Crysis?". Crysis as a game had garnered a reputation to melt the most cutting edge PC's and bring them down on their knees.

It's a nerdy joke. Jensen Huang mentioned it during his Computex 2023 keynote pondering if the state of the art 256 Grace Hopper super chip data center configuration could play Crysis.

I decided to take all the latest generative AI models and throw the toughest problem that all humankind face first thing in the morning.

"Wordle"

And hence the big question. Can it (genAI) solve Wordle?

Spoiler alert - It can't. Neither GPT-4o, nor Gemini, nor Claude

So here's me put Gemini, Claude and GPT-4o throught their paces to see if they can solve Wordle.

This is what my wordle screen looked three tries in.

I devised the following prompt.

I am looking for a 5 letter word. The word cannot contain the alphabets "C","R","O","W","N","S","I","M","E","B","U","K","Y". The word contains the alphabet "L". "L" cannot be in the second or third place in the word. What are the possible solutions for this word?        

Claude

Claude melted faster than butter on a hot skillet.

It generated a possible list of words that while structurally valid were not real English words. It also did not understand the positional constraint of "L".

It failed miserably.

Gemini

Gemini tried its reasoning by fixing the positional constraint of the alphabet "L" but totally missed out on the constraint on which alphabets cannot be part of the word.

So it did half the work and did not validate its end results with all the constraints.

GPT-4o

Finally, we get to the coolest kid on the block. GPT-4o.

To give credit where it's due, GPT-4o started like the proverbial first bencher in the classroom. It replayed the constraints that I gave and started to solve it logically. It got my hopes up and dashed them equally fast when it came with its final solution as "HALF".

Seriously ! This is not even a five letter word. I was left speechless.

In Conclusion

I won't reveal today's Wordle answer to avoid spoilers, but it's worth mentioning that none of the generative AI models got it right

The key to solving any problem is to devise a solution framework, apply, get an answer and validate your results through a parallel process.

Both Gemini and GPT-4o seem to break down the problem into logical solution steps but falter at the application level. I would say that I found GPT-4o's abilities better than Gemini's for this specific problem.

Claude follows a brute force approach and falters at the first step itself.

In conclusion, the answer to the question "Can it (genAI) solve wordle?" is a resounding No.

The objective of this exercise beyond general amusement was to test the ability of foundational generative AI models to breakdown a problem in to logical steps, execute them while maintaining constraint integrity, decomposing the problem, arriving at or deducing the answer, and finally validating the answer.

I believe that the investments in generative AI will yield in substantial increase in capabilities every year. While the hype on genAI and its potential still remains at an all time high, RAG based solutions, thorough testing and validation for your use case is critical to ensure consistent results.

The opinions in this article are the author's own and do not reflect the views of the organization he is employed with.