How well GPT-4 performs in simple but nuanced tasks?

How does GPT4 perform in highly nuanced language tasks that's fairly simple for a human adult?

My first test : Solving crossword puzzles

I am certainly not missing the fun that possibly 100 mn others are having these days – experimenting with chatGPT and GPT-4.

Working mostly with Computer vision in my health-tech startup, I am very keenly watching the advancements in NLP and possibly a convergence between the two that can create a tectonic shift in our industry. I believe with GPT-4, with its multimodal data ingestion, we are one step closer and it is infinitely exciting.

Meanwhile, I wanted to leverage the $20 I started to pay for chatGPT every month, by employing it as my companion in my daily morning routine: Solving crossword puzzles (to my wife’s chagrin)?

Conclusion?: It mostly failed spectacularly. BUT the little it succeeded, it succeeded spectacularly too. Although disappointed, I was amazed at the way it could explain its ‘thought’ process while solving the puzzles so cutely even when it failed. In that way, the experiment was a grand success. I am sure we can soon succeed with a few-shot training by curating a right list of training cases to guide GPT-4 to learn crossword solving better.

Now here are some examples from my experiment:

Me : Solve crossword clue : Medical attendant runs confused with a direction (5)

GPT-4 :?The correct answer to the crossword clue is NURSE.

Explanation (by GPT-4):

• "Medical attendant" refers to a nurse.
• "Runs confused" is an anagram indicator, telling us to rearrange the letters of "runs".
• "With a direction" is the definition part of the clue, indicating that the answer is a word that means a direction.
• Rearranging the letters of "runs" gives us the word "URNS".
• "URNS" can also mean "direction", especially in phrases like "to earn one's keep" or "to earn one's living".
• "Nurse" is a synonym for "medical attendant", and it fits the letters we have from the anagram and the definition.

My comment: Well, the answer was correct but It went a bit overboard to explain. I just added ‘E’ as a shorthand for ‘East’ while forming the clue. But after solving the anagram runs into NURS, to add the E, it went tangential. But it was so cute to see how it explains. You feel like giving it a chocolate.

Me?: Solve the crossword clue : Better than the other barbers? (1,3,5)?

GPT-4?: The answer to this crossword clue is "A CUT ABOVE".

Explanation:

• "Better than the other barbers" suggests that the answer is a phrase that means superior or better.
• "A cut above" is a commonly used phrase that means superior or better than average.
• "A cut" refers to a haircut, which is a service typically provided by barbers.
• "Above" means at a higher level or standard.
• Therefore, "A cut above" means superior or better than other barbers, and it fits the given clue of "Better than the other barbers?" with 1, 3, and 5 letters.

Spot on. Well done GPT-4!!! Here is your cookie!!

Me :?Solve crossword clue :Having the customers come in, spoke freely (6,2)?

GPT-4 :?Solution to crossword clue:

Having the customers come in, spoke freely (6,2) = Opened (6) up (2)

Explanation: "Having the customers come in" implies that a business or establishment has opened its doors, hence "opened." "Spoke freely" suggests a sense of openness and candidness, which can be represented by the word "up." The two-word phrase "Opened up" combines these ideas and fits the required pattern (6,2).

My comment : Wow wow.. I know it’s a simple clue but its absolutely adorable.. I can see the twinkle in the eyes of my pet now…

Now lets look at a failure. There were too many but I don’t want to bore the reader so selected one:

Me:?Solve crossword puzzle :When everything changed and you became a convert (7,5)

GPT-4 :?The given phrase is likely a cryptic crossword clue. In this case, the numbers (7,5) indicate the number of letters in each word of the answer, which is a two-word phrase with 7 letters in the first word and 5 letters in the second word.

Clue: When everything changed and you became a convert (7,5)

Answer: Instant (7) adept (5)

Explanation: "When everything changed" can be interpreted as an "instant," a moment when a significant shift or transformation occurs. "You became a convert" suggests that the person has become skilled or proficient in a new belief system or area, making them an "adept."

My comment: The real answer is ‘turning Point’ but the answer giving by GPT is very plausible. May be in real situations, if given a full grid and a few letters in the answers, it can do better.??I tried giving T and P as the first letters of the word but it gave random answers like ‘ Turned Pagan’.

It was a short experiment of around 100 puzzles and didn’t end well. But I feel that the LLMs have come far ahead in terms of solving nuanced tasks in an explainable manner. I am not giving up any time soon.?