Our jobs are safe from Chat GPT

In the past few weeks, I spent some time playing with #Chat and I have been getting some reasonable results. Most of the conversations I had with the AI produced results that I could understand and they seemed to be correct (at least for the domains I have an understanding of).

Today I decided to use ChatGPT as a math helper. I was helping my son prepare for a math test and I thought: "Why not use ChatGPT to validate the approach we take?". Maybe ChatGPT is better at explaining high school math to a high school student than I am. In all fairness, it's 30+ years since I've been in high school.

We first solved the problems ourselves with pen and paper (or Google Docs to be exact) and then we asked ChatGPT to solve it for us. I will not be showing our solution though (since this is not a post to teach high school math), just the interaction with ChatGPT.

Problem 1

Asking ChatGPT

The answer

The approach seems solid and the way ChatGPT explains the problem is quite understandable, both by the high-schooler and myself:

• Defined 2 variables
• Translated text into math expressions
• Created a system of equations and proceeded to show how to solve them

But, the problem was the answer was wrong. If you can't spot it immediately, here it is:

8 * 32 + 64 = 276 (it should have been 324)

Nudging

And... Victory!

Problem 2

Asking ChatGPT

The answer

Again, good approach, but what is this:

x = (1/2)(62) + 34° = 15 + 34° = 49°

I think we can all agree that 1/2 of 62 is 31, not 15. Let's ask ChatGPT to rethink

Nudging, once

On the plus side, ChatGPT recognizes that the answer provided was wrong since it can't validate the initial problem statements. On the other hand, what is this now?

x = (1/2)(62) + 34° = 60°

Does 1/2 of 62 have some magical property that makes ChatGPT give the wrong answers? Or is ChatGPT supposed to be bad at basic arithmetic?

Nudging, twice

At least it was able to validate our solution!

Grade

Overall, a 9/10 effort. ChatGPT showed its work, the approach was correct and it only failed to do a calculation correctly. If I was its teacher I would probably give it 80-90% of the marks for the effort.

However, I am perplexed about why the failures happen. It is obvious that ChatGPT knows how to approach the problem and it constructs a very reasonable and well-explained path to get to the solution. How is it then possible to fail in simple arithmetic calculations?

I am definitely NOT an AI expert, but if anyone happens to come across this article and can illuminate me I would appreciate it! Till then I feel that I don't have to worry that much about AI!