Show Me The Steps, ChatGPT

A few weeks ago, I did a session on ChatGPT at MinneAnalytics #DataTech conference. In that talk, I showed a number of different ways in which engineering your prompts will result in some wildly successful or not-so-successful outcomes. Unfortunately, the talk was only 45 minutes and I ended up having to cut out one very interesting finding, which I will share here and that is the concept of “stepping through it”.

The Logic of Large Language Models

One of the things that people get hung up on when working with Large Language Models is just how authoritative they can seem. It seems like anything you throw at it, it has an answer, and that answer can seem extremely plausible if you don’t do your homework. I’ve told business leaders and companies that I consult with a number of times that ChatGPT falls down both in areas of research, where you need to cite sources. However, there’s also another area and that is math.

It’s interesting to note that ChatGPT has not been trained on HOW to add two numbers, instead is seen enough data to know that when someone types “what is 2 + 2”, it will respond with 4. Additionally, it knows that when someone asks, “If Jane has 2 apples and Joe has 4 oranges, how much fruit do they have together?”, it will correctly respond with “6 pieces of fruit”.

However, what if you ask it the following question?

If John has 5 pears, then eats 2, buys 5 more, then gives 3 to his friend, how many pears does he have?

Take a look at the screenshot below to see what ChatGPT gives. Note, that I’m using what is called the OpenAI playground, but it is using the same model that ChatGPT would produce.

If you do the math yourself, you will notice that the answer of 7 pears is incorrect!

5 - 2 + 5 - 3 is not equal to 7.

So, what’s going on here? While I don’t claim to know the details, it’s obvious at some level that the language model is not understanding what is being asked. It’s been trained on the concept of adding pears (with buying), and subtraction (with eating and giving), but the semantics of how it got the answer are not shown or being followed.

Enter the statement that can help, “Let’s Step Through It”.

Stepping Thorough It

If you are doing anything requiring logic or would like to have it explained, I would encourage you to use this. It’s not the end-all, be-all solution, but I have found it to dramatically increase likely hood of success. In the screenshot below, you’ll see that we add this statement of "Let's step through this" to the question. See the results.

Voila! The answer of 5 is correct.

By asking ChatGPT to step through the scenario, you in a way force it into having to explain the steps. By doing this, the model needs to explain the logic and comes to the correct conclusion.

Summary and Key Findings

In closing, there are a few things I think we can learn here. Some are obvious while others are subtle.

1. Don’t take everything that ChatGPT says as truth. Hallucinations are definitely real, not only when asking for factual information, but when performing logic calculations.
2. When it comes to math and word problems, check your work. I’ve found it to be good when asking arithmetic, but there’s a lot of nuance in word problems and this is where ChatGPT can get confused.
3. When you have a series of steps you are asking ChatGPT to perform, don’t be afraid to ask it to step through the logic. I’ve seen cases where asking it to step through still gives the wrong answer, but just like when you were in school and needed to work through a complex problem, asking a large lange model like ChatGPT to do the same thing had the potential to yield much better results.
4. It’s not specifically covered here, but the more verbose you can be with the language the better. For example, just like using parenthesis in arithmetic: (4 x 3) - 2 = 9 is different from 4 x (3 - 2) = 4, it helps to spell out in greater detail to the language model what you are exactly calculating and asking for with a more verbose prompt.

I hope you have found this blog post useful. Over time I expect ChatGPT to get better and better at figuring this out, but it never hurts to be more explicit. Artificial Intelligence is here to help makes our lives better and the more you can clarify the task, the better the output will be.

Good luck and happy prompting!