How Asking ChatGPT to Count Nearly Broke Me

On a Friday afternoon, I had a simple task.

I had a list of names that included duplicate names and I needed to know how many names were on that list total (this will become important). Aha, I thought, I'll pop this quickly into ChatGPT.

Oh my sweet, summer child.

Here's my initial prompt (ChatGPT):

How many names are in this list? Count each line as one even if there are duplicates.

This gave me an incorrect number (29) that counted only the unique names. I told ChatGPT the correct answer (36) and asked how I could improve my prompt for next time:

I want to improve the prompt I asked you initially. Can you help me understand the process that lead you to 29?

It suggested I try this prompt:

Count the total number of names in this list, considering each unique line as one name even if there are duplicates.

Okay, maybe this is a semantic issue. My initial prompt asked the question before adding the qualifying instruction. We'll try the new suggestion.

Nope. Now it answered there were 26 names. When I asked what went wrong this time, it still flagged the duplicates as being the area that tripped it up. I asked for another revised prompt.

Count the total number of names in this list, considering each unique line as one name, including duplicates.

It now answered 34. Next variation, 38. Then 40. None of which are the correct number (36) which I had long-since counted with my human brain.

I decided to backtrack and remove the duplicates. Surely that would be a simple prompt I could use as a base. I provided a list of 15 names and confidently asked:

How many names are on this list?

14. What in the Sesame Street is going on here?

I continued to refine and test the prompts, as well as ask ChatGPT for direction (at one point, it even suggested I format my list as CSS). I did 10 iterations of this and then I thought, before I just give up, why don't I try Bing. I put in my original prompt. Bing's answer?

The list you provided contains 36 names. This count includes duplicates, as each line is considered separately.

If anyone needs me, I'll be taking a few minutes of quiet time. And next time, I'll just use Bing.