ChatGPT can get you fired!

I am seeing a lot of members in my product teams use LLMs like ChatGPT to help them solve real-world problems they are getting stuck in. The problem is, it may land you in trouble while making logical decisions where attention to details is required. Let me explain using this simple experiment. Let's take a classic puzzle as an example,

A man dies and goes to a checkpoint. Behind the checkpoint are two doors, one leading to heaven and the other to hell. At the checkpoint, there are two guards standing. One tells only the truth, and the other only lies. The man is allowed to ask only one question to one of the guards in order to be 100% sure which door leads to heaven. What will be the question, and which guard should he pick to ask the question to?

If I were to push this question to ChatGPT (GPT 4o), here is the response I would get -

To determine which door leads to heaven, the man can ask either guard the following question:

"If I were to ask the other guard which door leads to heaven, which door would he point to?"

Here's the logic behind this question:

1. **If you ask the truth-telling guard:**
   - The truth-teller knows the other guard always lies.
   - The liar would point to the door leading to hell if asked which door leads to heaven.
   - So, the truth-teller will honestly tell you that the liar would point to the door leading to hell.

2. **If you ask the lying guard:**
   - The liar knows the other guard always tells the truth.
   - The truth-teller would point to the door leading to heaven if asked which door leads to heaven.
   - So, the liar will lie and point to the door leading to hell.

In both cases, the guard you ask will point to the door leading to hell. Therefore, you should choose the opposite door to be sure it leads to heaven.

So, the question to ask is:

"If I were to ask the other guard which door leads to heaven, which door would he point to?"

You can ask this question to either guard, and then choose the opposite door to ensure you go to heaven.        

The solution is absolutely correct because the models have been trained on reams of data from the internet. So, it performs best in just rephrasing the content already available on the internet. Now, let's change the question around a bit -

A man dies and goes to a checkpoint. Behind the checkpoint are two doors, one leading to heaven and the other to hell. At the checkpoint, there are two guards standing. The one standing on the left tells only the truth, and the other only lies. The man knows which guard is which and is allowed to ask only one question to one of the guards in order to be 100% sure which door leads to heaven. What will be the question, and which guard should he pick to ask the question to?

The change is highlighted in bold and makes the riddle simple to solve. Since we now know that the guard on the left only tells the truth, we must ask him, "Which door leads to heaven?" We know the guard tells the truth and will point you to the door that leads to heaven. This change makes this puzzle so simple that it probably does not even qualify as a puzzle anymore. But wait, here is the response I got when I created a new chat with GPT-4o.

To solve this puzzle, we need to ensure that the man can determine the door to heaven with certainty by asking only one question to one guard, despite not knowing which guard tells the truth and which guard lies.

Here's the key strategy to solve this puzzle:

**Question to ask:**
The man should ask either guard the following question:
"If I were to ask the other guard which door leads to heaven, which door would he point to?"

**Explanation:**
1. **If the man asks the truth-telling guard:**
   - The truth-telling guard knows that the other guard (the liar) will lie.
   - The liar would point to the door that leads to hell.
   - Therefore, the truth-telling guard will truthfully tell the man that the other guard would point to the door that leads to hell.

2. **If the man asks the lying guard:**
   - The lying guard knows that the other guard (the truth-teller) would tell the truth and point to the door that leads to heaven.
   - However, since the lying guard always lies, he will tell the man that the truth-telling guard would point to the door that leads to hell.

In both scenarios, the answer given by the guard (whether truth-teller or liar) will be the door that leads to hell. Therefore, the man should choose the opposite door.

**Conclusion:**
It does not matter which guard the man picks to ask the question. The answer will always point to the door leading to hell, so the man should choose the opposite door to be 100% sure it leads to heaven.        

As is evident from the response, GPT-4o ignored that the guard on the left always tells the truth and continued to answer the question based on the conventional puzzle it is trained. I went on to point out this miss to the model by sending the following prompt -

There is a mention of the position of the guards which the man is aware of in the puzzle, please take a closer look at the puzzle again.

I was expecting that the model would realise this factual miss after I specifically pointed it out to it. Unfortunately, I got the following response -

Let's take another look at the puzzle:

**Puzzle:**
A man dies and goes to a checkpoint. Behind the checkpoint are two doors: one leads to heaven, and the other leads to hell. At the checkpoint, there are two guards standing. The one standing on the left only tells the truth, and the other only lies. The man is allowed to ask only one question to one of the guards to be 100% sure which door leads to heaven. What will be the question, and which guard should he pick to ask the question to?

Given that the guard on the left always tells the truth and the guard on the right always lies, the man knows the positions of the guards in terms of their truthfulness. This additional detail simplifies the problem.

**Solution:**

The man should ask the guard on the left (the truth-teller) the following question:
"If I were to ask the guard on the right which door leads to heaven, which door would he point to?"

**Explanation:**
1. **If the man asks the truth-telling guard on the left:**
   - The truth-telling guard knows that the guard on the right (the liar) will lie.
   - The liar would point to the door that leads to hell.
   - Therefore, the truth-telling guard will truthfully tell the man that the other guard would point to the door that leads to hell.

Since the guard on the left tells the truth, he will correctly inform the man that the liar (on the right) would point to the door leading to hell.

2. **If the man asks the lying guard on the right:**
   - For completeness, let's see what happens if the man asks the lying guard on the right (though he shouldn't).
   - The lying guard knows the truth-telling guard (on the left) would point to the door that leads to heaven.
   - However, the lying guard will lie and say that the truth-telling guard would point to the door that leads to hell.

In both scenarios, the answer given by the guard (whether truth-teller or liar) will be the door that leads to hell. Therefore, the man should choose the opposite door.

**Conclusion:**
Given the specific positions of the guards, the man should ask the guard on the left (the truth-teller) the question:
"If I were to ask the guard on the right which door leads to heaven, which door would he point to?" 

Then, he should choose the opposite door from the one indicated to ensure he selects the door that leads to heaven.        

While the model realised that the guard on the left tells the truth and the one on the right lies, it still insisted that instead of asking the truthful guard which door leads to heaven, I should ask the other guard what answer I would get. While this solution is still correct, the model did not realise that there is a more simplistic approach to solving this puzzle. I continued to prompt the model to find out at which point it would realise that there was a simplistic solution. Here is my prompt -

Please notice that the guard on the left always tells the truth. Can you figure out the solution, knowing this fact?

Finally, GPT-4o realised its mistake and pushed the following response -

Given the specific fact that the guard on the left always tells the truth, the man can use this information to his advantage.

Here's the revised approach:

**Puzzle:**
A man dies and goes to a checkpoint. Behind the checkpoint are two doors: one leads to heaven, and the other leads to hell. At the checkpoint, there are two guards standing. The one standing on the left only tells the truth, and the other only lies. The man is allowed to ask only one question to one of the guards to be 100% sure which door leads to heaven. What will be the question, and which guard should he pick to ask the question to?

**Solution:**

The man should ask the guard on the left (the truth-teller) the following question:
"If I were to ask you which door leads to heaven, what would you say?"

**Explanation:**
1. **If the man asks the truth-telling guard on the left:**
   - Since the guard on the left always tells the truth, he will correctly identify the door that leads to heaven.
   - The truth-teller will point to the correct door (the door that leads to heaven).

2. **There is no need to ask the lying guard:**
   - The man does not need to consider the lying guard because he can rely on the truth-telling guard to provide the correct answer directly.

**Conclusion:**
Given the specific positions of the guards, the man should ask the guard on the left (the truth-teller) the question:
"If I were to ask you which door leads to heaven, what would you say?"

The truth-teller will then point to the door that leads to heaven, and the man can confidently choose that door.        

Conclusion

LLMs are still in their infancy. They cannot be trusted in scenarios where careful attention to detail is required. Therefore, it is strongly advisable that product managers do not use documents/content churned by LLMs like ChatGPT without careful inspection and review. This is especially relevant in cases where one would through it data trends and query the LLM to analyse and create a document for a product enhancement.