ChatGPT and Genuine Thinking

After all the alarmist posts about ChatGPT and AI getting ready to take over the world, I thought it wise to learn more about the enemy. After passing it some decoy prompts like "Write a conversation between Spiderman and Indiana Jones as Shakespeare would" and "How would chicken nugget ice-cream taste?" I got straight to the point -

Rohan: Do you plan to take over the world one day?

ChatGPT: No, as an artificial intelligence language model, I do not have personal desires or intentions. I exist to provide information and assist with tasks to the best of my abilities within the scope of my programming. Additionally, I do not have any physical capabilities to take over the world even if I had such intentions, which I don't.

While I wasn't really convinced about its intentions or "limitations", I was impressed with the quality of its conversation (its conversational skills far surpassing some humans I know), so I took a detour and decided to read up more about the engineering behind it and it led me to this helpful article from Stephen Wolfram.

The long and short of it is that ChatGPT functions by figuring out at each step which is the best next token (think word) to add to the existing content. It gets and ranks the tokens at each step based on the all the data it has been trained on (basically the web). For example (taken from Wolfram's article), say the sentence you have till now is "The best thing about AI is it's ability to". An LLM will come up with feasible next words and rank them. So, the next few tokens to consider might be (in word-probability tuples) -

1. Learn - 0.05
2. Predict - 0.032
3. Make - 0.024
4. e.t.c.

The simplest approach is to always pick the highest rank word but this leads to a lot of repetition and hence, dull sounding essays. For example, for GPT-2's model this would produce -

"The best thing about Al is its ability to learn from experience. It's not just a matter of learning from experience, it's learning from the world around you. The Al is a very good example of this. It's a very good example of how to use Al to improve your life. It's a very good example of how to use Al to improve your life. The Al is a very good example of how to use Al to improve your life. It's a very good example of how to use Al to"

Clearly this would be a D- on any grading scale. So now we have to tweak the process a little. A good English teacher will always tell you to not repeat words, use synonyms e.t.c. when writing an essay. To incorporate this into an LLM, a notion of randomness was introduced. With some value of the parameter "temperature" we will pick a lower ranked word. This leads to much better results.

"The best thing about Al is its ability to see through, and make sense of, the world around us, rather than panicking and ignoring. This is known as Al "doing its job" or Al "run-of-the-mill." Indeed, taking an infinite number of steps, developing a machine that can be integrated with other systems, or controlling one system that's truly a machine, is one of the most fundamental processes of Al. Aside from the human- machine interaction, Al was also a big part of creativity"

It is definitely less repetitive but still a little weird. This was produced by the GPT-2 model. You can try the same thing with GPT-4 and get a much better response.

That got me to thinking - is ChatGPT truly capable of genuine thinking i.e., capable of coming up with original solutions to complex problems it hadn't seen before? So, I asked it a question that I used to use when interviewing TA candidates at Penn -

Rohan: I have 2 pieces of rope burn unevenly with respect to length but take exactly 1 hour to burn. How can I use them to measure 15 minutes?

The answer to this question is something that would be easy for a human mind to come up with given enough time - Take the first rope and light both ends at the same time (that way we know it will take 30 minutes to burn). At the same time, light the other rope at one end only. When the first rope is completely burnt out, light the second rope at its other end. The time between end of the first rope finishing burning and the second rope is 15 minutes.

When I asked this question to ChatGPT, I could tell that this question and its answer has been posted somewhere on the web since ChatGPT had the main elements of the answer in place. The answer started well enough -

ChatGPT: "One way to measure 15 minutes using the two pieces of rope is as follows:

1. Light both ropes at the same time from one end."

Okay strong start. Nuclear bunker seems required at this point.

ChatGPT: "2. Let one rope burn completely."

Okay... Deviating from what I expected but let's read on - maybe it came up with a different approach though at this point the answer is already wrong - lighting both ropes at one end and waiting for one to burn also means the other one burns completely.

ChatGPT: "3. At the moment the first rope finishes burning, light one end of the second rope."

Umm, both ropes take 1 hour to burn so if one is done burning then there wouldn't be an end on the second rope to burn? My bunker is looking unnecessary right now.

ChatGPT: "4. When the second rope has burned halfway (30 minutes), light the other end of the remaining rope."

Dude, haven't we attempted to burn both ends of the second rope already?

At this point, I knew the answer didn't make sense nor was it cohesive. But what's particularly interesting is that some key phrases were there - "light the other end", "second rope as a timer" e.t.c. For me, this was the prime example of how ChatGPT wasn't capable of ingenuity itself (for now at least) but rather, is great at collating and repeating information. Having decided that the human realm was safe* for now, I decided to go about my weekend. But who knows? Maybe it was even smarter and decided not to share its plans and secrets with the enemy. Regardless, I decided to just go ahead and ask it.

Rohan: Are you capable of ingenuity?

ChatGPT: As an artificial intelligence language model, I do not possess ingenuity in the same way that humans do. Ingenuity typically involves originality, creativity, and the ability to solve problems in unconventional or unique ways. While I can generate text responses based on patterns learned during training, I do not have the capability to independently generate truly original or creative ideas, nor do I possess problem-solving skills beyond the patterns learned from my training data. My responses are generated based on statistical patterns and associations learned from large amounts of text data, without true creative or innovative thinking.