How does AI work? How does ChatGPT work? Will it ever be alive?

WHAT IS AI? WILL IT EVER “WAKE”? HOW DOES CHATGPT WORK? WILL IT REPLACE JOBS? AND HOW YOU CAN BE BETTER THAN AI?

I was in Kai-Fu Lee’s lab at CMU in 1989 and he said, “The AI will understand anything you say. As long as you say something that could be considered a command on a Navy Destroyer ship.”?

The government had funded him. I forget the first ever words ever understood by a computer but it had something to do with killing people far away.?

--

Ka-Fu Lee ended up developing what became Siri for Apple before he went to work at Microsoft and then Google and then went to China where he became the largest venture capitalist there.?

Recently he came on my podcast and described how he beat cancer and what AI might mean to us in the future.?

He co-wrote the science fiction book of stories “2041” with a great science fiction writer. Chen Qiufan.

The first story was about a girl getting insurance. The insurance company used AI to read all of her emails and texts. And because she was getting into an inter-racial relationship and apparently there were statistics about this sort of thing in 2041, the AI of the insurance company decided her entire family needed to pay higher insurance premiums.?

This is one of the dangers of AI.?

(here’s Part 1 of my podcast with Kai-Fu about AI or just click on image: https://podcasts.apple.com/rw/podcast/754-kai-fu-lee-part-1-ai-2041-ten-visions-for-our-future/id794030859?i=1000535264189 )?

—-

A few years later IBM made a mistake and tried to offer me a job. I helped them correct their mistake by turning it down.?

They wanted me to work on “Deep Blue” the AI computer that eventually beat the world chess champion, Garry Kasparov.?

I had worked briefly on an earlier version of Deep Blue, when it was called “Chiptest” and later called “Deep Thought”. Before IBM got involved.?

I had an idea to improve it that they wanted me to work on. A chess program builds a tree of moves. You have 10 possible moves, say, and the opponent has ten possible moves to each of those moves (so now the computer has to look at 100 moves) and so on. Computers could look 20,30, 100 moves deep.?

I said, “in the tree of moves, let the players make two moves in a row. If, after making two moves in a row, the player is winning then you know the first move was an important threat and you can focus on that move before all of the others. This would speed up the computer.”?

Speed is very important to AI.?

They didn’t really need me to implement the idea and they did it and then beat the world champion.

I’m not saying it was my idea that helped Deep Blue beat the human world chess champion but 20 years later, when I told Garry of my role in Deep Blue, there might have been a flicker of disgust in his eyes bringing back some memory.??

—-

A decade later I was eating lunch at a men’s only club on Park Avenue in NYC.?

I was trying to explain my hedge fund strategy to an older guy who was thinking of investing. He had somehow been involved in Enron but also not involved so he was free to enjoy his wealth.?

“My software looks for patterns in the market that are statistically significant,” I said. “Like, it will notice that the past 20 times Microsoft had a bad earnings report and then went down three days in a row, then on the fourth day it always goes up. It has 1000s of patterns like that that it discovered in the data. Then I have another piece of software that simply waits for a pattern to show up and then makes the trade at the critical moment.”

“It’s like artificial intelligence,” he said.?

“Yes.”?

He didn’t invest.?

The guy who introduced us called me up to tell me why. “He said to me, ‘it’s interesting but it’s one of those things that will keep working until suddenly it doesn’t.’ “?

A few years later (2005) I wrote a book, ‘Trade Like a Hedge Fund’ where I revealed about 20 of the patterns. They all stopped working after the book came out.?

This is a limitation of AI.?

—-

STEP ONE: Basic AI

Here’s how AI works.?

1. Give the computer a large amount of data and label the data.?
2. Example: here’s 10,000 X-Rays of a lung. 5000 of the X-Rays are labeled “Cancer” and 5,000 are labeled “Healthy”.?
3. Give the computer a new X-Ray and ask it: does this X-Ray show cancer or not??
4. The computer runs its AI software on it. It determines, given the 10,000 labeled examples, if the new X-Ray is a closer match to the 5,000 labeled “Cancer” or a closer match to the 5,000 X-rays labeled “Healthy”.?
5. How does it do this??
6. It can use statistics.?
7. It can use neural networks.?

The method doesn’t matter so much in basic AI. More advanced AI uses neural networks to learn and categorize patterns.?

The important thing is: AI does pattern recognition. It asks the question, “What does this look like that I have seen before?” And then it asks, “And given that I have seen this before, what should I then tell the user?”?

Computer are much better than humans at detecting cancer in an X-Ray. It’s so much better that a law was made that says, “Human doctors are required to tell the answer to a patient.” Else there would be no need for radiologists.?

—---

STEP TWO: CAPTCHA AND COMPUTER VISION

AI was bullshit for a long time. Everyone realized it was just statistics. So AI stopped getting funded by the government and just became the topic for movies. Ex Machina, Terminator, etc.?

Except for computer vision.?

Using the early techniques of speech recognition, Computer Vision became a big deal.?

One time, early on, I applied for a job at MIT Lincoln Laboratories. The task: use early computer vision techniques to determine which objects in space were junk abandoned by rockets and which objects might be guided missiles heading for the US.?

Later, the same technology would be used by cars doing automated driving to “see” if something was a Stop Sign or a little baby crawling in the street.?

Here’s an interesting thing:?

You know when you log into a website and it says, “Prove you are not a bot? Click on the images that have bicycles in them?”

And then eight images show up and you have to click the five that have bicycles.?

Guess what, the computer already knows you are a human. “Bots” have a tendency to click the exact center of an image so as soon as you click, the system knows you are imperfect enough to be a human.?

But when you click the five out of eight (or three out of eight or two out of eight or whatever) images that contain bicycles, your response and those images are fed into a MASSIVE database of images with the label “bicycle” or “no bicycle”.?

The real purpose of Captcha is not to determine if you are human or not but to label billions of images with data that are used by computer vision systems. Gotcha!?

Computer vision systems use neural networks.?

The brain works like this:?

You have 100 billion neurons, give or take. When you see an image, a bunch of neurons fire up. Some of those neurons are related to color (the red neurons might fire up if you see see a Stop sign and the blue neurons fire up if you see the sky).?

Some of those neurons are related to edges. When you look at the sky, the edge neurons fire up when you focus on the edge that separates the top of the trees versus the beginning of the sky.?

They then send electrical signals to other neurons. “I just saw red! I just saw an edge!” Those neurons signal other neurons and so on. There is constant electricity going on in your brain.?

Eventually, enough neurons are triggered that they reach out to neurons that store all of your memories. A memory neuron might light up and say, “That looks like me! I’m a stop sign.”?

Neural networks work the exact same way.?

When a neural network is trained with labeled data (like the captcha examples) it creates weights that link neurons together. The more data it is fed per category (a category like “red” or “bicycle”) the stronger the weights get for future inputs in that category. So it knows which neurons to signal for which patterns.?

This is not that important. Just think of it as advanced statistics.?

Now we get to ChatGPT

—--

STEP THREE: GOOGLE SEARCH QUERIES

When you type the letter “t” into the Google search box, what comes up??

If it’s the first time you are using Google, the word “the” might pop up in a menu even if you just typed in “t”.?

That’s because “the” is the most popular word in the English language and, of course, the most likely word you intend to type into Google once you type the letter “t”.?

Google then “learns” what you usually search for, what members of your house usually search for, what people in your city usually search for, etc and they build a better AI model of how to fill out the rest of your search query.?

This is what comes up for me when I type in “t” right now on Google.?

This is basic AI using basic statistics.?

STEP FOUR: CHATGPT

Chat GPT goes several steps further.?

First off, very important, ChatGPT doesn't label data (in the beginning).?

It uses what is called “unsupervised learning”.?

Imagine if you feed a computer pictures of cats and pictures of dogs. You tell the computer there are two categories but nothing else. You don’t label the pictures.?

Unsupervised learning is when the AI system takes these pictures and uses neural networks to determine the “distance” between one input and another.?

Hmm, it might say, “this picture is very different from this other picture”. One was a photo of a cat and one was a photo of a dog.?

Gradually it realizes that the photos of cats belong in one category and the photos of dogs belong in another category.?

Later, if you say, “show me other pictures like this one” and you put in a brand new photo of your German Shepherd, it will output other photos of dogs.?

If it has categorized further it might show you photos of large dogs or, even further, photos of German Shepherds.

This is “unsupervised learning”. Where you feed in a large amount of data and it separates the data into clusters of similar data.?

NOW: MAKE YOUR OWN CHATGPT:?

1. Feed in all the text ever written. All articles, books, Wikipedia pages, tweets, reddit posts, Facebook posts, etc.?
2. Use unsupervised learning to categorize the text into millions of clusters.?
3. Example: the following example text appears all over the Internet. It’s the beginning lyrics of a Bob Dylan song:?

-----

• Early one mornin' the sun was shinin'
• I was layin' in bed
• Wondrin' if she'd changed at all
• If her hair was red

-----

This text might be in many categories in ChatGPT: “bob dylan”, “hair”, “poetry”, “rhyming”, “sentences about the color "red”, etc.?

So now if someone says, “write a unique poem in the style of Nobel Prize winner Bob Dylan”, maybe the output might be:?

“I was layin’ in the sun

And her red was hair

And she changed into a bed”.?

Now Supervised learning might take place at this point.?

Someone would have to say, “This is not good output”.?

So ChatGPT tries again.?

“Her hair was shining

And her hair was red

I went off to do mining

Where they found me dead.”?

And the person doing the supervised mining might say, “That’s better.”?

And so on.?

ChatGPT took all of the text ever written before 2021 (this is called an LLM - a large language model) and then fed it into neural networks to do unsupervised learning and develop millions and millions of clusters.?

This took 1.5 years.?

Then, when asked a query, it will try to create a sentence that overlaps all or most of the clusters of the query.??

In other words, it asks itself, “given the clusters I’ve seen, and given a sentence that matches all of these clusters, what is the most likely next sentence?”?

And despite the 1.5 years of unsupervised learning, it will still mess up the outputs.?

So 1000s of workers (mostly workers from Kenya getting paid $2 / hour) spent the next year doing supervised learning: labeling the outputs good or bad.?

So now ChatGPT then clusters the outputs so when it has to create new outputs it tries to figure out what are good outputs vs bad outputs.?

Doing this gives it knowledge of context, grammar, language structure, etc.?

This is how ChatGPT works.?

Now if I give it my biography and say, “Write my resume” it understands what are the important parts of my biography in terms of resume writing, what a resume looks like, what the most successful resumes look like, and it is able to write my resume.?

If I say, “Write an eloquent post about me” it understands what words are considered eloquent vs not-eloquent and it knows my biography and is able to write about me in an eloquent way.?

And so on.?

ChatGPT is not conscious in any way at all. It is not sentient. It never will be.?

It takes a set of words, figures out all the contexts for that set of words, and determines the most likely response to that set of words, given all the contexts.?

That’s it. It knows nothing. It just figures out probabilities of what words appear next. Just like the Google Search Query system except a billion times more data and clusters.?

—---

AI JAMES

By the way, at a website I created, notepd.com, I fed in about 1000 of my articles and created an AI James.?

https://notepd.com/write_with_james

If you want to create an AI version of yourself, let me know and I’ll do it.?

—--

WHAT CAN I DO WITH AI??

Some people think “ChatGPT” is like God.?

I said to my wife, “I can’t think of a topic for my next book.” She’ll say, “Just ask ChatGPT”.?

"What should we watch on TV tonight?"

"Ask ChatGPT."

And she might be right. Or wrong.?

ChatGPT is the fastest growing application in history. Going from zero to 100,000,000 users in just 60 days.?

And people have been developing hundreds of applications using ChatGPT.?

Here’s a list of 1413 applications created on top of ChatGPT:?

I helped create one app: proposalgenie.ai. This app helps workers on Upwork create proposals for jobs that are posted on Upwork.?

—-

Will ChatGPT take jobs away from people??

Yes and no.?

Did automobiles take jobs away from horse carriage drivers??

Yes but…

Did ATM machines take away jobs from bank tellers??

Yes but…

Anything created to increase productivity will grow the economy, create new industries, which will create new companies and new jobs.?

Already I see job postings for “prompt engineers” - people who can help direct AI for specific purposes.?

I spoke to a graphic designer who designs album covers for bands. He said, “I can do 10 jobs a day now instead of one job a week.” Bands still want to deal with a human.?

I spoke to Matt Barrie on my podcast. He is the CEO of Freelancer.com which has 60,000,000 freelancers looking for jobs.?

He said since ChatGPT started just two months ago he’s seen freelancers get more jobs and he’s see tons of AI startups created that are hiring these freelancers.?

We are just at the beginning of this. Nobody would’ve expected when the web was created the industries that would’ve developed around the making of websites, ecommerce, online media, streaming, etc. and the tens of millions of people employed by those industries.?

The same will be true for AI. Again, AI is not conscious and it never will be. But you are.?

Some industries that will change.?

We already saw an example from medicine above. But here are two more industries that are immediately changing. There are many more.

Legal:?

AI can automatically respond to parking tickets and send the appropriate letters to a judge to dismiss the case.?

AI can read through millions of documents (replacing paralegals) and tell you which documents are relevant for which cases.?

Journalism: You don’t need reporters anymore to summarize what happened in a local city council meeting. AI can do that.?

But AI can’t do investigative journalism and find NEW real world things that might effect a story.?

But you can.?

AI can’t replace humans who are seeking out new unique experiences and new, unique ideas.?

--

Will AI dominate humans and take over??

Have an interesting life and you will be the master of AI.?

Have. an. Interesting. Life.?