Eyes, Ears, Hands and Mouth

The Next Stage in AI

OpenAi and Google announced their AI offerings' next iteration this week. As @Om Malik explains in one of this week’s Essays of the Week, OpenAi won in this high-stakes battle.

Make no mistake — the reason OpenAI is achieving all this success (and hype) is because they have a product that for now is stellar. Nonetheless, OpenAI has created excitement that reminds me of the emergence of Palm, and later social networks. They stoked the imagination, and possibilities. Of course!

Om is right. Sam Altman did his own post later in the day of the announcements:

First, a key part of our mission is to put very capable AI tools in the hands of people for free (or at a great price). I am very proud that we’ve made the best model in the world available for free in ChatGPT, without ads or anything like that.

Free to consumers, or 8 billion earthlings, is possible due to the revenues OpenAI can make from business users. It represents a very big step forward. The company also released a desktop app, initially on the Mac, that can interact with other apps.

But for me, the best way to think about what was delivered, aside from free, is summed up in this week’s title—Eyes, Ears, Hands, and Mouth. OpenAI has enabled every smartphone camera on the planet to become the AI's eyes and ears. Both still images and video can be used as inputs to a conversation. Of course, the microphone, too. This week’s video of the week shows this for teaching a student how to solve a math problem. The mouth reference acknowledges that we can now speak to ChatGPT in a human-like way, including cross-talking and interruptions. And, of course, we can still type using our hands.

This changes the problem of giving AI data—images, video, sound, and speech can all become data for input and learning.

They also gave chatGPT a memory. It can remember things across sessions. The scope of what will now be possible is expanded to a much longer list.

Rohit Krishnan writes about what comes next in his essay:

The true change will come once we can enable large numbers of them to work together. And we’re getting glimpses of how they can do this across all modalities that are important to use. Whether that’s writing code or seeing something or listening to something or writing or reading something or a mixture of all of these.

He is talking about AI to AI interactions that can produce even better and faster outcomes. I did this myself earlier in the week. I was asking ChatGPT to create a chart showing the performance of the SignalRank Index against the S&P 500 and the NASDAQ over the 2014-2019 period.

ChatGPT did not have the NASDAQ data, so I asked Claude.ai for it. Once I had it I went back to ChatGPT and it completed the work. Here’s the chart:

It seems clear that almost any problem that can be described, shown, listened to can now be given to ChatGPT and answered. Eyes, Ears, Hands, and mouths are all part of our intelligent robotic future, too. The building blocks for rapid productivity advancement are being put into place.

Marc Andreessen and Ben Horowitz discuss the implications for manufacturing in their podcast this week.

This was a very important week.

Contents

• Editorial: Eyes, Ears, Hands, and Mouth
• Essays of the Week
• Video of the Week
• AI of the Week
• News Of the Week
• Startup of the Week
• X of the Week Sam Altman on Ilya leaving OpenAI

Congratulations to this week’s chosen creators: @sama, @openai, @om, @krishnanrohit, @peternixey, @eringriffith, @AndreRetterath, @ry_paddy, @cutler_max, @Kantrowitz, @PranavDixit, @ttunguz, @geneteare, @sarahfielding_, @carlfranzen