Google's Project Astra

At Google I/O 2024, the company gave a sneak peek on what’s next in the AI world. Until now, human-computer interaction has been limited to text-based conversations but last week, Google as well as Open AI – the company that developed ChatGPT – showcased an upcoming AI assistant that's designed to be more helpful and versatile than current options. Google calls its Project Astra while Open AI announced these capabilities on a new language model called GPT-4o.

What is Project Astra

In simpler words, under Project Astra, Google will bring the multimodal AI language on devices like smartphones and smart glasses that will allow users to interact with their surroundings using text, voice and photos/ videos.

The easiest way to understand this is to draw parallels between Project Astra and Tony Stark’s AI assistant. Remember that scene in Avengers Infinity War when Dr Strange and Tony Stark were having a conversation and Stark asks his assistant in his glasses for information on “Thanos’ children.”“Friday! What am I looking at?” he asks his AI assistant, which responds in a natural conversation style.

A similar video was also demoed at Google I/O 2024. In a short clip, a tester interacts with a prototype of AI agents supported by the company’s multimodal foundation model, Gemini. The prototype runs on a Google Pixel phone and another on a prototype glasses device. "The agent takes in a constant stream of audio and video input. It can reason about its environment in real time and interact with the tester in a conversation about what it is seeing,” the company said.

Notably, Project Astra is still under development and is seen having the potential to be a significant leap forward in AI assistants.

Open Ai's ChatGPT powered by GPT-4o

Open AI hosted an event a day ahead of Google I/O and showcased similar capabilities in its GPT-4o AI model, where “o” stands for “omni”. It showcased that GPT-4o can translate spoken language, solve math problems, debug code, among other things. While Open Al claims an early access program for its voice mode, Google CEO Sundar Pichai said that Google's Project Astra will be available later this year for its Gemini users. Moreover, the demo also showed that Gemini will be available on both smart glasses and smartphones while Open AI just teased the model working on smartphones.