ChatGPT-4o: Daft Punk 2.0 - Faster, Cheaper & "Better"

OpenAI claims that ChatGPT-4o is faster, cheaper, and "better". Challenge accepted! Let’s see if this Elon Musk-like statement holds up.

The hypothesis we want to test is whether both models have the same training data, ensuring they have fair conditions for benchmarking.

ChatGPT-4 - Latest date included

Let's simply ask ChatGPT-4 the most recent data included in its model:

It's trained data contains information up to December 2023. That was one of the biggest improvements compared to other models.

ChatGPT-4o - Latest date included

Surprisingly the first answer on this model is not what we were expecting and we found a cork.

Based on its answer it only holds up data until September 2021, but we know for a fact that the training data for ChatGPT-4o extends at least to some point in 2023.

Upon further insistence and testing, we discovered that the training data includes information up to April 2023, consistent testing confirms that.

Is then "Better", really?

This raises the question: How can a model with less training data be considered better?

For sure one part lies in the significant improvements made in merging multimodal capabilities (text, voice, image) into a one highly sophisticated model (before it was 3 models).

Confirming the statement

Let's test the claim that ChatGPT-4o provides "better" answers than ChatGPT-4, even though we know ChatGPT-4o has less updated data.

To do this, we'll use a well-known event from October 2023:

On October 2, 2023, Katalin Karikó and Drew Weissman won the Nobel Prize in Medicine for their research on messenger RNA (mRNA). Source

ChatGPT-4

Here we can see that ChatGPT-4 provides inaccurate data despite having a larger dataset that extends until December 2023.

ChatGPT-4o

Surprise, surprise! Despite ChatGPT-4o having less updated data, it fetches real-time data, cites its sources, and delivers more accurate information that wasn't part of its initial training.

In addition, if you ask the model for the same information later, it acts like it always knew the answer. The citations disappear, and the knowledge is presented as if it were always there.

NOTE: this part is still volatile so it will not keep forever as learned (yet)

Summary

Indeed ChatGPT-4o is faster, cheaper and better than ChatGPT-4, even though at first glance it appears to have less up-to-date data.

Speaking about Nobel prizes ... OpenAI ??