AI Model o3 Scores More in Programming Tests than its Developers
Satish Mehta
Cost Accountant, Software Requirement Analyst, AI Faculty, MIS & Data Analyst and Tax professional
21-Dec-2024
On 20-Dec-2024, OpenAI announced new AI model o3 which is much better in doing complex work like ?programming, maths and science than the previous model o1. But more interesting fact is that the model scores more in programming test than its developers. ??This model has achieved Codeforce (a competitive programming g test) ?score of 2727. The score of Mark, who presented the o3 model (in presence of Sam Altman) was 2500. Score of OpenAI's Ex Chief Scientist and Cofounder, Ily Sutskever was 2665. Thus the AI model has got better score than their developer.??
?
The model is not only good in programming. It has also scored high in several other difficult tests. I would like to mention two tests below. The first is GPQA or Google Proof Question Answer. It asks ?Ph. D. level questions and the user is free to use Google to find reply up to 30 minutes. But questions are “Google Proof” so you will not get direct answers from internet. Average Ph. D. scores ?65 to 74 % in this test in their respective area. ?Model o1 scored 78 %. ?Model o3 scored 87.7 % making it the only model to reach this score.
?
Another difficult test is ARC AGI (Artificial General Intelligence) test. ?O3 scored 75 to 87 %. The corresponding score of O1 was 8 to 30 %. (ARC's president remained present in the o3 announcement function ?and ?announced the score of O3 so that people can trust these unbelievable figures)
?
Based on the above benchmarks, it is clear that the model o3 is highly intelligent. Now we have to see how applications utilize intelligence of this model for different type of tasks. It will take another year or two before such applications are available.
?
At present the model is only released to Safety testers. It may be released to public for testing by end of Jan-2025.
?
You can read view more about these here: -
领英推荐
?
OpenAI Video:
?
Other articles: -
?
?
?
CMA Satish Mehta
If you want to get frequent updates about AI, join my free WhatsApp groups. Send me e-mail at [email protected] to get links to join the groups.