OpenAI's o1: Brilliant Mind or Costly Thinker?

In a week marked by OpenAI's unveiling of what many are calling the most impressive AI model to date, we at Nido have found ourselves reflecting not only on the model itself but also on its role within the larger AI ecosystem and its business implications.?

So, let’s start with the basics: what’s going on in the AI world and why is it important?

OpenAI? just announced its latest model: o1. This model promises to revolutionize AI problem-solving by introducing a new era of reasoning capabilities. Yet beneath the surface of this bold announcement, we must ask: Is o1 truly the game-changer it appears to be, or are we witnessing the AI equivalent of a well-rehearsed job interview?

OpenAI claims to have “developed a new class of AI models that take more time to think before responding. They can reason through complex tasks and solve harder problems than previous models, excelling in fields like science, coding, and mathematics” (more on OpenAI). This brings to mind management consulting case interviews or coding challenges, where candidates are trained to meticulously show their work and think step-by-step before responding. In these interviews, the process to get to the answer is almost as important as the answer itself.

Imagine a management consultant or software engineer who never gets tired, never asks for a raise, and processes vast amounts of data in seconds. That’s o1. Its “chain of thought” process mirrors human-like, step-by-step reasoning, akin to how a consultant might methodically work through a business case. But unlike humans, o1 doesn’t need breaks or sleep.

The benchmarks are impressive:? o1 ranks in the 89th percentile on Codeforces programming questions, scores 83% on International Mathematics Olympiad qualifying exams (compared to GPT-4’s 13%), and performs at a PhD level in subjects like physics, chemistry, and biology (more on OpenAI).

At first glance, it seems like o1 could be the obvious choice for tackling complex tasks. However, as any business leader knows, credentials don’t always equate to on-the-job performance.

These results remind us of hiring a PhD scientist or a star software engineer who excels in academic settings. Yet, just as a brilliant researcher might struggle with the practical realities of project management or client communication, o1's extraordinary performance in structured tasks doesn't necessarily extend to every business scenario.

Academic excellence is not always a strong predictor of career success. (more on Global Leadership Network) Similarly, o1 could encounter challenges when faced with the messy realities of daily business operations. Indeed, human expert reviews have already shown a preference for GPT-4o in certain natural language tasks, suggesting that o1 may not be the ideal fit for every use case (more on Ars Technica).

The training behind o1 is fundamentally different from its predecessors. OpenAI's research lead, Jerry Tworek, states that o1 "has been trained using a completely new optimization algorithm and a new training dataset specifically tailored for it." Unlike previous GPT models, which primarily mimicked patterns from training data, o1 uses reinforcement learning to solve problems on its own, rewarding correct steps and penalizing mistakes (more on The Verge). Yet, significant hurdles remain: speed and cost. O1 is about 30 times slower than GPT-4o, with its "mini" version still lagging 16 times behind. In a world where time is money, this sluggishness could be a major drawback. Imagine waiting minutes for an AI to "think" in a customer service scenario—clients might prefer the speed of a human interaction, even if it's less polished.?

Moreover, o1's pricing structure highlights its significant cost implications. In the context of AI language models, tokens are the basic units of text processing - typically, a token is about 4 characters or 3/4 of an English word. At $15 per million input tokens (the text you feed into the model) and $60 per million output tokens (the text the model generates), o1 is three times more expensive than GPT-4o.? Even the mini version remains 20 times pricier than GPT-4o mini. This creates a divide where only large enterprises can afford access to the most advanced AI tools. Additionally, o1 introduces "reasoning tokens"—the hidden process of breaking big problems into small steps—further driving up costs (more on TechCrunch).

This cost-speed tradeoff mirrors the dilemma of hiring a world-class expert versus a competent team. A PhD scientist might solve complex problems elegantly, but that expertise comes with a high price tag that isn't always justified for everyday tasks.

o1's implementation poses additional challenges. OpenAI has hidden the model's "chain of thought," providing only a summary of its reasoning. This black-box approach makes it difficult to assess performance or understand decision-making, crucial for building reliable business processes. Even more concerning, running the same query multiple times can yield different results, introducing an element of unpredictability that could be problematic in production environments.

In addition, while quantitative evaluations suggest that o1 models hallucinate less frequently than their GPT-4 counterparts, anecdotal feedback indicates the opposite. This discrepancy between controlled tests and real-world observations highlights the complexity of AI performance and the potential pitfalls of relying solely on benchmark results.(more on OpenAI o1 System Card)? It serves as a crucial reminder that as AI models become more sophisticated, their practical applications may yield unexpected outcomes, emphasizing the need for comprehensive, real-world testing alongside traditional evaluations.

Despite these hurdles, o1 represents a significant leap forward in AI capabilities. Its potential to reshape industries is immense—imagine legal documents drafted in minutes or supply chain optimizations in real time. Its precision in fields like science, math, and coding has already proven invaluable for complex problem-solving, from software debugging to healthcare research.?

The future of AI in business isn't about selecting the "best" model but strategically deploying each one where it can create the most impact. Just as software engineers or consultants focus their expertise on complex challenges, o1’s advanced reasoning should be applied to high-value problems, while more routine tasks are handled by faster models like GPT-4o. The real advantage comes from building a diverse AI toolkit that enhances productivity across all functions, from streamlining code to optimizing business processes.

As we stand on the brink of this new AI frontier, it's clear that success will belong to those who can orchestrate a symphony of AI models, each playing its part in the grand composition of business operations. o1 isn’t just a tool—it’s a catalyst for rethinking how work gets done. The question isn't whether o1 is groundbreaking—it undeniably is. The real challenge is: Are we ready to collaborate with AI to reshape how we solve the problems of tomorrow?

Written by Ana Carolina Mexia Ponce