FOD#67: o in o1 – the first star in Orion constellation

plus the little dictionary of terms and collection of interesting reads on the topic

Follow us on Twitter or LinkedIn

This Week in Turing Post:

• Wednesday, AI 101: we introduce a new size-bite cards with concepts
• Friday, a surprise project: we unpack generational stereotypes in four major image generation models.

The main topic

I don’t know why no one has noticed that the "o" in OpenAI's newest model, o1, likely stands for Orion – the long-ago announced model. My chain of thought (haha) is this: during the developers' AMA, the OpenAI team mentioned that o1 is a "model" and "not a system." Meanwhile, Sam Altman tweeted a cryptic message:

"I love being home in the Midwest. The night sky is so beautiful. Excited for the winter constellations to rise soon; they are so great."

Of course, Orion is a winter constellation in the Northern Hemisphere.

This suggests that OpenAI is working on a larger system – a constellation – where o1 is just one star. Maybe all of this is inconsequential, but I’m fascinated by the metaphor. Constellations have been used for navigation and storytelling since the dawn of time, and I believe OpenAI is mapping out its own constellation of AI models, each playing a distinct role in a broader, interconnected system. It also plans to build a narrative and navigate the discourse.

What fascinates me most is the possibility that each “star” in this constellation, like o1, represents not just an individual model but a piece of a larger, more integrated framework. These models might be designed to collaborate, enhancing their reasoning and decision-making capacities as a unified whole. It’s a compelling vision –?a new kind of AI ecosystem, where each component is aligned with a purpose, like stars aligning to form a pattern in the sky.

I think there's something poetic about this, don’t you?

But back to the ground: in his thorough and insightful deep dive into o1, Nathan Lambert aptly noted , “This is a soft segue into real language model agents.” The reason for this lies in the unique capabilities o1 brings to the table. By combining reinforcement learning (RL), search-based reasoning, and chain-of-thought (CoT) mechanisms, o1 represents a significant step forward. These foundational elements are critical for developing more advanced, autonomous AI systems, making o1 not just another language model but a piece in a bigger picture toward the future of intelligent agents. And as a single piece, it doesn’t have to be perfect.

I’ll provide a list of links worth reading about o1, but before that, it might be useful to be equipped with a little dictionary. First, let’s clarify the chain: q* → Strawberry → o1.

• Reinforcement Learning (RL): The technique used to train o1, where the model improves by getting feedback (rewards) based on its actions or reasoning. RL allows o1 to try different approaches, learn from mistakes, and continuously improve.
• RL-Based Search Algorithm: Refers to the reinforcement learning mechanism that helps o1 search over reasoning spaces to solve problems more efficiently.
• Chain-of-Thought (CoT) Reasoning: The process where the model breaks down complex tasks into smaller steps and systematically works through them, similar to how humans solve intricate problems step-by-step. This results in more accurate conclusions.
• Inference-Time Scaling: In most models, the heavy computational work happens during training, but with o1, the real action happens during inference. As the complexity of a task increases, o1 spends more time thinking, scaling its computations dynamically as it generates responses.
• Test-Time Compute Scaling: A new approach where the model dedicates more computational resources while it's actively solving tasks, leading to improved reasoning at the cost of increased compute power. This scaling happens in real time during the problem-solving process.
• Self-play Reinforcement Learning: A method where the model learns by solving problems independently, similar to how AI models mastered games like Go. In o1, this approach helps improve decision-making in real-world tasks.
• Hidden Reasoning Tokens: These are the internal, unseen steps o1 takes while reasoning through a problem. OpenAI has chosen not to make these visible, citing safety concerns and competitive advantages, which adds a layer of mystery to the reasoning process.
• AIME and ARC Benchmarks: These are tests used to measure o1's problem-solving and reasoning performance, particularly in mathematics and science. OpenAI claims that o1 surpasses GPT-4 in these domains.

To read more:

OpenAI o1 hub

OpenAI o1-preview

OpenAI technical research blog

OpenAI o1 System Card

OpenAI platform docs

Contributions

OpenAI o1 Results on ARC-AGI-Pub

Summary of AMA with OpenAI team

Amazing analysis by Nathan Lambert

Amazing analysis by Jim Fan

Simon Willison’s notes

Healthcare application of o1-preview

Trying it out by Ethan Mollick

Trying it out by Rohit Krishnan

Importance of self-play

Trying to reason like o1

Our Twitter library

Weekly recommendation from AI practitioner????:

• V0 by vercel – it's like talking to your wireframes and it's going to launch a million of similarly looking prototypes.

News from The Usual Suspects ?

• Microsoft Makes Waves: UI layer for better AI

• World Labs: The Next Dimension of AI

Fei-Fei Li's World Labs is setting its sights on spatial intelligence with Large World Models (LWMs) that can perceive and interact in 3D. By moving beyond 2D, they aim to revolutionize AI’s understanding of the world, from virtual realms to real-life applications. With $230M in funding and big-name investors, 3D creativity is about to get a major upgrade.

• OpenAI's $150B Question: Corporate Revolution

OpenAI’s next $6.5 billion financing round comes with strings attached: restructuring its nonprofit to remove a profit cap for investors. The $150 billion valuation depends on this shift, promising huge returns to early backers while raising concerns over the company's mission to balance commercial ambition with AI safety. A gamble, but investors seem eager.

• Salesforce’s AgentForce: AI Gets to Work

Salesforce debuts AgentForce , its AI-driven solution for businesses looking to add always-on autonomous agents to their teams.

• Oracle's Zettascale Ambition: A New Era in Cloud AI

Oracle has unveiled the first zettascale cloud supercomputer , featuring up to 131,072 NVIDIA Blackwell GPUs for AI workloads. Boasting 2.4 zettaFLOPS of peak performance, it's a game-changer for industries needing AI at scale. Zoom and WideLabs are already leveraging Oracle’s AI sovereignty and performance to drive innovation. AI just hit hyperspeed.

• Musk’s Sky Monopoly: Two-Thirds of All Satellites

Elon Musk’s SpaceX now controls over 62% of all active satellites , thanks to its ever-growing Starlink constellation, which adds about three satellites per day. With more than 6,300 satellites in low-Earth orbit, SpaceX aims to reach 42,000 for global internet coverage. That’s crazy.

• Hugging Face - ZeroGPU

??♀? Hot Models ??♂?

that didn’t receive nearly as much attention as o1

• Google's DataGemma: Fighting AI Hallucinations with Facts

Google introduces DataGemma , the first open model linking language models to real-world data from the extensive Data Commons. With 240 billion data points, this innovation tackles AI hallucinations by grounding responses in factual information. Through new RIG and RAG techniques, it enhances accuracy and reliability, moving LLMs closer to trustworthy AI.

• Mistral’s Pixtral 12B: Seeing is Believing

French AI startup Mistral just dropped Pixtral 12B , a 12-billion-parameter multimodal model capable of processing both text and images. Whether captioning photos or counting objects, Pixtral is setting its sights on becoming a serious player in AI image understanding. Open for fine-tuning under Apache 2.0, it's the latest move in Mistral’s rise as Europe's answer to OpenAI.

We are watching/reading:

• Highly recommended: but we would rename it to?

GitHub - stas00/ml-engineering: Machine Learning Engineering Open Book

• Common LLM Settings by Stella Biderman?
• https://x.com/karpathy/status/1835024197506187617 ?
• https://www.platformer.news/mark-zuckerberg-acquired-podcast-interview

The freshest research papers were published. We categorized them for your convenience ????