OpenAI o1 System Card

Today's paper introduces OpenAI's new o1 model series, which uses large-scale reinforcement learning to perform chain-of-thought reasoning. These models demonstrate improved safety and robustness compared to previous versions, particularly in areas like avoiding unsafe content generation and resisting jailbreak attempts. The paper outlines the safety evaluations and red teaming conducted on the o1-preview and o1-mini models.

Overview

The o1 models are trained using large-scale reinforcement learning to perform complex reasoning tasks. Unlike previous models that provide immediate responses, o1 models generate a chain of thought before answering user queries. This allows them to refine their thinking process, try different strategies, and recognize mistakes.

The training data for these models comes from a combination of public datasets, proprietary data accessed through partnerships, and custom datasets developed in-house. This diverse data helps the models develop robust reasoning and conversational capabilities across various domains.

A key feature of the o1 models is their ability to reason about safety policies in context when responding to potentially unsafe prompts. This leads to improved performance on benchmarks related to avoiding illicit advice, stereotyped responses, and known jailbreak attempts.

The paper describes extensive safety evaluations conducted on the o1 models, including tests for disallowed content generation, jailbreak resistance, hallucination tendencies, and bias. They also investigate potential risks associated with the chain-of-thought feature and describe ongoing research on chain-of-thought detection monitoring.

Key Results

The o1 models show significant improvements in safety and robustness compared to previous versions:

• They outperform GPT-4o on challenging refusal evaluations and jailbreak resistance tests.
• The models demonstrate reduced hallucination rates in certain evaluations.

• o1-preview shows improved performance on fairness and bias evaluations, particularly in avoiding stereotyped responses.
• Initial monitoring of chain-of-thought outputs shows promising results in detecting potential deceptive behavior.
• Significant improvements over GPT-4o were observed in OpenAI Research Engineer interview tasks, both in multiple-choice questions and coding problems.

• In agentic tasks, both models could not complete primary tasks related to advanced autonomy but showed strong performance on contextual subtasks.

• Multilingual capabilities are notably higher in o1-preview compared to GPT-4o, with strong performance across 14 languages, including low-resource languages like Swahili and Yoruba.

Conclusion

The paper introduces OpenAI's o1 model series, which uses chain-of-thought reasoning to improve safety and performance in language AI systems. For more information please consult the?full paper.

Congrats to the authors for their work!

OpenAI. "OpenAI o1 System Card.", https://cdn.openai.com/o1-system-card.pdf.