Reasoning Series, Part 1: Understanding GPT-o1

Oct 8, 2024 • Han Lee | 7 min read (1220 words)  

OpenAI released the long-awaited GPT-o1 preview on September 12, 2024. This model was previously known as Q* in 2023 and superceded by Project Strawberry in 2024. In this first installment of the Reasoning series, we aim to separate rumors from facts about how the GPT-o1 model works and validate our conjectures through experiments. This will help users better understand and utilize GPT-o1.

In Thinking, Fast and Slow, Daniel Kahneman defined System 1 as the automatic, intuitive mode of thinking, and System 2 as the slower, more analytical mode. In the context of autoregressive language models, the usual inference process is akin to System 1—models generate answers directly.

Reasoning, however, gives models the ability to perform System 2 thinking by introducing “reasoning tokens.” This is similar to the Zero-Shot Chain of Thought (CoT@0) approach by Kojima 2022, where we prompt a model to “think step by step” before answering a question. The o1 model formalizes this concept by providing a “scratchpad” Nye 2021, allowing models to reason more effectively, especially for tasks that benefit from more thinking time, measured in reasoning tokens.

Leave a Comment