In recent months, the AI industry has been abuzz with predictions, rumors, and unexpected decisions. From shocking resignations to astounding innovations, the atmosphere was electric. However, there was one common belief: until now, AI hasn't "thought." Sure, it could answer questions, but it wasn't really reflecting. Well, it seems something has changed.
GPT o1, OpenAI's new model, has taken a step towards what we might call "thinking." In essence, while previous models responded instantly to questions, much like anyone facing a pressing inquiry, GPT o1 takes its time and reflects. It doesn't respond immediately. Before answering, it creates a plan, evaluates different strategies, and decides how to reply. In short, it's getting closer to the way we humans think.
Let me explain briefly, knowing that we'll need to revisit this topic later and that we're all a bit dazzled in this hype phase.
A Leap from Sci-Fi to Reality
As a science fiction fan, this development makes me wonder: how fast are we moving from sci-fi ideas to concrete science? Think about inventions that once seemed far-fetched, like the universal translator or voice-activated computers we use daily. Just a few months ago, the idea of a "positronic brain" seemed like a dream, but today, we're inching closer to an AI model that truly "thinks."
The time between idea and practical realization is shrinking, with many implications for investment and how costs are distributed between AI producers and their users.
This reminds me of an article from some time ago about the Technological Anticipation GAP.
How Does GPT o1 Work?
Instead of immediately answering a question like previous models, GPT o1 enters a phase of internal reflection. It pauses to consider what to do next.
These "thinking" moments allow it to analyze complex problems, simulating a more human-like problem-solving process. The result? More thoughtful and accurate responses, especially for tasks requiring deep analysis.
So, the old question of 1+1 now turns into something where the system "thinks" for eight seconds and then delivers, with a conclusion, the well-reasoned answer we expected.
An interesting experiment is to ask the same question in multiple languages.
This is what happens if I ask it in Italian:
This is what happens if I ask it in English (If anyone still doubts the usefulness prompt in English or other languages with GPT )
Key Features of GPT o1
There’s a lot of info and news about how it works (I'll list some of the details below for convenience), but here are the features I’d like to highlight:
Advanced reasoning capabilities: Ideal for tasks requiring slow, deliberate thinking.
Reflection time: It can "think" for seconds, and the goal is to extend this time to minutes or hours. (I've made it think for up to 122 seconds).
Improved performance: It surpasses previous math, programming, and science models.
Reasoning tokens: Introduces special tokens that allow the AI to reflect on itself and improve responses.
Click here if you don't know what tokens are
Tokens are kind of like the “bricks” that AIs, like ChatGPT, build their answers with. Every word, word part, or symbol we type is broken down into small pieces called tokens.
For example, the word “intelligence” might be broken down into several tokens, while simpler words, such as “hello,” might be considered one token. The longer and more complex the sentence, the more tokens will be used.
Imagine making an international phone call: the more you speak, the more minutes you use. Similarly, the longer your request, the more tokens you need.
To date, they have different prices depending on whether they are Input tokens (Your prompt) or Output tokens (The response).
Reasoning Tokens?
Imagine answering a complex question to understand better what "reasoning" means in GPT o1.
The silent time you spend thinking about your response, maybe doodling on a notepad to map out your plan, is a reasoning time.
At that time, different tokens are assigned.
Reasoning tokens are the units GPT o1 uses to think and reflect, unlike the normal output tokens used to generate rapid responses, as it's done so far.
To summarize with a technical touch:
Before o1: The inference calculation scaled linearly with model size.
With o1: The inference calculation scales exponentially with the "thinking time."
Why Is This Important?
This new approach allows GPT o1 to solve more complex problems without increasing its size. Instead of becoming larger and more expensive to train, GPT o1 spends more time reasoning and offering more sophisticated answers.
When to Use It?
Tasks where o1 excels:
Complex mathematical problems: Solving advanced equations or mathematical proofs.
Deep scientific analysis: Interpreting experimental data or developing scientific hypotheses.
Advanced programming: Developing complex algorithms or debugging sophisticated code.
Tasks where it doesn’t make sense to use o1:
General conversation: Casual chats or answering simple questions.
Short creative content generation: Writing short texts or social media posts.
Quick fact-finding: Looking up simple facts or easily accessible data.
In these cases, faster, less expensive models like GPT-4 are more efficient and suitable.
But How Much Does Thinking Cost?
One of the most exciting implications of GPT o1 is that reasoning comes with a cost. A few days ago, there were rumors that the current version of GPT would incur a 100x increase in operating expenses.
Today, we understand why:
Before, the computational costs were entirely borne by the AI producer.
With GPT o1, some of these costs shift to the users. Reasoning requires much more computing power.
OpenAI has decided to charge only 6x for reasoning tokens, but this makes reasoning a new operational cost that companies must consider carefully.
Implications for Organizations
As we've seen before, there is no one-size-fits-all solution: no longer a "universal model" suited for all tasks. As I mentioned in my book, we'll need an 'AI team.'
When it comes to o1, it will be crucial to evaluate when it’s worth using, given its reasoning costs. Watch out for waste!
So what...
O1 means:
Advanced reasoning: The first step towards an AI that tries to think like a human.
Higher costs: Using reasoning costs more than traditional models and requires significantly more computing resources.
Model selection: GPT o1 is not a replacement for previous models but a companion. Carefully evaluate when to use it compared to cheaper models.
GPT o1 represents an essential and exciting evolution, but it's not the solution for everything.
Learning how to balance the benefits of advanced reasoning with the associated costs remains very important!
Having algorithms that can think now raises the bar, challenging both OpenAI's competitors (Claude is already doing something similar for complex tasks… but without saying too much) and those of us who use AI.
Just like in the real world, sometimes it makes sense to pay more for better answers, but other times, it’s better to hear opinions and reflect on the decisions we need to make.
See you soon!
Max
P.S. If you want to dig deeper into the technical, economic, and philosophical aspects of o1, here are some interesting links I’ve found on the subject:
Comments