What does it mean to "count OOMs"? New common sense for deciphering the key to AI evolution
Will we reach AGI by 2027? A vision of the future that emerges when deciphering the evolution of deep learning through "OOM (Orders of Magnitude)."
Introduction
How do you feel when you hear that "AI will reach a PhD level in just a few years"? For those who thought "I can't believe it," I want you to know about the concept of "counting OOMs."
If you understand this keyword, you will be able to logically predict why the speed of AI evolution is so fast and what will happen next.
In this article, we will break down the scale-up metric called "OOM (Order of Magnitude)" through the evolution from GPT-2 to GPT-4, and approach the future image of AI.
1. What is OOM? -- A growth curve where "one digit" increases by 10 times
OOM (Order of Magnitude) means an increase of "10 times = 1 digit." When thinking about the evolution of AI, tracking growth in OOM units becomes a "lens" for predicting model performance improvements.
For example:
3x improvement → 0.5 OOM
10x improvement → 1 OOM
100x improvement → 2 OOM
Using this, you can quantitatively estimate "how much will it evolve in the future?"
2. Why does AI evolve? -- 3 OOM growth drivers
In just four years from GPT-2 to GPT-4, AI evolved from a preschooler level to a high schooler level. Supporting this leap is "OOM-scale scaling" along the following three axes.
① Computing (Hardware enhancement)
With the advent of more powerful GPU clusters and TPUs, models have become massive all at once.
From GPT-2 to GPT-4, roughly 100 times more computational resources were invested (2 OOM)
This physical expansion of computational power has dramatically boosted processing capability
② Algorithmic efficiency (Evolution of how the brain is used)
Technological innovation that allows for smarter learning even with the same amount of computation.
Representative examples: Improvements to Transformer architecture, Chain of Thought, etc.
Mechanisms that produce high-precision results with less data and less training time
This is easier to understand if we rephrase it as "the computer's IQ is increasing."
3. Removing Inhibitory Factors (Unlocking Potential)
AI models are in a "sealed capability state" when they are first trained. We unlock this through methods such as:
Reinforcement Learning from Human Feedback (RLHF)
Prompt Engineering
Integration with external tools (Tool-use agents)
This allows the AI to show its "true potential" that it was previously keeping quiet.
3. What happened in the 4 years from GPT-2 to GPT-4?
There was significant scaling across each of the three axes.
Item Scale OOM Conversion Computing Approx. 100x 2 OOM Algorithm Efficiency Approx. 10x 1 OOM Inhibitor Removal (Usability) Approx. 10x or more performance improvement 1 OOMTotal Approx. 1,000x 4 OOM
In just four years, there was an approx. 1,000x improvement in "effective performance."
4. So, what will happen in the next 4 years? — The world after GPT-5
If the scale-up continues at the same pace, by 2027, we can expect a further 3 to 4 OOM evolution.
In other words:
1,000 to 10,000 times the effective capability of today's GPT-4
Capable of passing GPQA (specialized exams) at the PhD level
The arrival of the **era where AI improves AI (recursive improvement)** also comes into view
This is no longer just a "chatbot," but an entity that becomes an "AI engineer colleague" or "research partner."
5. Summary: Understanding OOMs reveals the future
When viewed in OOM (Orders of Magnitude) units, AI evolution is a predictable linear trend
4 OOMs in the past four years, with equal or greater evolution expected in the future
Expansion driven by the three pillars of computing, algorithms, and removing bottlenecks
I encourage you to develop an "eye for counting OOMs" and try to anticipate the future of AI and societal changes.