How OpenAI Handles '800 Million Users a Week'—Behind the Scenes of Multi-Model Strategy, RFT, Agents, and Usage-Based Billing
OpenAI's ChatGPT has become a product used weekly by about 10% of the world's population (800 million people). Once you reach this scale, the idea that 'you just need to build one huge, smart model' no longer works. Fine-grained specialization of models based on internal and external needs, incorporating company-specific data, and keeping them running as agents for long periods—such 'operational-first design' becomes crucial.
In this article, using statements from Sherman Woo, who leads OpenAI's developer platform, as a guide, we will organize the collapse of the one-size-fits-all model myth, the roles of model specialization and fine-tuning, approaches to agents and open source, and even usage-based pricing design.
1. The Collapse of the 'Single Model is Enough' Worldview
1-1. The 'One Model to Rule Them All' Idea Existed Even Within OpenAI
Sherman recalls that 'until two or three years ago, even within OpenAI, we were discussing things under the premise that one model would rule them all.' At the time, it was thought that once AGI was achieved, a single model would handle every use case, making fine customization or switching between models unnecessary.
However, in reality, a group of models with different areas of expertise—such as ChatGPT-5, o3, 4o, code-specialized models, and reasoning-specialized models—are appearing one after another. As each has begun to be used as a separate 'first-string' option, the composition of 'taking over the world with one universal model' is crumbling.
1-2. Why Model Specialization Has Become Inevitable
The biggest reason for the advancement of model specialization is the breadth and depth of use cases. The required capabilities—code completion, long-form summarization, project planning, search integration, long-term reasoning, etc.—are vastly different. Rather than cramming everything into one model, it is easier to achieve higher cost-efficiency and quality by preparing models optimized for each.
Furthermore, companies have a strong need to leverage their 'treasure trove' of internal logs and business data. Sherman states, 'Companies want to use their own data to build the world's best models,' emphasizing that the model itself has become something that is 'specifically requested.' As users and developers get used to the 'personality' and 'behavior' of a certain model and build products based on that premise, simple switching becomes increasingly difficult.
2. The Destructive Power of Fine-Tuning and Reinforcement Fine-Tuning
2-1. Conventional SFT Was Limited to 'Tone Adjustment'
The supervised fine-tuning (SFT) that OpenAI initially provided was mainly used for instruction following and adjusting writing style. However, this was insufficient for essentially utilizing the vast domain data held by companies, and it had the limitation that 'even if you think you are customizing, you are essentially just changing the personality a little.'
2-2. The Era of Creating 'Use-Case Specific SOTA Models' with RFT
What changed this situation is Reinforcement Fine-Tuning (RFT). Sherman says, 'With RFT, we enable actual RL to run, and from that data, we can create world-class models for specific purposes.' The decisive difference is that the goal is not just to change the tone of the response, but to push the performance itself in a specific task to the limit.
Companies can use their own expert labels and user behavior logs to create 'in-house SOTA models' optimized for their own operations. Moreover, OpenAI has begun to offer incentives such as discounts on inference fees and free training to companies that share data for RFT, and a new relationship that could be called a 'barter of data and GPUs' is emerging.
3. From Prompt Engineering to Context Engineering
Sherman points out that the view on Prompt Engineering has changed significantly over the past few years. It was once believed that 'once models become smart enough, there will be no need to devise prompts.' However, in reality, although the model's instruction-following capability has improved significantly, the essence still lies in the design of 'what to pass to the model.'
He explains this as Context Engineering. Which tools to call at what timing, which data sources to incorporate into the context and when, how to build the logic for RAG and tool calling—the 'design of peripheral systems' including these factors determines the results just as much as model performance. It is no longer at the level of 'fine-tuning a single sentence in a prompt,' but the work of designing the environment in which the AI operates that has become important.
4. Agents are not a 'third category,' but an operational form of intelligence
4-1. What is an agent?
Sherman defines agents as 'AI that acts on behalf of humans and executes tasks over a long time horizon.' The key point here is that they do not treat agents as a 'third category' separate from APIs or products. At OpenAI, each product—such as ChatGPT, API, Sora, and CodeX—is considering how to implement 'agentic behavior' within its own context. Agents are positioned not as a 'separate product,' but as a form of how intelligence is delivered and presented.
4-2. Why is a node-based Agent Builder required?
The Agent Builder announced at DevDay drew reactions such as 'it feels like no-code and is too restrictive' or 'it should be left to AGI with more freedom.' However, according to Sherman, real-world business involves not only high-freedom work like that of software engineers, but also a vast amount of work where 'the value lies in being faithful to Standard Operating Procedures (SOPs).'
In many fields such as customer support, sales, marketing, and operations in regulated industries, 'not deviating from established procedures' is more important than 'unbridled creativity.' That is precisely why a mechanism like Agent Builder, which fixes procedures using flows and nodes and then runs models on top of them, is necessary. Agents are designed not as 'AI that can do anything,' but as 'AI that works for long periods within a defined framework.'
5. Coexistence strategy for API (horizontal) and ChatGPT (vertical)
OpenAI is a very rare company that simultaneously develops both 'horizontal' developer APIs and 'vertical' products like ChatGPT and Sora. Usually, when a company's own app and an external service built on its API offer similar services, channel conflict or cannibalization occurs.
However, Sherman explains, 'OpenAI's mission is to distribute the benefits of AI widely, and to do that, we need not only our own apps but also an ecosystem through APIs.' ChatGPT is growing its user base at an unprecedented speed, while the API is further expanding the total number of end users it can reach by being embedded in diverse industries and services. Internally, there are indeed situations where 'our own apps and startups on the API compete,' but even assuming that, they prioritize the idea that 'the growth of the entire ecosystem is what matters.'
6. Open source does not kill APIs: Open Weights strategy
Open-weight models like GPT-OS released by OpenAI might at first glance seem to create 'competition for their own API business.' However, Sherman denies this concern. In reality, he says, there is almost no overlap between companies that use open models and those that choose fully managed APIs.
Inference infrastructure for large models is extremely difficult, and ensuring high-speed, stable throughput requires a sophisticated team and infrastructure investment. For many companies, APIs are easier to manage in terms of total cost and risk than self-hosted inference. Also, the core of OpenAI's revenue lies in its 'largest closed models,' which are not provided as open weights in the first place.
As a result, open source plays a role in pioneering new use cases and developer segments, broadening the base of the market. Far from stealing API revenue, it acts as a catalyst to increase the 'total number of companies adopting AI.'
7. Usage-Based Pricing will become an irreversible standard
Sherman also speaks candidly about pricing design. OpenAI's API is basically usage-based (token billing), and he says that internally, it is set based on 'cost plus a reasonable margin.'
The words of Rockset founder Venkat, whom he quotes, are symbolic.
'Once an industry experiences usage-based pricing, it does not return to the original fixed licensing.'
Just as the transition from on-premise to subscription once created a massive industrial shift, the transition to usage-based is considered an irreversible 'one-way ratchet.' Outcome-based pricing is also being considered, but because performance metrics differ by industry and automating evaluation is difficult, it is highly likely that usage-based will remain the mainstream in the short term.
Conclusion: Multi-model × Customization × Context Design × Agents
Synthesizing Sherman's points, OpenAI's vision can be summarized as follows:
A world where a single giant model does not dominate, but rather a portfolio of purpose-specific models exists side-by-side
A world where specialized models created through RFT that leverage corporate data function as 'local SOTA' in each field
A world where Context Engineering and agent design, rather than prompts, become the source of competitive advantage
A world where open source and APIs do not conflict, but coexist as the two wheels of market expansion
A world where billing converges into a new infrastructure economy centered on usage-based models
Discussions surrounding AI often tend to be abstract, but looking at OpenAI's practical decision-making reveals a vivid design philosophy regarding 'how to distribute intelligence, how to monetize it, and how to nurture an ecosystem.'
As you consider how to integrate AI into your own company, the design choices and compromises OpenAI is making at a 'scale of 800 million weekly users' will serve as an extremely important reference case.