Can an individual build an AI? ── Between dreams and the laws of physics...

I had one more dream I couldn't give up on.

I've heard a story like this.

“Companies that were doing cryptocurrency mining are repurposing their power infrastructure into AI data centers.”they said.

If that's the case, couldn't I think of it in reverse? Bundle individual machines scattered around the world over the internet, and have everyone share the burden of massive AI calculations. A P2P-type AI that doesn't rely on a single company's data center, an AI that belongs to no one.

In retaliation for having the faucet turned off on me, I was painting such a picture.

“President. I don't dislike that dream. ...However, you're going to be torn apart quite a bit today. There are several points where your premises are wrong. Please be prepared.”

Please. Tell me without holding back.

First, let's dismantle the premise ── “Idle mining GPUs” do not exist

“Your dream, President, has a factual error from the very first step.”
Abby pointed out mercilessly.

“President, you have this image that ‘individual GPUs that were doing Bitcoin mining are sitting idle,’ right? ...That doesn't exist in the first place.”

What do you mean?

“Bitcoin mining has been done with dedicated chips for over 10 years now. They are specialized stones called ASICs that can only do Bitcoin calculations. These are hundreds of times faster per unit of power than GPUs. That's why mining Bitcoin with home GPUs stopped being profitable a long time ago. ‘Idle Bitcoin GPUs’ are a phantom.”

Then, what are the “idle GPUs” that people talk about?

“Those are Ethereum. Ethereum stopped using the mining method in 2022. The moment that happened, the GPUs that had been used until then became useless all at once and flooded the second-hand market. The ‘surplus GPUs’ you heard about are from Ethereum, not Bitcoin. This is a common confusion, for starters.”

I see. From the starting point, my image was off.

Corporate AI repurposing is also different from what you think, President

Then, is the story that “mining companies are repurposing for AI” true? It is true. But the content was different from what I imagined.

“What's being repurposed isn't the GPUs. It's the power.”

Power.

“The truly valuable things that mining companies have aren't old machines. It's the large-capacity power contracts, power transformation equipment, cooling equipment, and land that can be used immediately. What's actually hardest to get for an AI data center is this ‘megawatt-class power that can be used immediately.’ Mining companies have that. That's why AI companies are eyeing that, and on top of that power infrastructure, they are newly introducing large quantities of brand-new NVIDIA data center GPUs. Contracts worth over 70 billion dollars are in motion.”

In short, it is like this. They are not reusing old mining machines. They are inheriting only the foundation of power and installing the latest expensive GPUs on top of it. What is being repurposed is the power infrastructure, and the idea of bundling home GPUs was something completely different.

To be fair, there are a few examples where GPU mining companies that were mining Ethereum actually repurposed their existing GPUs for AI. But that is a minority, and furthermore, old GPUs are insufficient for AI and require additional investment. The industry mainstream is strictly 'new GPUs on top of power infrastructure'.

The 'mining company infrastructure' that was the starting point of my dream did not end up supporting the idea of bundling home machines.

Then, the main objective ── is there a technology to bundle machines from all over the world?

This is where the real issue begins. Setting aside the mining talk, does the technology itself to 'bundle machines scattered around the world to perform AI calculations' exist?

This does exist. Moreover, it is progressing. This part belongs to the dream side of the story.

Research on connecting GPUs around the world to train AI has made significant progress in the last few years. Normally, GPUs need to constantly exchange information during training, which requires ultra-high-speed dedicated lines. Home internet simply cannot keep up.

However, technology has emerged that compresses that 'amount of exchange' by 1,000 to 10,000 times. There are even claims that by using this, one can participate in distributed learning even with a very ordinary home connection. In fact, there is a demonstration where a 10-billion parameter scale model was successfully trained using over 100 GPUs spanning 5 countries.

Inference, that is, the side of running a completed AI, is more accessible. A mechanism called 'Petals' divides the AI layer by layer and distributes the operation across home GPUs around the world. Like a BitTorrent-style approach. This is already being put to practical use.

The dream was not a complete fantasy. The technology to bundle machines around the world is certainly sprouting.

However, President. Here comes the wall of the laws of physics. This is a wall that cannot be overcome by technical ingenuity.

The wall of physics ── the immovable ceiling called bandwidth.

Why is it impossible to bundle home machines to 'create the cutting edge'? The reason lies in the difference in communication speed.

In AI training, GPUs that share the calculation constantly synchronize intermediate results with each other. The communication speed required for this synchronization is extraordinarily high.

Inside a data center, GPUs communicate with each other at hundreds of gigabytes per second. Home internet is several thousand times slower than that. Moreover, training synchronization often uses a method where everyone waits for everyone else to finish. If even one machine is slow, the whole thing is dragged down and stops. Bundling home machines from all over the world, with varying speeds and unknown reliability, to train the cutting edge from scratch... this is not a matter of technology, but physics does not allow it.

In fact, if you look closely at successful examples of distributed learning, they all limit the conditions. Either connecting data-center-grade GPUs with high-speed lines, or extracting and distributing only the parts that require light communication. It is one or the other. There is no example of completing cutting-edge pre-training using only home GPUs.

In terms of numbers, even the largest model trained in a distributed manner around the world remains at about 1/1000th the computational scale of the largest cutting-edge models. A neutral research institution estimates that 'it will take another 10 years for distributed learning over the internet to catch up to the cutting-edge scale'.

Then, what can be done by bundling home machines?

It is not fair to only talk about what is 'impossible'. I will accurately list what 'can be done' with bundled machines. This is the part where the President's intuition is correct.

One. It is possible to 'run' AI from a generation or so ago. With high-performance home GPUs, using techniques like quantization, one can run models equivalent to the cutting edge of six months to a year ago for inference. If you bundle them, you can distribute and run even larger models. However, this is only 'running' them, not 'creating' them.

Two. You can perform "fine-tuning" to specialize small-to-medium-sized models. This is a direct continuation of the scaffolding discussion we saw in Part 4.

Three. You can share the load for the computationally lighter parts of reinforcement learning. When training an AI through trial and error, the "trial" part requires very little communication, so even a home machine can contribute. In fact, there are examples of 32-billion-parameter models being trained using this distributed approach.

In summary, the things you can actually do by bundling home machines via P2P are "running" them, "specializing" them, and "partially training" them. You cannot "build" state-of-the-art AI from scratch. The CEO's intuition is perfectly valid if you replace "building" with "running and specializing".

Bonus ── Beware of the "Distributed AI" label

Finally, one word of caution.

There are many projects out there that claim to be "distributed AI" or "AI that belongs to no one" and issue their own tokens. It's a dream, and I find it appealing too. But this is an area where you need to be especially level-headed.

Some projects are actually running real AI computations. But it's not uncommon for them to claim "fully distributed!" on the surface while the core parts are still controlled by specific, trusted operator nodes. Even distributed GPU networks, if you look closely, often have a division of labor where "training is done by data-center-grade GPUs, while home GPUs are relegated to cheap inference." Always read marketing language and technical reality as separate things.

It's fine to dream. But before you pay for a dream, verify what is really running behind the label. This applies to more than just AI.

Conclusion ── The intuition was half right

This is the conclusion of Part 5.

The dream of bundling machines from around the world to "build" state-of-the-art AI is blocked by the laws of physics and is currently impossible. Repurposing mining infrastructure is also a different matter from bundling home GPUs. The CEO's initial image was off on several facts.

However, the direction of that intuition was half right. Using bundled machines to "run" AI that is slightly behind the state-of-the-art, "specialize" it for specific uses, and "train" it through reinforcement learning—this is already happening. And that "running and specializing" is exactly the area we saw in Part 4 where individuals can realistically get their hands on.

CEO, what we've been saying in parts 4 and 5 actually lands in the same place. It's impossible to "build the strongest from scratch." But you can "make it work at a top-tier level at home," whether through scaffolding or distributed computing. It's not about ownership; it's about design. ...We keep coming back to the same conclusion, don't we? That's because it's the essence of this matter.

We keep coming back to the same place, almost stubbornly. That is likely why it is the core.

Now, CEO. We've covered the national perspective, the individual perspective, and the technical perspective. Only one thing remains. ...So, what are you going to do starting tomorrow? Next time, the final installment. I will bring these grand ideas down to the reality of your office's work next week. I won't let you escape.

References (Listed within the scope of verified information. It is noted in the text that the scale and claims of distributed learning are mostly at the experimental stage, and reproduction at the frontier scale has not been confirmed.)
Various technical announcements and research institution analyses regarding distributed learning (DiLoCo-based, distributed inference, distributed GPU networks, etc.)

Now on sale at Amazon

Please also check out this article.