What Clogs Before GPUs: The 'Network' Bottleneck in the AI Era

While GPUs and massive models appear to be the stars of the AI boom, the final hurdle that actually 'turns computation into value' is the network.Arista Networks CEO Jayshree Ullalexplains this point using metaphors and a sense of on-the-ground reality. Based on our conversation, this article breaks down and organizes what AI-era networking is changing, where the bottlenecks lie, and what companies are betting on.

1. 'If chips are the engine, the network is the highway'

This is the most intuitive metaphor from our conversation.
'If a car's performance is 100 mph, but the road only allows for 30 mph, you cannot fully utilize the engine (=GPU)'—in other words, the efficiency of AI investment is increasingly determined by 'how things are connected' rather than the computing resources themselves.

What Arista Networks provides here is a high-performance, low-latency, and highly reliable switching/routing foundation for modern data centers. It handles the high-density traffic generated by users, servers, storage, and 'AI killer apps' behind the scenes. As she jokingly puts it, it is infrastructure that is hard to see from the outside, but if it stops, everyone notices.

2. AI networking is not just an 'extension of the cloud'

2-1. From front-end to back-end—the 'otherworldly traffic' increased by AI

The stars of the pre-cloud to cloud era were front-end networks connecting general CPU-centric processing (search, DB, general workloads). However, since generative AI, the situation has changed. The need to bundle hundreds, thousands, or even more accelerators (GPUs, etc.) has emerged, and traffic has become faster, lower-latency, and more prone to extreme peaks.

She refers to this as back-end/scale-up/scale-out networking, suggesting that traditional extensions are insufficient in terms of both metrics and design philosophy.

2-2. Standardizing the 'islands'—bringing Ethernet/IP to the AI back-end

Before AI, back-ends tended to be separate mechanisms ('islands') for each use case. Arista Networks introduced the standard-based (Ethernet/IP) philosophy that has also been used in the cloud. The context here is that this is being re-evaluated in a way that fits the 'scale of the AI era'.

3. Technical bottlenecks are 'power' and 'speed acceleration'

3-1. The biggest wall is power—the 'cannot build facilities' problem

The biggest constraint repeated in the conversation is, surprisingly, not semiconductors butpower. From GPUs and networks to cables and optics, power consumption has jumped far beyond past common sense. Moreover, she states that 'it can take 3 to 5 years to find the necessary power,' emphasizing the reality that AI infrastructure is bound not only by capital but also by location, grid capacity, and construction timelines.

3-2. Evolution cycle shortened from '5 years to 12–18 months'

The generational shift in network speed is also accelerating rapidly. While it used to take about 5 years to transition from 100M to 1G to 10G to 100G, now we are starting to incorporate the next steps almost annually, from 100G to 200/400 and beyond—the remark that 'we just reached 400G last year, but we are already thinking about 800G and 1.6T' succinctly expresses the pressure of demand.

4. The answer to 'Is it a bubble?'—investments where 'demand comes first'

There is a scene where the moderator compares this to the dot-com era (the growth period of Cisco Systems) and asks, 'Will this be over-investment again?' Her answer, simplified, is this:

• Back then, there was 'build it and they will come' style over-construction.

• Now it is the opposite: 'Demand is there first, and supply (power/facilities) cannot keep up.'

• It is also different in that it is being led by 'responsible large corporations' (in addition to Microsoft, Google, Meta, Oracle, etc., new forces like OpenAI and Anthropic have also emerged).

In short, I am not asserting that it is 'not a bubble,' but rather that its structure is at least different from a 'construction race based on empty demand.'

5. The Core of Management—Culture is 'Doing the Right Thing'

Between technical discussions, she repeatedly talks about culture. To summarize, the keyword is 'Do the right thing.' A symbolic example is the story of how, despite having the option to 'hide' a hardware quality issue with a software patch, she chose to explain it to customers and perform a full replacement at the company's expense. She even went so far as to say it could have led to bankruptcy, demonstrating that the decision to prioritize trust over short-term profit is what builds culture.

The same applies to hiring; she states, 'Ability is a prerequisite. Beyond that, we look at character, morality, values, and cultural fit,' which reflects a talent philosophy typical of a long-term infrastructure business.

6. The Next 3 Years—AI from 'Massive Centralization' to 'Distribution'

Finally, her outlook on the future. She is cautious, noting that her range is 'most accurate for 1 to 3 years,' but the direction is clear.
Current AI is 'mainframe-like (centered on massive clusters),' but it will next expand into a more distributed form. The point is that 'miniaturization' is difficult. Inference, inference optimization, and designing for results with limited resources will become important, and as a result, the outlook is that AI will move closer to 'more accessible computing environments, not just large-scale facilities.' Here too, the network will increase in importance as the 'blood vessel' that enables both centralization and distribution.

Recommended Articles

Next Big Wave (Growth Stocks, Seeds of Ideas, Deep Dives into Trends)