When the Network Became the Bottleneck

100G to 400G in the Data Center: Building the Fabric AI Actually Needs 

In Brief

• AI clusters move huge volumes of data between GPUs, and 100G is often no longer enough. 
• 400G is becoming the standard for AI fabrics, with 800G emerging for the densest clusters. 
• A non-blocking fabric matters as much as raw speed: every port running at full speed at once. 
• Under-build the fabric and you throttle the GPUs it feeds. Plan headroom for growth. 

For twenty years, network speed was a problem you solved once and forgot about. 

1G became 10G became 25G became 100G, and each jump landed comfortably ahead of what the servers could actually use. The network was rarely the thing holding you back. It was the reliable part, the background you did not have to think about. 

AI ended that quiet. 

For the first time in a long while, the network is struggling to keep up with the compute, not the other way round. A GPU cluster generates a torrent of traffic between its accelerators, and 100G, the speed that felt like plenty, is now often the ceiling. Building an AI-ready data center means rebuilding the fabric to a bandwidth most enterprise networks were never designed for. This is what that takes. 

Why Does AI Need So Much More Bandwidth? 

Because of where the traffic goes, and how much of it there is. 

A traditional application mostly talks north-south: users to servers, in and out of the data center. AI training is different. It talks east-west, GPU to GPU, constantly, as the accelerators exchange and combine their results thousands of times over the course of a run. That east-west traffic is enormous, and it all has to move fast, because the GPUs are waiting on it. A single link that cannot keep up slows the whole cluster. 

So AI did not just add more traffic. It changed the shape of it, and pointed a firehose of data sideways across the fabric, exactly where enterprise networks had the least headroom. 

100G, 400G, 800G: What Changed? 

The ceiling moved, twice, in a hurry. 

100G was the comfortable standard for demanding data center workloads. For AI clusters, it is increasingly the floor, or below it. 400G has become the emerging standard for AI back-end fabrics, giving the GPUs the bandwidth they need to stay synchronised, and 800G is already arriving for the densest, largest clusters, where even 400G is a constraint. To put the stakes in numbers: on a typical eight-node GPU cluster, a 400G AI fabric delivers on the order of 300 GB/s of effective bandwidth for the all-reduce operation that dominates training. Drop below that, and the accelerators spend their time waiting. 

The direction is one-way. As GPUs get faster and clusters get bigger, the bandwidth they demand rises with them, which is why the fabric you build today should be sized with the next generation in mind, not just this one. 

What Is a Non-Blocking Fabric, and Why Does AI Need One? 

Here is the concept that matters as much as the headline speed, and that many designs get wrong. 

A non-blocking fabric is one where every port can communicate with every other port at full speed, all at the same time, with no contention. The opposite is an oversubscribed fabric, where the network assumes not everyone will need full bandwidth at once, so it provides less than the sum of the ports, cheaper to build, and fine for ordinary workloads where traffic is bursty and uneven. 

AI breaks that assumption. In a training run, every GPU genuinely does want to talk to every other GPU at full speed, at the same moment. An oversubscribed fabric, faced with that, becomes a traffic jam, and the cluster slows to the pace of the congestion. This is why a non-blocking, or near-non-blocking, spine-and-leaf design is not a luxury for AI. It is the difference between a fabric that keeps the GPUs fed and one that quietly throttles them. Raw link speed with an oversubscribed design still starves the cluster. 

How Much Bandwidth Does Your AI Cluster Actually Need? 

Enough that the network is never the slowest part, which is a moving target set by your GPUs and your scale. 

The table gives a rough sense of where each speed fits. 

But the number on the link is only half the answer. The real question is whether the fabric, end to end, can carry full bandwidth between all the GPUs at once. That depends on the design, the spine-and-leaf topology, the oversubscription ratio, the switch buffers, as much as the port speed. Sizing an AI fabric properly means starting from the GPUs and the training pattern, then designing the whole path to keep them fed, rather than picking a link speed and hoping. 

The Whole Path Matters, Not Just the Links 

A fabric is a chain, and AI finds the weakest link. 

You can install 400G links and still bottleneck the cluster if the fabric is oversubscribed, the buffers are too small to absorb the bursts of an all-reduce, or the topology forces traffic through congested paths. Conversely, a well-designed 400G non-blocking fabric can outperform a faster but poorly-architected one. The lesson is that bandwidth is a system property, not a port speed. Building an AI fabric is a design exercise across the whole path, from the GPU to the leaf to the spine and back, not a shopping decision about which switches to buy. 

Do You Actually Need 400G or 800G? 

Not always, and it is worth being honest about it rather than over-building on reflex. 

If your AI footprint is modest, a smaller cluster, lighter workloads, a well-designed 100G fabric may still be enough, and paying for 800G would be waste. The trigger is the size of the cluster and the intensity of the GPU-to-GPU traffic, not the word "AI". That said, the trend is unmistakable, and the cost of retrofitting a faster fabric later is high, so the sensible middle path is to build for the workload you have with clear headroom for the one you are heading toward. Size for today, design for tomorrow. 

Building It for India 

For Indian enterprises, the fabric decision comes with the usual local realities. 

High-speed optics and switches at 400G and 800G carry a cost, availability and support dimension, and the skills to design a non-blocking AI fabric are specialised. The pragmatic approach is to build the fabric in stages, sized to the current cluster with headroom, using genuine, well-supported equipment, and to design it so it can scale to the next speed without a rebuild. And as with the rest of AI infrastructure, if the facility or the fabric cannot support the density, a purpose-built colocation environment is a valid path. The goal is the same everywhere: a fabric that never becomes the reason your GPUs sit idle. 

Don't Starve the Fabric 

The network is the least glamorous line in an AI budget and one of the easiest to under-build, which is exactly why it so often becomes the bottleneck. Sizing the fabric to the cluster, designing it non-blocking, and building it to scale to the next speed is the kind of engineering that decides whether your GPUs run at full potential or wait on the wire. 

Proactive Data Systems designs and builds AI-ready data center networking for Indian enterprises, including 100G, 400G and 800G non-blocking spine-and-leaf fabrics on Cisco, NVIDIA and other platforms. We are a Cisco Preferred Cloud and AI Partner, Dell Platinum Partner and NetApp Preferred Partner, with 35 years in enterprise IT, deep networking expertise, more than 1,500 organisations served, and a 24/7 service desk in India. To build the fabric your AI actually needs, you can ask Proactive for a data center networking assessment.

Sources 

AI fabric bandwidth trends (400G as the emerging AI training standard, 800G for the densest clusters) and effective all-reduce bandwidth on eight-node GPU clusters (~300 GB/s at 400G): AI networking and GPU cluster analyses, 2025–2026. Speeds and adoption move quickly; verify current figures before acting.