Updated: 05 October 2026
AI training rewrote the rules of the data centre network. A cluster of GPUs learning a large model doesn't send traffic the way applications do; it sends enormous, synchronised bursts between thousands of processors that all need to talk at once, and it punishes any network that slows them down. The fabric became the bottleneck, and the industry is rebuilding it at 800 gigabits, around the GPU.
Two things are happening at once. Speeds are jumping from 400G to 800G and beyond. And Ethernet, long told it couldn't handle this job, is winning it back from InfiniBand.
For Indian data centres, hyperscale, sovereign-AI and the R&D centres of global firms, this shift is arriving now. Here's what's changing, and where Cisco's Nexus and Silicon One fit.
Because GPU communication is unlike anything a traditional data centre carries. When thousands of GPUs train a model together, they synchronise constantly, exchanging results in tight, simultaneous bursts across the network. This is the "back-end" or GPU fabric, and it's a different animal from the "front-end" network that connects the data centre to users and storage.
That back-end fabric has demands ordinary switching doesn't meet. It must be near-lossless, because the protocols carrying GPU traffic collapse if packets drop. It needs very low latency, because every microsecond of delay extends how long the training job takes. And it needs deep buffers to absorb the bursts when thousands of GPUs speak at once.
Get the fabric wrong and the GPUs, the most expensive thing in the building, sit idle waiting for the network. That's why the fabric, not the servers, is now the design problem.
Raw bandwidth demand. GPU clusters generate so much traffic that the speeds are climbing faster than anywhere else in networking. Most switch ports in AI clusters already run at 800 gigabits in 2025, and the industry expects the majority to move to 1.6 terabits by 2027 and 3.2 terabits by 2030.
That's a generational jump compressed into a few years. A data centre standardising on 100G a short while ago is now looking at 400G as the floor and 800G as the target for AI.
The pace matters for planning: the fabric you build for AI needs a clear path to the next speed, because the workloads will demand it sooner than any previous networking cycle would suggest.
The long-standing answer was InfiniBand; the emerging answer is Ethernet, and the gap is closing fast. InfiniBand still holds a latency edge, roughly 1 to 2 microseconds against Ethernet's 5 to 10, which matters at the extreme end of training (RoCE vs InfiniBand).
But Ethernet brings advantages InfiniBand can't match: it's open, multi-vendor, built on skills every network team already has, and proven at hyperscale, Meta runs 24,000-GPU training clusters on Ethernet with RoCEv2. Two developments made this credible. RoCEv2 carries GPU traffic over standard Ethernet, and the Ultra Ethernet Consortium, whose members include Cisco, AMD, Broadcom, Arista, Intel, Meta and Microsoft, is redesigning Ethernet transport specifically for AI, with packet spray, multi-path delivery and modern congestion control.
So the honest framing isn't that one is simply better. InfiniBand leads on raw latency; Ethernet leads on openness, ecosystem, familiarity and cost, and is closing the performance gap. For most enterprises and many hyperscalers, open Ethernet is the pragmatic choice, which is why the whole industry is investing in it.
Not just speed, tuning. An 800G port on its own doesn't make an AI fabric; the fabric has to be engineered for lossless, low-latency, bursty GPU traffic:
| Requirement | Why AI Needs It | How It's Delivered |
|---|---|---|
| High bandwidth (400 → 800G → 1.6T) | Massive GPU-to-GPU collective traffic | 800G Ethernet ports and optics |
| Lossless transport | RoCEv2 performance collapses on packet loss | PFC for flow control, ECN with DCQCN for congestion |
| Deep buffers | Thousands of GPUs burst simultaneously | 32–64MB or larger packet buffers |
| Low latency | Cuts job completion time | Congestion-aware load balancing |
| Multi-path and congestion control | Even, efficient fabric use | Ultra Ethernet features like packet spray |
The single measure that ties these together is job completion time, how long the training run takes. Every fabric feature exists to keep the GPUs fed so the job finishes sooner. That's the metric AI networking is judged on.
At the centre of Cisco's AI data centre story. The Nexus 9000 Series, powered by the Cisco Silicon One G200 ASIC, delivers high-density 800G fabrics built for AI/ML leaf-spine designs, for example the Nexus 9364E-SG2, a 2RU switch with 64 ports of 800 gigabit Ethernet (Cisco Nexus 9000 for AI).
Silicon One is the thread connecting Cisco's AI networking: one ASIC architecture spanning routing and switching, with the high radix, deep buffers and congestion management AI fabrics need. On top of the silicon, Cisco makes the Nexus 9000 UEC-ready, complying with the Ultra Ethernet baseline of PFC, ECN and multiple traffic classes, and adds AI-specific intelligence, congestion-aware load balancing and fault detection to cut job completion time, plus GPU-aware telemetry that correlates network behaviour with the AI workload itself.
The practical upshot: Cisco offers a single family covering the back-end GPU fabric, the front-end, storage and management networks, on open Ethernet, with the AI tuning built in. For a data centre team that already knows Cisco, that's an AI fabric without learning an entirely separate technology stack.
India is building AI infrastructure quickly, and the fabric decision is landing now. Hyperscale operators, colocation providers, the sovereign-AI push, and the R&D and GCC centres of global firms are all standing up GPU clusters on Indian soil, and every one of them faces the 800G, back-end-fabric question.
Ethernet's openness is especially valuable here. Indian data centre teams have deep Ethernet skills and a mature Cisco ecosystem, so an open 800G Ethernet fabric, rather than a specialist InfiniBand island, means building AI infrastructure with people, partners and tools the market already has. That lowers the barrier to standing up AI capacity locally.
The trend is clear: as India's AI data centre capacity grows, the network under it is moving to 800G Ethernet fabrics, and Cisco's Nexus and Silicon One are among the platforms that fabric will be built on.
Designing an 800G AI fabric, the topology, the lossless tuning, the buffer and congestion strategy, the front-end and back-end split, is specialised work where a mistake leaves expensive GPUs idle. Proactive Data Systems, a Cisco Preferred Partner with 35 years of experience and more than 1,500 customers, designs data centre and AI-ready networks on Cisco Nexus, from the GPU back-end fabric to the front-end and storage networks, tuned for the workloads they carry. If you're planning AI or high-performance infrastructure in an Indian data centre, ask us to design the fabric under it.
Keshav Kumar Director & Head, IT, Proactive Data Systems
We'll get back to you shortly.
