AI Data Center Networking Stack

Networking is one of the most confusing yet interesting parts of an AI data center buildout. This

article aims to help investors understand the topic better and make more informed decisions.

AI models don’t run on one chip. They run across hundreds, thousands, even hundreds of thousands of GPUs simultaneously - and how fast those GPUs can talk to each other determines how fast you can train and how cheaply you can serve inference. Networking is no longer a footnote in the data center bill of materials. It’s a first-class constraint.

At GTC 2025, Nvidia announced that moving from a 3-layer pluggable-optics network to a 2-layer co-packaged optics (CPO) network in a 400,000-GPU deployment could cut total cluster power by 12%, reducing transceiver power from 10% of compute resources down to just 1%. That’s the scale of what’s at stake.

This piece breaks down the full AI networking stack layer by layer - from silicon die to submarine cable - and maps exactly where each technology and company competes.