NVIDIA’s Training Moat Is Splitting in Two: The Fabric Layer Still Holds, the CUDA Monopoly Does Not

Cerebras CEO Andrew Feldman estimates CUDA has lost ~70% of frontier training share in 24 months. The structural reason that number isn’t a death sentence for NVIDIA lives one layer deeper.

What Happened

On a recent episode of the Mad Podcast, Cerebras CEO Andrew Feldman made an observation worth sitting with: two years ago, essentially every state-of-the-art AI model was trained on NVIDIA GPUs inside a CUDA workflow. Today, Google’s Gemini trains on Google’s own TPUs, Anthropic’s Claude trains on AWS Trainium, and Google has begun making TPUs available to external customers — a first. Of the major US frontier labs, OpenAI still trains primarily on CUDA, though it is actively co-developing custom silicon. Feldman’s estimate is that CUDA has lost roughly 70% of frontier training share across that 24-month window. He is careful to present this as his estimate; the directional shift is not seriously in dispute.

The public record supports the direction. Google’s TPU program is well-documented going back to its 2016 first-generation announcement; the decision to use TPUs exclusively for Gemini training was confirmed through Google DeepMind communications and infrastructure disclosures. Amazon’s Trainium chips — designed specifically to reduce AWS’s dependence on NVIDIA for AI training workloads — have been publicly tied to Anthropic’s training infrastructure through AWS’s own partnership announcements. These are not experimental deployments; they are the primary substrate for two of the three most-cited frontier model families.

In Beyond NVIDIA’s Moat, Gennaro Cuofano of The Business Engineer argues that treating this as “CUDA losing share” misframes the structural question. The more useful analysis starts one layer down — at the fabric.

The Structural Read

Frontier model training is not a single-chip problem. A training calculation for a model at GPT-4 or Gemini 1.5 scale is far too large to fit on one accelerator, so it is split across thousands of chips that must constantly share state to remain consistent — what engineers call tensor model parallelism. The bottleneck is not raw compute; it is how fast information moves between those chips. If the interconnect fabric is slow or unreliable, additional compute is literally worthless.

This is why NVIDIA paid $6.9 billion for Mellanox in 2019. NVLink (chip-to-chip within a node) and InfiniBand (node-to-node across a data center) gave NVIDIA end-to-end ownership of the distributed training stack. CUDA held because the fabric held underneath it. When developers said “NVIDIA’s software ecosystem is a moat,” they were largely describing the compounding advantage of having the only interconnect fabric optimized for this specific workload at scale.

Google and Amazon have now built vertically-integrated alternatives: TPU pods with their own high-bandwidth interconnects (Google’s ICI fabric), and Trainium clusters with EFA (Elastic Fabric Adapter) networking. These are not NVIDIA-compatible — and that is the point. They are closed, optimized stacks that allow their owners to train frontier models without touching NVIDIA’s supply chain or licensing any NVIDIA software. The CUDA layer has been circumvented at the top two hyperscalers. What has not been circumvented is NVIDIA’s continuing dominance of the open-market, multi-tenant, and enterprise training infrastructure where no single operator has the scale to justify building a custom fabric.

The two-sided read, then: NVIDIA’s infrastructure position is secure for the foreseeable future. It remains the volume leader by a wide margin, with the only broadly-available distributed-compute ecosystem (CUDA plus NVLink plus InfiniBand) that third-party enterprises can actually buy and operate. What is loosening is long-term control at the frontier training layer specifically. The training tier has become structurally multi-silicon: TPU and Trainium as credible frontier-scale alternatives, a second tier including Cerebras wafer-scale chips, AMD’s MI-series, and Broadcom-designed custom ASICs for the next-largest labs, and a Chinese branch built around Huawei Ascend operating entirely outside US export controls.

There is one more layer to this: NVIDIA’s support for the Open-Weight Alliance reads differently once you understand the fabric logic. NVIDIA does not want AI value and pricing power to concentrate at the model layer — a world dominated by two or three proprietary closed models would allow those labs to extract margin from NVIDIA on hardware procurement. Vigorous competition at the model layer, including open-weight models that commoditize the model itself, keeps the industry structurally dependent on the infrastructure layer NVIDIA still dominates. The Open-Weight Alliance is partly a product of genuine open-source conviction; it is also entirely consistent with NVIDIA’s infrastructure lock-in strategy.

Three Implications