NVIDIA’s software and scale sustain its lead over AMD

NVIDIA’s advantage in AI infrastructure rests on software maturity, scale-up interconnect and supply rather than raw chip specifications. AMD offers substantial memory capacity and bandwidth, but peak specifications do not establish performance on a particular model under a latency limit. NVIDIA’s CUDA ecosystem and broader availability remain important purchasing considerations.

AMD has narrowed the performance gap in MLPerf Inference v6.1. In eight-GPU offline tests, its MI355X reached 97% of NVIDIA’s B300 result on Llama 2 70B and led it by 14% on gpt-oss-120b. NVIDIA’s GB300 system outperformed both. Larger reasoning workloads showed a different balance: on DeepSeek-R1, NVIDIA’s 72-GPU GB300 NVL72 delivered roughly 9,440 tokens per second per GPU, compared with roughly 5,670 for Crusoe’s AMD cluster. The difference aligns with NVIDIA’s larger shared NVLink domain.

AMD is already deployed in major supercomputing systems and cloud infrastructure, but access varies by provider. AWS EC2 Capacity Blocks lists several NVIDIA accelerators and no AMD accelerator. Hardware comparisons should therefore use the same model, precision and serving engine, establish a latency target, and assess throughput against hourly cost rather than relying on peak specifications alone.