Nvidia faces growing competition in inference

Nvidia GPUs are no longer essential for large-scale AI inference, with competing hardware already running major commercial services. Google serves Gemini on TPUs, Anthropic uses more than one million Amazon Trainium2 chips for Claude training and inference, and Meta operates hundreds of thousands of proprietary MTIA accelerators. AMD also supports commercially important inference workloads through its Instinct hardware and ROCm software.

Specialized processors are particularly attractive for repetitive, high-volume workloads that can be optimized around predictable computing and memory requirements. OpenAI’s Cerebras-powered GPT-5.6 Sol Ultrafast demonstrated up to 750 output tokens per second, reportedly as much as 14 times faster than standard processing. That speed advantage does not establish lower operating costs.

Nvidia remains commercially strong: quarterly data-center revenue reached ? billion, growing 117% year over year. Its CUDA ecosystem, broad model support and mature deployment tools remain advantages, especially for businesses frequently changing models. Public disclosures do not separate inference revenue sufficiently to establish precisely how its market share is changing.

Hardware decisions increasingly depend on total serving costs, including utilization, networking, memory and engineering effort, rather than processor prices alone. Planned systems combining Trainium for prompt processing with Cerebras for token generation suggest different architectures could handle separate stages of the same request. Alternatives are commercially viable, but comparable production data does not establish that they consistently cost less than Nvidia.