Arm gains ground in enterprise AI infrastructure

Arm is emerging as a more serious data center contender as enterprises evaluate AI inference, governance and infrastructure costs. Its shift from consumer and mobile technology into cloud servers gained momentum after Softbank acquired the company in 2016 and AWS adopted Arm for Graviton servers in 2018, helping broaden software and open source support.

Momentum accelerated in March 2026 with the launch of Arm’s AGI CPU for AI infrastructure. Arm said it has shipped more than 1.5 billion Neoverse processing cores, including 500 million in the last nine months. A planned IBM processor for the next System Z refresh in 2029 would run both System Z’s s390x and Arm Aarch64 instruction sets on the same cores, potentially reducing Linux porting work for mainframe environments.

Arm is also emphasizing power efficiency and density for on-premises deployments. In a 36-kilowatt air-cooled rack, Arm said its AGI CPU can support 30 1U servers and 8,160 CPU cores, compared with 17 2U servers and 4,352 CPU cores for a comparable x86 rack. Analysts noted that x86 vendors are improving efficiency and that many enterprises still depend on existing x86 estates.

Silicon choices are likely to remain mixed. In an Omdia survey of 400 respondents in North America, 76% agreed that silicon diversity is more important now than it was two years ago. Memory shortages may make CPU-based AI workloads more attractive, though high costs could keep some enterprise AI deployments in the cloud for now.