Nvidia readies Grace CPU for AI systems beyond x86

Nvidia announced Grace, an upcoming data center CPU designed to work closely with Nvidia GPUs as AI models become larger and more complex. The company described the chip as the result of “10,000 engineering years of work” and said it will use a future Arm Neoverse processor architecture that has not yet been revealed.

Grace is intended to reduce bottlenecks between CPUs and GPU accelerators in current x86-based AI systems. Nvidia said models have grown from 100s of millions of parameters to 175 billion parameters in just the last two years, while Google Brain researchers have claimed training a 1.6-trillion-parameter model using Google’s TPU processors. A Grace-powered system is expected to train a 1 trillion-parameter NLP model 10 times faster than today’s fastest Nvidia DGX computers.

The CPU uses Nvidia’s NVLink interconnect, which the company says provides a 900Gbps connection between processors, along with an LPDDR5x memory subsystem that doubles DDR4 bandwidth and provides 10x DDR4’s energy efficiency. Nvidia expects the first Grace chips to be available in early 2023, with Grace-based supercomputers planned for the Swiss National Supercomputing Centre and Los Alamos National Laboratory.