Intel and SambaNova pitch modular inference architecture

Inference is becoming a central battleground for compute providers, with the industry increasingly shifting away from the idea that GPUs alone can dominate inference. Intel and SambaNova are now advancing a heterogeneous inference architecture that pairs SambaNova RDUs with Intel Xeon 6 CPUs, following a broader move toward disaggregated inference designs.

The configuration assigns GPUs to prefill workloads, Intel Xeon 6 processors to host, orchestration, and general-purpose tasks, and SambaNova RDUs to decode. SambaNova described the setup as a heterogeneous hardware solution that combines GPUs for prefill, Intel® Xeon® 6 processors as both host and “action” CPUs, and SambaNova RDUs for decode to deliver premium inference for the most demanding Agentic Artificial Intelligence applications. The partnership does not tie the design to a specific hyperscaler GPU option, leaving room for other accelerators or ASICs, although no detailed GPU-specific performance figures were provided.

SambaNova’s SN50 is the key accelerator in the design. The SN50 features 2TB of DDR5 memory, along with 64 GB HBM3 and 520 MB SRAM, and the company positions that combination as a way to deliver minimal latency, high throughput, and high capacity. SambaNova says the DRAM + SRAM + HBM combo creates ‘agentic caching’. Intel and SambaNova also say Xeon 6 CPUs were found to be well suited for “end‑to‑end coding agent workflows” compared to ARM options.

The pairing differs from NVIDIA’s approach by emphasizing a more modular and comparatively lower-commitment path to disaggregated inference infrastructure. The setup is framed as a practical option for hyperscalers that want rack-scale systems built around the “prefill + decode” split without committing to a tightly defined infrastructure stack. Intel’s role, at least for now, appears focused on providing the Xeon host CPU rather than deeper RDU integration.

The collaboration also reflects a broader relationship between the two companies. Intel’s CEO has participated in SambaNova’s latest funding round, and Lip-Bu is also an early investor in the company. Plans to acquire SambaNova were reportedly halted after a board disagreement, leaving Intel as a funding participant instead.