Spotify and NVIDIA cut recommendation inference costs

Spotify’s collaboration with NVIDIA has reduced infrastructure costs for generative recommendations by improving TensorRT LLM. For the 1.7B variant of Qwen3, Spotify reported up to 54% cost reduction for beam search workloads and up to 7% cost reduction for scoring workloads. The work brings together capabilities previously spread across separate inference engines.

Spotify introduced LLM-powered podcast recommendations in January 2026. Its initial deployment used vLLM for beam search, which explores potential recommendation sequences, but CPU bottlenecks and redundant GPU work limited hardware utilization. Spotify also used SGLang to efficiently score multiple candidates within a single request, rather than processing each separately.

The NVIDIA collaboration added support for returning multiple beam search sequences, custom models, user embeddings and multi-item scoring. Spotify built a customized Qwen3 extension that projects existing user vectors into the model’s embedding space as soft tokens, enabling personalized recommendations without lengthy text histories. TensorRT LLM’s sampler and scheduling improvements also increased GPU utilization. Multi-item scoring currently requires a custom serving layer for efficient tokenization and request batching.

Spotify plans to expand generative recommendations across its catalog and listening surfaces while controlling latency and infrastructure spending. Further work with NVIDIA will focus on quantization and NVIDIA Blackwell-specific optimizations.