DeepSeek releases DSpark to accelerate LLM inference

DeepSeek has released DSpark, an MIT-Licensed framework for speeding up large language model inference without changing the target model’s intended output. The system uses speculative decoding, where a lighter draft component proposes likely tokens and the larger model verifies them, reducing the bottleneck created by token-by-token generation.

In live production tests, DeepSeek reported aggregate throughput gains of 51% for DeepSeek-V4-Flash at an 80-token-per-second-per-user service target and 52% for DeepSeek-V4-Pro at a 35-token-per-second-per-user target. At matched system capacity, it reported per-user generation speedups of 60% to 85% for V4-Flash and 57% to 78% for V4-Pro over its prior MTP-1 baseline.

DSpark combines semi-autoregressive generation with confidence-scheduled verification, aiming to draft more coherent token blocks and avoid wasting compute on low-confidence guesses. DeepSeek also released DeepSpec, a codebase for training and evaluating draft modules, with checkpoints for open model families including Qwen and Gemma. The approach is most practical for teams that control model weights and serving infrastructure, while hosted API users would depend on vendors to implement similar optimizations internally.