NVIDIA model removes the draft step from speculative decoding

NVIDIA has released Nemotron-Labs-Diffusion, a language model family that runs in autoregressive, diffusion and self-speculation modes from the same set of weights. The models are available in 3B, 8B, and 14B parameter sizes on Hugging Face under a license that permits commercial use, according to an NVIDIA-authored July 2026 arXiv preprint that has not yet been peer reviewed or independently replicated.

The main advance is self-speculation, which uses the diffusion pathway to draft candidate tokens and the autoregressive pathway to verify them, avoiding the separate draft model used by conventional speculative decoding systems. With an optional LoRA adapter of approximately 36 million parameters, the 8B instruct model averaged 6.82 accepted tokens per draft step on SPEED-Bench, compared with 2.75 for Eagle3 and 4.24 for Qwen3-9B-MTP.

NVIDIA says the 8B instruct model in autoregressive mode scored 63.61 percent average across a ten-task evaluation, while self-speculation with the LoRA adapter reached 62.81 percent average accuracy. The framework is supported through vLLM and SGLang, requires CUDA-capable GPU hardware, and can be loaded from the nvidia/Nemotron-Labs-Diffusion checkpoints on Hugging Face.