Strata pushes large local models onto gaming PCs

Strata, a GitHub runtime from developer Niko1221, lets a quantized version of Qwen3.8-Flash-Next run locally on high-end consumer gaming PCs. The software uses the model’s MoE design by loading the full model into system RAM while caching only frequently used expert sub-models in GPU VRAM, then combines that approach with speculative decoding.

Qwen3.8-Flash-Next has 24,576 small expert sub-models, but only 10 are required for each generated token. Strata requires an NVIDIA or AMD GPU with at least 12GB of VRAM, 32GB or more of RAM, 80GB or more of storage, and Windows 10/11 or Linux. The model download is approximately 70GB, and startup loads 35-55GB into RAM, making the PC sluggish for the first 1-3 minutes.

Reported performance reached 94 tokens per second for the 2-bit “Q2_0” version and 53 tokens per second for the 3-bit “IQ3_S” version on an NVIDIA GeForce RTX 5070 system with a Ryzen 5 7600 and 64GB of RAM. An AMD Radeon RX 9070 XT system with a Ryzen 9 3900X and 47GB of RAM achieved 60 tokens per second with “Q2_0”.

Strata also exposes web interaction plus OpenAI- and Anthropic-compatible localhost APIs for existing chat frontends and coding tools. Lower-bit quantization can reduce quality: “IQ2_XS” has shown repeated-token loops, and Hackaday verification found factual confusion, making parity with large commercial models unrealistic.