Subquadratic looks past sparse attention

Subquadratic is trying to turn skepticism around its long-context model into an enterprise pitch. After launching with broad claims but no public benchmarks, the company published a model card for SubQ 1.1 Small, added third-party verification from Appen and began giving design partners access to the model.

SubQ 1.1 Small uses Subquadratic Sparse Attention, which the company says scales close to linearly with context length by avoiding redundant token-to-token comparisons. The model scored near-perfect on needle-in-a-haystack retrieval from 1 million tokens out to 12 million, reached 99.12 percent on Nvidia’s RULER test across a 128,000-token context, and landed at 85.4 on GPQA Diamond versus 87.5 for Sonnet 4.6. On LiveCodeBench, it scored 89.7, below Opus 4.8 and GPT-5.5 but slightly ahead of Sonnet 4.6.

The company says the main advantage is efficiency. At 1 million tokens, SubQ uses 64.5x less compute than dense attention and runs 56x faster than FlashAttention-2 on a single attention layer, with attention compute reduction close to 1,000x at the 12-million-token window. Subquadratic says the model has fewer than 100 billion parameters and is aimed at enterprises handling large data-processing workloads.

Whedon pushed back on defining Subquadratic as a sparse-attention company, saying the team is also working on “zero attention” architectures inspired by world models and long-range state. A next model is expected to be a mid-tier system rather than a frontier-class release, with broader availability still undecided.