Developer runs small language model on $10 microcontroller

A developer using the name slvDev has run a 28.9-million-parameter language model fully offline on an ESP32-S3 microcontroller, generating TinyStories-style text at 9.88 tokens per second with nothing leaving the chip. The esp32-ai project was published on GitHub under an MIT license in late July 2026, showcased on the Better Stack YouTube channel, and has gathered over 3,600 stars and more than 470 forks.

The project works around severe memory limits by combining four-bit quantization with a memory layout that leaves much of the model in flash storage. At 16-bit precision, the model would need nearly 60 MB of RAM, while the ESP32-S3 has 512 KB of SRAM and 8 MB of PSRAM. Quantization reduces the footprint to 14.9MB, and about 25 million parameters sit in the board’s 16 MB of flash using Per-Layer Embeddings borrowed from Google’s Gemma 3n.

The result is technically notable but narrow. The model can produce short, mostly coherent stories, but it cannot answer questions, follow instructions, write code, or provide factual knowledge. slvDev says the constraint comes from the reasoning portion of the model, listed as ~4M parameters, while the same repository also includes Barista, a model limited to espresso questions.