EuLLM builds a local-first stack for sovereign models

EuLLM is an Apache 2.0 open-source platform for running sovereign LLMs on local hardware or European infrastructure. Its Engine is available today as a Rust-based drop-in replacement for Ollama, exposing Ollama-compatible and OpenAI-compatible APIs while adding continuous batching, quantized KV cache support, multimodal input, daemon mode and a built-in browser chat.

The project emphasizes compliance-oriented operations for European users. It records every inference to a local audit trail, sends no analytics, crash reports or usage statistics, and includes controls for IP allowlists, API keys, browser origins and hardened web fetching. The README frames these features around GDPR needs and the EU AI Act (Regulation 2024/1689), while noting that compliance depends on the full deployed system, not a runtime alone.

Prebuilt binaries cover Linux, Windows, macOS and ARM configurations, with tested or community-validated CPU, CUDA, Metal and Vulkan builds. The roadmap extends beyond the Engine: Forge is intended to verticalize models through pruning, distillation, quantization and identity fine-tuning, while Hub is planned as an EU-hosted model registry with model cards and AI Act compliance cards. First domain models, including Italian law, German medicine and French finance variants, are planned for Q4 2026.