Rapid-MLX targets fast local AI on Apple Silicon

Rapid-MLX is a local AI engine built for Apple Silicon that runs on M-series Macs using native MLX kernels rather than llama.cpp fallback or a Metal shim. The project positions itself as a drop-in OpenAI and Anthropic API for local development, with performance described as 2-4× faster than Ollama and support for continuous batching, prompt caching and a TurboQuant K8V4 KV codec.

The tool can run as a terminal chat client or as an HTTP server at http://localhost:8000, exposing OpenAI-compatible routes such as /v1/chat/completions and Anthropic-compatible messages support. The default quick-start flow installs an isolated environment, selects a starter model based on Mac memory and downloads qwen3.5-4b-4bit on first use, with the base install described as text-only at (~460 MB).

Rapid-MLX emphasizes developer workflow integration, with Tier-1 support for 8 agent CLIs and 3 Python frameworks, including Codex CLI, Claude Code, OpenCode, Qwen Code, OpenHands, Hermes Agent, Aider, Kilo Code, LangChain, PydanticAI and smolagents. The repository also includes model management, benchmarking, troubleshooting, optional extras for vision, audio and embeddings, and opt-in anonymous telemetry that is off by default.