Google updates LiteRT-LM for on-device LLM apps

Google’s LiteRT-LM is a production-ready orchestration layer for running LLMs with LiteRT across Android, iOS, Web, Desktop, and IoT hardware including Raspberry Pi. The runtime targets on-device execution with GPU and NPU acceleration, support for vision and audio inputs, and function calling for agentic workflows using constrained decoding.

The v0.15.0 release adds an Apple Foundation framework adapter that integrates Apple’s backend with LiteRT-LM for text and multimodal execution. It also introduces centralized config.json support for the CLI, allowing global defaults and per-model settings such as specialized hardware backends without lengthy command-line flags.

Google is positioning the stack for a wide range of app developers, with APIs for CLI, Python, Kotlin, Swift, JavaScript, Flutter, and C++. Web updates include Gemma 4 Web Support (up to 31B), a new use_autosized_ringbuffers option for memory efficiency, and AutoToolChat capability.

The Google AI Edge Gallery showcases offline on-device generative AI using LiteRT-LM on Android and iOS, with source code available on GitHub. Supported models include Gemma, Llama, Phi-4, Qwen and more, alongside published benchmark tables for devices such as Samsung S26 Ultra, iPhone 17 Pro, MacBook Pro M4 Max, Windows systems, Linux hardware, and Raspberry Pi 5 16GB.