NVDA 145.98 ▼3.30%GOOGL 272.66 ▼3.46%MSFT 498.30 ▼1.78%AMD 489.19 ▼7.72%INTC 55.72 ▲1.92%TSMC 352.09 ▼2.90%AMZN 257.37 ▼3.87%META 596.22 ▼3.27%AAPL 349.21 ▼0.92%PLTR 153.68 ▼1.91%
Markets at last close

Sony · Apps

Garlic LLM brings offline AI chat to jailbroken PS4

·1 min read

Garlic LLM, an open-source application created by developer Leonardo Alegria, runs an AI model with 9 billion parameters locally on a jailbroken PlayStation 4. It supports text chat and voice conversations without an internet connection. The software requires GoldHEN to run unofficial applications, and Alegria has confirmed compatibility with firmware version 11.52.

Without APIs or machine learning libraries for general-purpose GPU computing in unofficial applications, Alegria built a system that assembles low-level instructions directly for the console’s AMD GCN GPU. Garlic LLM uses Alibaba’s Qwen3.5-9B model, with weights quantized to 3-bit and 4-bit. The model and optimization data occupy approximately 4.5 GiB. Combined with speculative decoding, which generates and verifies multiple tokens together, these optimizations deliver 31-39 tokens per second for ordinary text and up to 43 tokens per second for code. The context window supports 16,000 tokens.

Additional features include speech recognition, speech synthesis, a reasoning mode and execution of AI-generated Python code on the console. Voice conversations reduce text generation speed to approximately 22 tokens per second because the voice model also consumes memory. The source code is available on GitHub under the GPLv3 license, while a free installation package of approximately 5.7GB, including the AI model, is available from Hugging Face.

Originally reported by gigazine.netRead the source →
Related coverage
All Sony news →