Tether moves BitNet fine-tuning onto edge devices

Tether released a fine-tuning framework for Microsoft’s BitNet b1.58 LLM designed to run across desktop GPUs and consumer-grade handheld devices. The framework uses a Vulkan-based GPU backend intended to make BitNet efficient on almost any graphics card, reducing reliance on NVIDIA processors and CUDA runtime.

The work builds on BitNet’s ternary quantization approach, which replaces floating-point multiplications with ternary (-1,0,1) integer addition and subtraction. Tether says BitNet-1B uses up to 77.8% less VRAM than FP16 Gemma 3-1B and 65.6% less VRAM than FP16 Qwen3-0.6B, while a 13 billion parameter BitNet LLM runs with 29% less GPU resources (VRAM) than a 4-bit quantized 4 billion parameter Qwen3 model.

The framework supports variants up to the 13-billion-parameter model on devices including the Samsung S25, Google Pixel 9, and iPhone 16. Users can fine-tune 125M-parameter BitNet models in ~10 minutes on a Samsung S25 for a biomedical dataset of ~300 documents (~18k tokens), while a 1B-parameter model takes 1 hour 18 mins on the same device and 1 hour 45 minutes on the iPhone 16.

Tether positions the release as a step toward local-first AI, with potential benefits for development costs, performance, and privacy because user data can remain on device rather than being uploaded to centralized servers.