LiteLLM updates span routing, agent tools and release security

LiteLLM is expanding its open-source AI gateway with provider integrations, decision models and tools for managing agent workloads. OpenCode Go support lets users route requests through their subscriptions while preserving sessions and tracking estimated usage. Tokenization support provides a shared request format across vLLM and Watsonx, while decision-model integrations support routing, triage and guardrail checks using probabilities rather than generated text.

Security disclosures include the compromised v1.107.0 release, which LiteLLM marked as contained. A subsequent account traced the attack to a master key committed to a public repository in May, leading to three stolen GitHub tokens. Separate security and stability efforts include stronger release gates, isolated CI/CD environments, vulnerability fixes and regression testing.

Performance work targets gateway overhead, database access and prompt handling. LiteLLM reported cutting median time to first byte from 553 ms to 35 ms in a long-prompt benchmark by removing unnecessary prompt-cache routing work. Routing improvements include session affinity for self-hosted models, context-aware escalation and production shadow evaluations. Agent tooling includes the self-hosted Moyai coding agent, interchangeable agent harnesses and LiteLLM Lens, which investigates repeated failures across agent traces.