NVIDIA NeMo Switchyard routes agent requests across models

NVIDIA NeMo Switchyard helps AI agents select models for individual requests, combining efficient models with more capable alternatives. It supports routing across providers while preserving native OpenAI and Anthropic API compatibility. Developers retain control of model clients, credentials and service operation, while Switchyard supplies routing decisions through its Rust core, libsy.

Routing can assess the task, use recent tool activity and execution outcomes, or combine those approaches. The default Auto preset uses execution-based routing without a classifier call. Other options include escalation, advisor and sub-agent strategies, although support varies by integration. Developers can access Switchyard through OpenRouter, use gateway integrations with LiteLLM or NeMo Relay, or embed it in their own agent infrastructure.

Recent additions include routing between Sonnet and Opus in Claude Code, Mercury Decide and Jev as task judges, and TypeScript bindings with Pi integration. The project recommends evaluating the complete agent and routing configuration against a single-model baseline, warning that cheaper individual calls do not necessarily reduce the cost of successfully completing a task.

Deployment maturity remains a constraint. The embedded routing core is in beta, while supporting client and runner components are alpha. The standalone server is intended only for demonstrations and local evaluation, not supported production deployment. APIs, configuration and routing behavior may change between releases, so developers are advised to pin their integration version.