Nvidia has released Switchyard, a Rust-based proxy and library for LLM traffic. It routes requests between providers, translates between OpenAI and Anthropic API formats, and logs metrics. Currently in pre-release and experimental stage, not recommended for production. It supports three operation modes: launcher for coding agents, standalone proxy server, and embeddable library. Routing algorithms include pass-through, random, LLM classifier-based, and staged router. The tool accepts OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages as input formats, and can route to backends like vLLM, NVIDIA NIM, or Ollama. Licensed under Apache 2.0, documentation available on Nvidia's website.
Nvidia Unveils Switchyard: A Rust Proxy for LLM Traffic
Nvidia has rolled out Switchyard, a Rust-based proxy and library built to manage large language model (LLM) traffic. It sends requests across different providers, converts between OpenAI and Anthropic API formats, and records operational metrics. The point is simple: teams keep having to rewrite code agents when backend model API formats don’t line up. Switchyard goes after that mess.
Pre-release and Experimental
Right now, Switchyard is in pre-release and marked experimental. Evaluation only. Nvidia does not recommend it for production use. It comes with three operating modes: a launcher for coding agents, a standalone proxy server, and a library that can be embedded into Rust applications.
Routing Algorithms and Input Formats
Its routing options include pass-through, random, LLM classifier-based routing, and staged router. On the input side, the tool takes three formats: OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages. It can then route requests to backends like vLLM, NVIDIA NIM, or Ollama.
Open Source and Documentation
The project is open source under the Apache 2.0 license, and the documentation is available on Nvidia's official website.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
