NVIDIA-NeMo/Switchyard
Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization. observed · 2026-08-28
Health v2 · maintenance only
80/100
- Activity 99
- Release rhythm 97
- Longevity 7
Flags: young
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: 20.0
- age_days: 106
- days_rel: 23
- days_push: 7
- n_releases_24m: 3
Adoption not part of the score
2490 stars · 211 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded
Switchyard is a Rust proxy and library that routes LLM traffic across models and providers while translating between OpenAI Chat, Anthropic Messages, and OpenAI Responses API formats. It supports pluggable routing algorithms (random, LLM-as-classifier, signal-driven stage routing) and exposes Prometheus metrics for requests, latency, tokens, and routing overhead.
Use cases
- route coding agents like Claude Code or Codex to open-source models served by vLLM or Ollama
- translate Anthropic Messages API requests to OpenAI-compatible endpoints
- A/B benchmark multiple LLM backends behind one endpoint
- optimize cost and performance with signal-driven model routing
- embed LLM routing logic in a Rust application via a library
- monitor LLM proxy traffic with Prometheus metrics
When to choose
- you need OpenAI/Anthropic API compatibility across heterogeneous model backends
- you want to experiment with routing algorithms for LLM traffic in Rust
- you are benchmarking or spreading traffic across several models
When to avoid
- you need a production-hardened gateway today (pre-alpha, APIs expected to change)
- you want a turnkey server without writing or tuning routing configuration
- your stack is not Rust and you only need the library path
Facets
service · maturity experimental
proxy routing llm-inference monitoring middleware large-language-models artificial-intelligence developer-tools apis rust cli self-hosted cross-platform llm-gateway api-translation openai-compatible anthropic-api model-routing benchmarking prometheus-metrics vllm ollama
1 source
- readme: https://github.com/NVIDIA-NeMo/Switchyard · fetched 2026-08-28 · 84818161fc13
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| NVIDIA-NeMo/Switchyard | main | 80 |
For agents
markdown · JSON · MCP: product_card(name="NVIDIA-NeMo/Switchyard")
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem