# NVIDIA-NeMo/Switchyard

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Repository: https://github.com/NVIDIA-NeMo/Switchyard
Canonical: https://ross.abutalabs.com/products/switchyard
Language: Rust
License: Apache-2.0
License Family: permissive
Last push: 2026-08-26T21:55:49+00:00

## Health v2 (maintenance only)
Score: 80/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 97, longevity 7
- inputs: {"age_days": 106, "days_push": 7, "days_rel": 23, "gap_med": 20.0, "n_releases_24m": 3}
- flags: young
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2490, forks 211 (observed 2026-08-28T04:06:55.057534+00:00)

## What it is
Switchyard is a Rust proxy and library that routes LLM traffic across models and providers while translating between OpenAI Chat, Anthropic Messages, and OpenAI Responses API formats. It supports pluggable routing algorithms (random, LLM-as-classifier, signal-driven stage routing) and exposes Prometheus metrics for requests, latency, tokens, and routing overhead.

## Use cases
- route coding agents like Claude Code or Codex to open-source models served by vLLM or Ollama
- translate Anthropic Messages API requests to OpenAI-compatible endpoints
- A/B benchmark multiple LLM backends behind one endpoint
- optimize cost and performance with signal-driven model routing
- embed LLM routing logic in a Rust application via a library
- monitor LLM proxy traffic with Prometheus metrics

## When to choose
- you need OpenAI/Anthropic API compatibility across heterogeneous model backends
- you want to experiment with routing algorithms for LLM traffic in Rust
- you are benchmarking or spreading traffic across several models

## When to avoid
- you need a production-hardened gateway today (pre-alpha, APIs expected to change)
- you want a turnkey server without writing or tuning routing configuration
- your stack is not Rust and you only need the library path

## Facets
- artifact type: service
- maturity: experimental
- function: proxy, routing, llm-inference, monitoring, middleware
- domain: large-language-models, artificial-intelligence, developer-tools, apis
- platform: rust, cli, self-hosted, cross-platform
- tags: llm-gateway, api-translation, openai-compatible, anthropic-api, model-routing, benchmarking, prometheus-metrics, vllm, ollama

## Member repositories
- NVIDIA-NeMo/Switchyard (main) score 80

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:55.057534+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:27:48.551506+00:00, confidence not recorded.
  - readme: https://github.com/NVIDIA-NeMo/Switchyard (fetched 2026-08-28T04:06:55.057534+00:00, sha 84818161fc13)
- Data as of 2026-08-30T08:39:29.467469+00:00.