theopenco/llmgateway

Health v2 · maintenance only

Flags: no_license

Adoption not part of the score

1576 stars · 175 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

LLM Gateway is an open-source API gateway for large language models that sits between applications and 40+ LLM providers (OpenAI, Anthropic, Google, etc.) with an OpenAI-compatible unified API. It provides multi-provider routing with fallbacks, API key management, cost and usage analytics, caching, guardrails, and can be used hosted or self-hosted via Docker.

Use cases

• route llm requests across multiple providers with one api
• track token usage and costs for openai and anthropic calls
• switch from openai to claude without changing code
• self-host an llm proxy gateway with docker
• add fallbacks and automatic provider failover for llm calls
• detect prompt injection and redact pii before hitting llms
• manage all my provider api keys in one dashboard
• compare model latency and cost across providers

When to choose

• you need a unified OpenAI-compatible endpoint across many LLM providers
• you want cost, token, and latency analytics for LLM usage
• you need provider failover, routing, or rate limiting for LLM traffic
• you want guardrails like PII redaction and prompt-injection blocking
• you want to self-host an LLM gateway on your own infrastructure

When to avoid

• you only call a single LLM provider and don't need routing or analytics
• you need to self-host model weights or run local inference — this proxies APIs, it doesn't serve models
• you need a lightweight in-process SDK rather than a gateway service
• you require an OSI-approved open-source license (license is custom/unclear)

Facets

service · maturity active

api-gateway llm-inference analytics monitoring rate-limiting caching security middleware self-hosted large-language-models artificial-intelligence apis developer-tools analytics security self-hosted self-hosted cloud llm-gateway llm-proxy openai-compatible multi-provider byok guardrails cost-tracking api-key-management observability fallback-routing docker web-server nodejs typescript

Member repositories

For agents

markdown · JSON · MCP: product_card(name="theopenco/llmgateway")