BerriAI/litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM] observed · 2026-08-28
Health v2 · maintenance only
91/100
- Activity 99
- Release rhythm 87
- Longevity 81
Flags: no_license
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: 0
- age_days: 1134
- days_rel: 11
- days_push: 7
- n_releases_24m: 880
Adoption not part of the score
57340 stars · 10918 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded
LiteLLM is an open-source AI gateway and Python SDK that provides a unified OpenAI-format interface to 100+ LLM providers (OpenAI, Anthropic, Bedrock, Azure, Vertex AI, vLLM, and more). It can be used as an in-code SDK or deployed as a self-hosted proxy server with virtual keys, cost tracking, budgets, load balancing, fallbacks, guardrails, and an admin UI.
Use cases
- call openai anthropic and bedrock models through one unified api
- self-host an llm gateway with api keys and spend tracking
- set budgets and rate limits per team for llm usage
- load balance and failover between multiple llm providers
- proxy claude code or cursor through a single llm endpoint
- add guardrails and pii masking to llm requests
- track llm costs and token spend across an organization
- route requests to cheaper models based on prompt complexity
When to choose
- you use multiple LLM providers and want one OpenAI-compatible interface
- you need centralized key management, budgets, and spend tracking for LLM access across teams
- you want retries, fallbacks, and load balancing across model deployments
- you need guardrails, PII masking, or observability hooks in front of LLM calls
When to avoid
- you call a single LLM provider directly and don't need an abstraction layer
- you need a fully permissive license - the repo has a custom license with enterprise-gated features
- you want a lightweight client without running a proxy server or extra dependency
Facets
library · maturity active
llm-inference api-framework http-server proxy rate-limiting caching monitoring logging sdk mcp large-language-models artificial-intelligence developer-tools self-hosted apis python rust self-hosted cross-platform cli ai-gateway llm-gateway openai-compatible llm-proxy load-balancing cost-tracking guardrails virtual-keys spend-tracking fallbacks multi-provider mcp-gateway ai-agents docker
10 sources
- readme: https://github.com/BerriAI/litellm · fetched 2026-08-28 · d186682f4c11
- homepage: https://docs.litellm.ai/docs/ · fetched 2026-08-28 · 70a0666f233f
- site_page: https://docs.litellm.ai/docs/learn · fetched 2026-08-28 · f2254b1272a6
- site_page: https://docs.litellm.ai/docs/integrations/ · fetched 2026-08-28 · 8f337a818244
- site_page: https://docs.litellm.ai/docs/enterprise · fetched 2026-08-28 · 8017f01ae8bb
- site_page: https://docs.litellm.ai/release_notes · fetched 2026-08-28 · 1fc6bd8bb48e
- site_page: https://docs.litellm.ai/docs/learn/autorouter_cli · fetched 2026-08-28 · f37a8c0d5651
- site_page: https://docs.litellm.ai/docs/simple_proxy · fetched 2026-08-28 · 393224b2dacd
- site_page: https://docs.litellm.ai/docs/supported_endpoints · fetched 2026-08-28 · 8aeee86e8103
- site_page: https://models.litellm.ai · fetched 2026-08-28 · e7abcda2298f
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| BerriAI/litellm | main | 91 |
For agents
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem