# BerriAI/litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

Repository: https://github.com/BerriAI/litellm
Canonical: https://ross.abutalabs.com/products/litellm
Homepage: https://docs.litellm.ai/docs/
Language: Python
License: NOASSERTION
License Family: other
Topics: anthropic, langchain, llm, llmops, openai, ai-gateway, azure-openai, bedrock, gateway, openai-proxy, vertex-ai, llm-gateway, litellm, mcp-gateway, rust, rust-ai
Last push: 2026-08-27T00:41:22+00:00

## Health v2 (maintenance only)
Score: 91/100 (v2, computed 2026-09-03T02:39:23.370411+00:00)
- activity 99, release rhythm 87, longevity 81
- inputs: {"age_days": 1134, "days_push": 7, "days_rel": 11, "gap_med": 0, "n_releases_24m": 880}
- flags: no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 57340, forks 10918 (observed 2026-08-28T04:12:18.517415+00:00)

## What it is
LiteLLM is an open-source AI gateway and Python SDK that provides a unified OpenAI-format interface to 100+ LLM providers (OpenAI, Anthropic, Bedrock, Azure, Vertex AI, vLLM, and more). It can be used as an in-code SDK or deployed as a self-hosted proxy server with virtual keys, cost tracking, budgets, load balancing, fallbacks, guardrails, and an admin UI.

## Use cases
- call openai anthropic and bedrock models through one unified api
- self-host an llm gateway with api keys and spend tracking
- set budgets and rate limits per team for llm usage
- load balance and failover between multiple llm providers
- proxy claude code or cursor through a single llm endpoint
- add guardrails and pii masking to llm requests
- track llm costs and token spend across an organization
- route requests to cheaper models based on prompt complexity

## When to choose
- you use multiple LLM providers and want one OpenAI-compatible interface
- you need centralized key management, budgets, and spend tracking for LLM access across teams
- you want retries, fallbacks, and load balancing across model deployments
- you need guardrails, PII masking, or observability hooks in front of LLM calls

## When to avoid
- you call a single LLM provider directly and don't need an abstraction layer
- you need a fully permissive license - the repo has a custom license with enterprise-gated features
- you want a lightweight client without running a proxy server or extra dependency

## Facets
- artifact type: library
- maturity: active
- function: llm-inference, api-framework, http-server, proxy, rate-limiting, caching, monitoring, logging, sdk, mcp
- domain: large-language-models, artificial-intelligence, developer-tools, self-hosted, apis
- platform: python, rust, self-hosted, cross-platform, cli
- tags: ai-gateway, llm-gateway, openai-compatible, llm-proxy, load-balancing, cost-tracking, guardrails, virtual-keys, spend-tracking, fallbacks, multi-provider, mcp-gateway, ai-agents, docker

## Member repositories
- BerriAI/litellm (main) score 91

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:12:18.517415+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T16:18:38.451750+00:00, confidence not recorded.
  - readme: https://github.com/BerriAI/litellm (fetched 2026-08-28T04:12:18.517415+00:00, sha d186682f4c11)
  - homepage: https://docs.litellm.ai/docs/ (fetched 2026-08-28T18:03:22.351212+00:00, sha 70a0666f233f)
  - site_page: https://docs.litellm.ai/docs/learn (fetched 2026-08-28T18:03:22.360202+00:00, sha f2254b1272a6)
  - site_page: https://docs.litellm.ai/docs/integrations/ (fetched 2026-08-28T18:03:22.362001+00:00, sha 8f337a818244)
  - site_page: https://docs.litellm.ai/docs/enterprise (fetched 2026-08-28T18:03:22.364560+00:00, sha 8017f01ae8bb)
  - site_page: https://docs.litellm.ai/release_notes (fetched 2026-08-28T18:03:22.367207+00:00, sha 1fc6bd8bb48e)
  - site_page: https://docs.litellm.ai/docs/learn/autorouter_cli (fetched 2026-08-28T18:03:22.369192+00:00, sha f37a8c0d5651)
  - site_page: https://docs.litellm.ai/docs/simple_proxy (fetched 2026-08-28T18:03:22.372387+00:00, sha 393224b2dacd)
  - site_page: https://docs.litellm.ai/docs/supported_endpoints (fetched 2026-08-28T18:03:22.373989+00:00, sha 8aeee86e8103)
  - site_page: https://models.litellm.ai (fetched 2026-08-28T18:03:22.370907+00:00, sha e7abcda2298f)
- Data as of 2026-08-30T08:39:29.467469+00:00.