LiteLLM

Health v2 · maintenance only

Flags: no_license

Adoption not part of the score

61456 stars · 12439 forks observed · 2026-10-18

What it is AI-extracted, prompt v1, taxonomy v1, 2026-09-21, confidence 0.97

LiteLLM is an open‑source Python library and self‑hosted gateway that provides a single OpenAI‑compatible interface to call 100+ LLM providers. It includes routing, load‑balancing, spend tracking, and guardrail features for production LLM deployments.

Use cases

• Call any LLM provider (OpenAI, Anthropic, Vertex AI, Bedrock, etc.) using the same completion() function.
• Track and limit LLM spend per user or team with virtual API keys.
• Route requests to the cheapest or fastest model automatically with the built‑in auto‑router.
• Add safety guardrails and content moderation to LLM calls.
• Deploy a shared LLM gateway for multiple applications or micro‑services.

When to choose

• You need a unified API for many LLM vendors without rewriting code for each provider.
• You want self‑hosted control over authentication, logging, and cost management.
• Your organization requires routing, fallback, and guardrail policies for production LLM usage.
• You prefer a Python SDK with optional Docker‑based proxy for easy deployment.

When to avoid

• You only need a single provider and prefer the native SDK of that provider.
• Low‑latency, on‑device inference is required and a remote gateway would add overhead.
• Your stack is not Python‑centric and you cannot run the proxy in your environment.
• You need a fully managed commercial service with SLA guarantees.

Facets

library · maturity active

api-framework http-client http-server routing rate-limiting logging monitoring auth middleware cli artificial-intelligence large-language-models python cloud serverless cli cost-tracking guardrails llm-gateway virtual-keys admin-ui auto-router llmops retrieval-augmented-generation devops docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="BerriAI/litellm")