maximhq/bifrost

Health v2 · maintenance only

Adoption not part of the score

7587 stars · 1112 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Bifrost is a high-performance, open-source AI gateway written in Go that unifies access to 23+ LLM providers (OpenAI, Anthropic, AWS Bedrock, Google Vertex, etc.) behind a single OpenAI-compatible API. It provides automatic failover, load balancing, semantic caching, guardrails, MCP gateway support, observability, and budget/rate management with very low latency overhead.

Use cases

• route llm requests across multiple providers with automatic failover
• drop-in openai-compatible proxy for openai anthropic bedrock and vertex
• track llm costs and set budgets per team or api key
• cache llm responses semantically to reduce spend
• add guardrails to block unsafe model outputs
• expose mcp tools to agents through a gateway
• monitor llm latency and throughput with otel metrics
• load balance across provider api keys at high rps

When to choose

• you need a fast self-hosted llm gateway with sub-millisecond overhead
• you want one openai-compatible endpoint for many model providers
• you need failover, load balancing, and semantic caching out of the box
• you want built-in observability, budgets, and rate limits for llm usage
• you need an mcp gateway or guardrails in front of your models

When to avoid

• you only call a single provider directly and need no routing or governance
• you want a python-native gateway you can extend in python code
• you need cluster mode, adaptive load balancing, or SSO without an enterprise license

Facets

service · maturity active

api-gateway llm-inference caching load-testing monitoring rate-limiting middleware mcp agent-framework large-language-models artificial-intelligence apis self-hosted monitoring self-hosted go cli cross-platform llm-gateway openai-compatible model-routing semantic-caching guardrails fallbacks observability token-management drop-in-replacement enterprise ai-agents devops docker kubernetes web-server

Member repositories

For agents

markdown · JSON · MCP: product_card(name="maximhq/bifrost")