# maximhq/bifrost

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

Repository: https://github.com/maximhq/bifrost
Canonical: https://ross.abutalabs.com/products/bifrost
Homepage: https://www.getmaxim.ai/bifrost
Language: Go
License: Apache-2.0
License Family: permissive
Topics: gateway, llm, llm-gateway, mcp-client, mcp-server, guardrails, mcp-gateway, ai-gateway, gateway-services, generative-ai, llm-cost, llm-observability, llmops, load-balancing, token-management, model-router
Last push: 2026-08-26T21:02:37+00:00

## Health v2 (maintenance only)
Score: 83/100 (v2, computed 2026-09-03T02:39:23.370411+00:00)
- activity 99, release rhythm 87, longevity 38
- inputs: {"age_days": 532, "days_push": 7, "days_rel": 7, "gap_med": 0, "n_releases_24m": 2006}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 7587, forks 1112 (observed 2026-08-28T04:10:01.754204+00:00)

## What it is
Bifrost is a high-performance, open-source AI gateway written in Go that unifies access to 23+ LLM providers (OpenAI, Anthropic, AWS Bedrock, Google Vertex, etc.) behind a single OpenAI-compatible API. It provides automatic failover, load balancing, semantic caching, guardrails, MCP gateway support, observability, and budget/rate management with very low latency overhead.

## Use cases
- route llm requests across multiple providers with automatic failover
- drop-in openai-compatible proxy for openai anthropic bedrock and vertex
- track llm costs and set budgets per team or api key
- cache llm responses semantically to reduce spend
- add guardrails to block unsafe model outputs
- expose mcp tools to agents through a gateway
- monitor llm latency and throughput with otel metrics
- load balance across provider api keys at high rps

## When to choose
- you need a fast self-hosted llm gateway with sub-millisecond overhead
- you want one openai-compatible endpoint for many model providers
- you need failover, load balancing, and semantic caching out of the box
- you want built-in observability, budgets, and rate limits for llm usage
- you need an mcp gateway or guardrails in front of your models

## When to avoid
- you only call a single provider directly and need no routing or governance
- you want a python-native gateway you can extend in python code
- you need cluster mode, adaptive load balancing, or SSO without an enterprise license

## Facets
- artifact type: service
- maturity: active
- function: api-gateway, llm-inference, caching, load-testing, monitoring, rate-limiting, middleware, mcp, agent-framework
- domain: large-language-models, artificial-intelligence, apis, self-hosted, monitoring
- platform: self-hosted, go, cli, cross-platform
- tags: llm-gateway, openai-compatible, model-routing, semantic-caching, guardrails, fallbacks, observability, token-management, drop-in-replacement, enterprise, ai-agents, devops, docker, kubernetes, web-server

## Member repositories
- maximhq/bifrost (main) score 83

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:10:01.754204+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:36:53.718130+00:00, confidence not recorded.
  - readme: https://github.com/maximhq/bifrost (fetched 2026-08-28T04:10:01.754204+00:00, sha 7fd9967c4e90)
  - homepage: https://www.getmaxim.ai/bifrost (fetched 2026-08-29T08:32:14.062954+00:00, sha f0da9d3c8998)
  - site_page: https://www.getmaxim.ai/pricing (fetched 2026-08-29T08:32:14.072253+00:00, sha d4f4098bebeb)
- Data as of 2026-08-30T08:39:29.467469+00:00.