# envoyproxy/ai-gateway

Manages Unified Access to Generative AI Services built on Envoy Gateway

Repository: https://github.com/envoyproxy/ai-gateway
Canonical: https://ross.abutalabs.com/products/ai-gateway
Homepage: https://aigateway.envoyproxy.io
Language: Go
License: Apache-2.0
License Family: permissive
Topics: ai, ai-gateway, api-gateway, cncf, kubernetes, llm, inference
Last push: 2026-08-26T19:27:02+00:00

## Health v2 (maintenance only)
Score: 88/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 98, longevity 48
- inputs: {"age_days": 681, "days_push": 7, "days_rel": 12, "gap_med": 28.0, "n_releases_24m": 15}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1961, forks 344 (observed 2026-08-28T04:05:59.435895+00:00)

## What it is
Envoy AI Gateway is an open-source, Kubernetes-native gateway built on Envoy Gateway that manages traffic from application clients to Generative AI services. It provides unified routing across 16+ LLM providers and self-hosted models, with authentication, rate limiting, failover, MCP gateway support, and cost/usage observability.

## Use cases
- route LLM traffic to multiple providers like OpenAI, Anthropic, and AWS Bedrock through one gateway
- add authentication and rate limiting to OpenAI API calls from my apps
- set up automatic failover between LLM providers
- track token usage and cost per team or user for GenAI services
- expose self-hosted model serving clusters with fine-grained access control
- put per-user identity on MCP tool calls at the gateway layer
- unify access to generative AI services in a Kubernetes cluster

## When to choose
- you run workloads on Kubernetes and want an Envoy-based, CNCF-aligned AI gateway
- you need multi-provider LLM routing with failover, auth, and usage limiting in one control plane
- you need enterprise observability and cost analytics for GenAI traffic
- you serve both cloud LLM providers and self-hosted inference endpoints

## When to avoid
- you need a simple single-provider proxy without gateway features
- you are not running Kubernetes or cannot operate Envoy Gateway
- you want a lightweight client-side SDK rather than infrastructure

## Facets
- artifact type: service
- maturity: stable
- function: api-gateway, routing, auth, rate-limiting, middleware, monitoring, llm-inference
- domain: large-language-models, artificial-intelligence, apis, infrastructure-as-code, cloud-computing
- platform: go, self-hosted, cloud
- tags: ai-gateway, envoy, envoy-gateway, llm-routing, genai, mcp-gateway, cncf, observability, failover, cost-management, devops, kubernetes, docker

## Member repositories
- envoyproxy/ai-gateway (main) score 88

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:59.435895+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:05:37.781734+00:00, confidence not recorded.
  - readme: https://github.com/envoyproxy/ai-gateway (fetched 2026-08-28T04:05:59.435895+00:00, sha 4c1a84b3688e)
  - homepage: https://aigateway.envoyproxy.io (fetched 2026-08-29T10:45:45.749333+00:00, sha 2d490ad904dc)
  - site_page: https://aigateway.envoyproxy.io/docs (fetched 2026-08-29T10:45:45.758710+00:00, sha 8df56e293533)
  - site_page: https://aigateway.envoyproxy.io/docs/next (fetched 2026-08-29T10:45:45.760921+00:00, sha 26c0e69f1bbb)
  - site_page: https://aigateway.envoyproxy.io/docs/1.0 (fetched 2026-08-29T10:45:45.762927+00:00, sha b04f0601b3a2)
  - site_page: https://aigateway.envoyproxy.io/docs/0.7 (fetched 2026-08-29T10:45:45.764660+00:00, sha c7947d852785)
  - site_page: https://aigateway.envoyproxy.io/docs/0.6 (fetched 2026-08-29T10:45:45.766357+00:00, sha d15cd8a6b8a0)
  - site_page: https://aigateway.envoyproxy.io/docs/0.5 (fetched 2026-08-29T10:45:45.768037+00:00, sha 8d770b1624cc)
  - site_page: https://aigateway.envoyproxy.io/docs/0.4 (fetched 2026-08-29T10:45:45.769666+00:00, sha 1f2c1d7c3ebc)
  - site_page: https://aigateway.envoyproxy.io/docs/0.3 (fetched 2026-08-29T10:45:45.771208+00:00, sha 7bc8acd24940)
- Data as of 2026-08-30T08:39:29.467469+00:00.