Envoy AI Gateway
Open-source Kubernetes-native gateway for GenAI and MCP traffic.
About
Envoy AI Gateway is an open-source AI gateway built on Envoy Proxy and Envoy Gateway, co-developed by Tetrate and Bloomberg and maintained under the envoyproxy GitHub organization with contributors from Bloomberg, Tetrate, Google, Tencent, and Netflix. It gives platform teams a single Kubernetes-native endpoint for app-to-LLM traffic with a unified OpenAI-compatible API.
Capabilities include multi-provider routing across 16+ AI providers, credential injection at the edge, token-based rate limiting, cross-backend failover, model name virtualization, and a dedicated MCPRoute resource for agent tool traffic. It is Apache 2.0 licensed, emits GenAI-semantic OpenTelemetry metrics, and an open 2026 proposal would move it under the Agentic AI Foundation.
What it can do
- Unified OpenAI-compatible API for 16+ providers
- Token-based rate limiting and provider failover
- Model virtualization and A/B routing
- MCP traffic routing via MCPRoute
- Apache 2.0 with GenAI OpenTelemetry metrics
Setup
Install on Kubernetes following the Envoy AI Gateway installation guide and define Gateway API routes for your providers.
Prerequisites
A Kubernetes cluster, Envoy Gateway, and LLM provider credentials.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 108; Trending score 76/100 from researcher assessment.
★ 1,700 GitHub stars
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.