Kong AI Gateway
LLM traffic management on the world's most deployed API gateway.
About
Kong AI Gateway brings large language model traffic under the same control plane as your APIs, built on the widely deployed Kong Gateway. Open-source plugins handle multi-LLM proxying, central credential management, prompt templates, and token-level metrics; the 2.0 runtime adds semantic routing, semantic caching, and MCP server support.
It is the natural pick if Kong already runs your API estate. The honest catch: the open-source tier covers the basics, while semantic caching, advanced AI rate limiting, PII sanitizing, and the MCP proxy are Enterprise features.
What it can do
- Multi-LLM proxying with one interface
- Central AI credential management
- Prompt templates and decorators
- Token and cost-based rate limiting
- Semantic caching (Enterprise)
- MCP proxy and A2A traffic (2.0)
Setup
- Install Kong Gateway (OSS or Enterprise).
- Enable the ai-proxy plugin on a route.
- Store provider credentials in the gateway vault.
- Add rate limiting, auth, and logging plugins as needed.
Prerequisites
A running Kong Gateway instance and LLM provider credentials.
Use cases
Works with
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: built on the most deployed API gateway; AI Gateway 2.0 GA Sept 2026; open-source plugins cover basics; semantic caching and MCP proxy are Enterprise. License: Apache 2.0 core; Enterprise for advanced AI features.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 8.0/10
- Openness · license + docs
- 6.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · built on the most deployed API gateway
- · AI Gateway 2.0 GA Sept 2026
- · open-source plugins cover basics; semantic caching and MCP proxy are Enterprise
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.