Open source Managed or self-host AGPL-3.0

LLM Gateway

Name: LLM Gateway
Author: Polar Lights LLC

AGPL-licensed OpenAI-compatible gateway available as one self-hosted Docker image or a hosted service with credit fees.

Built by Polar Lights LLC · US company

Compare with others Estimate cost Visit site ↗

Access at a glance

Whether this product can work for you at all, before features matter: who pays the model bill, where it can run, and which of your existing API calls keep working. Every value is the vendor’s own claim, linked to the page it came from.

LLM Gateway calls itself "an open-source API gateway for Large Language Models (LLMs)" acting as "middleware between your applications and various LLM providers", routing requests, managing provider keys, and tracking token usage and cost (LLM Gateway overview, 2026-04-21). Its site frames it as "One API for 40+ providers" (llmgateway.io).

Who pays the model bill

Your keys or their credits

You can start on their credits and move to your own provider accounts later.

And mixable per request: credentialSource is either byok ("your own provider key" — the provider bills the customer and the attempt "is not deducted from your credits") or platform ("an LLM Gateway credential" — deducted from the credit balance); in hybrid mode a BYOK attempt can fail and fall back to an LLM Gateway credential within the same request (LLM Gateway routing, 2026-08-27). "Bring your own keys — free forever" (llmgateway.io).

Merchant of record: Split by mode: with BYOK "you pay your providers directly" and there is "no platform fee"; on credits, LLM Gateway invoices, charging "a flat 5% when you buy credits — no seats, no minimums, no subscription" (plus a +1.5% international card fee), with volume discounts and negotiated terms on Enterprise (LLM Gateway pricing; LLM Gateway routing, 2026-08-27).

Key handling: Provider keys are added on a provider-keys page and can be labelled, rotated across multiple keys per provider, and are surfaced per attempt as providerKeyId / providerKeyLabel with an opaque apiKeyHash fingerprint; candidate keys appear as routingMetadata.eligibleProviderKeys (omitted in credits mode), and the dashboard badges each call "your key" or "LLM Gateway key" (LLM Gateway routing, 2026-08-27). Caller keys are project-scoped (LLM Gateway quickstart, 2026-06-22). Storage mechanism (vault/KMS): n.a. (LLM Gateway overview, 2026-04-21). On hosted, calls appear in the dashboard with latency, cost and provider breakdown; self-hosting means "your LLM traffic never leaves your infrastructure if desired" (LLM Gateway overview, 2026-04-21). The vendor states it is "SOC 2 Type II Certified" (llmgateway.io).

Where it can run

2 of 5 shapes documented

Vendor-hosted
Self-host
Your VPC
On-premise
Air-gapped

Dimmed shapes are not documented by the vendor, which is not the same as unsupported.

hosted SaaS ("Hosted Version: For immediate use without setup, visit llmgateway.io to create an account and get an API key") and self-host ("Deploy LLM Gateway on your own infrastructure for complete control over your data and configuration") (LLM Gateway overview, 2026-04-21). Hybrid/VPC, on-prem, air-gapped: n.a.

Self-hosting is offered under AGPLv3 and is free: "LLM Gateway is open source under AGPLv3, so you can self-host the gateway for free — or use the hosted platform and pay only the credit fee" (LLM Gateway pricing); "the self-hosted version… ensures your LLM traffic never leaves your infrastructure if desired" (LLM Gateway overview, 2026-04-21). Self-hosted deployments are tuned with environment variables such as EXPLORATION_RATE, PREFERRED_PROVIDER_TTL, PREFERRED_PROVIDER_UPTIME_THRESHOLD and PREFERRED_PROVIDER_SCORE_MARGIN; per-project routing customization from the dashboard is gated to the Enterprise plan (LLM Gateway routing, 2026-08-27). The dedicated self-hosting doc page failed to load (see gaps).

API surfaces your code can keep using

2 of 7 documented, 1 partial

OpenAI chatPOST /v1/chat/completionsYes
POST /v1/chat/completions under https://api.llmgateway.io/v1/… (LLM Gateway chat completions, 2026-06-08; LLM Gateway quickstart, 2026-06-22).
Anthropic messagesPOST /v1/messagesYes
"the Anthropic Messages endpoint (/v1/messages)" is documented and, per the quickstart, can reach "any LLM model" (LLM Gateway routing, 2026-08-27; LLM Gateway quickstart, 2026-06-22).
OpenAI ResponsesPOST /v1/responsesNot documented
n.a.
EmbeddingsPOST /v1/embeddingsNot documented
n.a. (not documented on the pages fetched).
ImagesPOST /v1/images/generationsPartly
Image-generation models are supported in routing (an "image price" weight "replaces the price weight for image-generation models"), but no image endpoint path is documented (LLM Gateway routing, 2026-08-27).
AudioPOST /v1/audio/*Not documented
n.a.
Batch jobsPOST /v1/batchesNot documented
n.a.

An asterisk marks a qualified verdict: support that is indirect (SDK compatibility or provider passthrough rather than a native endpoint), or a gap that is narrower or wider than the label suggests. Read the note before porting.

Drop-in base-URL swap: "Just swap out your current LLM provider URL with the LLM Gateway API endpoint" — "a single drop-in endpoint" at https://api.llmgateway.io/v1/…, "maintains compatibility with the OpenAI API format" (LLM Gateway overview, 2026-04-21; quickstart, 2026-06-22). Streaming: "pass stream: true to any request — Gateway will proxy the event stream unchanged" (LLM Gateway quickstart, 2026-06-22). Caveats: the latency routing factor applies only to streaming requests (non-streaming falls back to uptime selection), and a non-auto tool_choice may be downgraded to auto when no eligible mapping can honor it (LLM Gateway routing, 2026-08-27). Auth is bearer-token with project-scoped API keys (LLM Gateway chat completions, 2026-06-08).

How much it reaches

Models~200

Upstream providers~40

"Route requests across 200+ models" (llmgateway.io, undated); docs state no number (LLM Gateway routing, 2026-08-27).

"40+ providers" (llmgateway.io, undated). Docs pages state no number, referring only to "multiple LLM providers" (LLM Gateway overview, 2026-04-21).

Whose models: All third-party routed; the gateway "operates between applications and various LLM providers" and does not state owning infrastructure (LLM Gateway overview, 2026-04-21). Region pinning is supported (provider/model:region) (LLM Gateway routing, 2026-08-27).

Your own endpoints: Partially documented: "custom providers" exist, with keys "scoped by their own catalogue", and a customer-specific upstream can be routed (setting the Alibaba Model Studio workspace ID "routes through the customer's own endpoint"); no vLLM/Ollama/SageMaker/private-URL registration flow is documented (LLM Gateway routing, 2026-08-27).

How it behaves in production

What happens when an upstream model is slow, wrong, or down — and what you can see and stop while it happens. Reliability features are recorded as where you configure them, not whether the vendor lists them, because almost every product here lists all of them.

1 of 6 reachable from codenothing documented on the request pathno documented export

When something goes wrong

Each row says where the knob is, not whether the feature is on the marketing page. A control you can only reach by hand in someone else’s dashboard cannot be reviewed or version-controlled.

Request timeoutNot documented
The vendor does not document this, so any behaviour you observe today is unversioned and may change.
not_documented across all four fetched pages (docs home, self-host, self-host/docker, repo) (LLM Gateway docs).
RetriesNot documented
not_documented; default count and backoff: n.a. (LLM Gateway docs).
Fallback to another modelNot documented
not_documented as a configurable key. Behavior is asserted without configuration: the gateway "fails over to a healthy provider when one errors" — ordering, weighting and key name are all unstated (LLM Gateway docs).
Load balancingIn config
Changing it means editing configuration and shipping it, so behaviour is uniform across traffic until you redeploy.
config_file at the credential level only: "LLM Gateway supports multiple API keys per provider for load balancing — provide comma-separated values" via environment variables. No key name for the policy and no weights (LLM Gateway Docker self-host).
Upstream health trackingNot documented
Only the same "fails over to a healthy provider when one errors" statement; no health check, breaker or ejection key (LLM Gateway docs).
Cross-region failoverNot documented
Self-hosting on AWS/GCP/Azure and any Kubernetes cluster is documented, but no cross-region failover (LLM Gateway self-host).

Architecture is documented in operational terms (stateless services, PostgreSQL as source of truth, Redis for cache and worker queue, AGPLv3 license), but the reliability control surface is essentially undocumented — a developer choosing on reliability would have to read the source (LLM Gateway self-host, LLM Gateway docs).

How fast the hop is

Interpreted proxy

Runs on an interpreted or JIT runtime (Lua, Python, Node). Overhead is higher than a compiled binary and more sensitive to concurrency, though a Lua-on-nginx proxy and a Python one are far apart.

interpreted_proxy (JS runtime). Repo is TypeScript 95.7%; components are apps/ui (Vite + React), apps/api (Hono backend) and apps/gateway (theopenco/llmgateway).

You can run the request path yourselfYes

StreamingNot documented

Single unified Docker container bundling UI, API, gateway, PostgreSQL and Redis; Docker Compose for per-service scaling; an "official Helm chart" for EKS/GKE/AKS. Image name and chart repo are not stated (LLM Gateway self-host, LLM Gateway Docker self-host).

Streaming caveats: not_documented on the fetched pages; no streaming statement or caveat found (LLM Gateway docs).

This vendor publishes no latency or throughput figure for the routing layer. That is the most common case here, and it is why the architecture class above carries the comparison instead of a number.

No figures published (theopenco/llmgateway).

What it will stop

nothing documented on the request path

No request-path policy controls are documented. That is not a fault in a product built purely for routing — but it means anything you need blocked has to be blocked before the call reaches here.

What you can see

No documented export

What gets loggedMetadata only

Token counts, latency and model names are stored, but not the text itself.

You can turn bodies offYes

Organization Settings → Policies → Data Retention Level; set to metadata-only or none. DevPass and chat subscriptions are always metadata-only with no way to enable payload storage. Responses API records are the exception: kept 30 days for previous_response_id chaining regardless of policy, unless store: false is sent (Data retention)

TracesNot documented

n.a. — no OpenTelemetry or agent tracing documented (Overview)

"Metadata Only" is the default retention level, storing timestamps, model, tokens, and costs "without full payloads"; "Retain All Data" stores complete request and response payloads including messages, tool calls, and attachments. The log table has content, messages, and reasoningContent columns used only when payload retention is on (Data retention, Logging and analytics)

Where telemetry can go

No documented export. Whatever this product records stays in its own interface, so it cannot become part of the monitoring you already run.

n.a. — activity and logs are exposed through LLMGateway's own management API rather than shipped to external sinks (Activity API)

Records user feedbackNo

Scores live trafficNo

n.a. (Activity API)

n.a. (Overview)

Depends on the vendor’s SaaS: No — all components including logging can be self-hosted (Self-host, LLMGateway repo)

Retention: The pages disagree; see Contradictions. The data-retention page says 30 days for all users with Enterprise custom (Data retention); the policies page says Free 30 days, Enterprise custom (Policies); the activity API reference says Free 3 days, Pro 90 days, Enterprise up to 1 year (Activity API); the repo advertises "Extended data retention (90 days vs 3 days)" (LLMGateway repo)

Whether it fits how you work

How much work stands between you and a first call, how different that is from running it in production, and whether it slots into the stack you already have. Recorded as the shape of the work rather than a number of minutes — how long it takes you depends on which accounts and quota you already hold, which no comparison can know.

to try: base url swapto run: base url swapfits 4 of 10 common stacks

Getting to a first call

6 numbered steps

Shape of the workChange one base URL

Your existing OpenAI-compatible client keeps working. You change a base URL and a key, and nothing else in your code moves.

Read off: the vendor’s own quickstart — 6 numbered steps.

Before step one

Your own provider keyNot hosted, yes self-hosted
The hosted service works with just its own key. Self-hosting means bringing your own provider keys — a different prerequisite for the same product.
No for the hosted platform: "With your own provider keys (BYOK), routing through LLM Gateway is free" and you may instead "use the hosted platform and pay only the credit fee" (Pricing - LLM Gateway). Yes for self-hosting: prerequisites include "API keys for the LLM providers you want to use (OpenAI, Anthropic, etc.)" (Docker | LLM Gateway Docs)
Payment methodNo card needed to start
No: "Start free with no credit card." and "You can start free without a credit card."; plan is "Free $0 forever." (Pricing - LLM Gateway)
Gate before models answerNo gate
Every catalogue model is callable as soon as you have a key.
No model gate documented; the licensing gate applies to enterprise features only: "Enterprise features require a signed license in production," with "An expired license has a seven-day grace period." (Docker | LLM Gateway Docs, Self Host LLM Gateway)

Everything you need first: Dashboard sign-in and an API key from a created Project (exported in a shell or .env file). No credit card, cloud account, cluster, or provider key is stated (Quickstart | LLM Gateway Docs)

Copyable snippet: incomplete. Partially: the endpoint shape https://api.llmgateway.io/v1/… authenticated with LLM_GATEWAY_API_KEY is documented and the page has per-language tabs, but no copy-pasteable request body rendered in the fetched content (Quickstart | LLM Gateway Docs)

Running it in production

The same scale applied to the path the vendor recommends for production traffic. Kept separate from the quickstart because for several products here the two are barely related pieces of work.

Shape of the workChange one base URL

Your existing OpenAI-compatible client keeps working. You change a base URL and a key, and nothing else in your code moves.

If you self-host it insteadDeploy it on your infrastructure

This is an infrastructure project, not an integration. Expect charts or Terraform, networking, secrets and someone who owns the deployment.

Trying this and running it are almost unrelated pieces of work. The quickstart is not a preview of the production effort.

What production needs: Stateless gateway, API, UI and background worker services; PostgreSQL for users/projects/keys/usage; Redis for response caching and the worker queue; provider API keys (OpenAI, Anthropic, Google, others); a Kubernetes cluster for the Helm deployment (EKS, GKE, AKS or self-managed); managed PostgreSQL and Redis recommended; cloud service mappings (AWS EKS/RDS/ElastiCache/Secrets Manager; GCP GKE/Cloud SQL/Memorystore/Secret Manager; Azure AKS/Azure Database for PostgreSQL/Azure Cache for Redis/Key Vault) (Self Host LLM Gateway)

Can you run it yourself

Install command publishedOffered, but no command published

Self-hosting is advertised and the requirements are described, but no page publishes a command to start from. Expect to talk to the vendor before you can run it.

Offered via Docker (all services in one container), Docker Compose, or the official Helm chart on Kubernetes, but no literal command string is printed on the self-host pages; the Docker page's post-install procedure is 4 steps ending "Test the gateway by making API calls to http://localhost:4001" (web UI at http://localhost:3002) (Self Host LLM Gateway, Docker | LLM Gateway Docs)

How it fits your stack

4 of 10

Each row is a thing you might already run. “With a caveat” means it works but not the way the vendor’s marketing implies — read the reason, because that is usually where the surprise lives.

FitsThe OpenAI SDKDrop-in: change the base URL and key, nothing else.
FitsThe Vercel AI SDK@llmgateway/ai-sdk-provider
NoCloudflare WorkersNo Workers guidance published.
FitsKubernetesOCI chart on GHCR; deploys gateway, API, UI and worker, and needs PostgreSQL plus Redis
With a caveatTerraform or OpenTofuOfficial modules
NoAn existing API gatewayNothing published about running behind your gateway.
NoCloud IAM I already runNo identity integration published.
NoLangChain or LlamaIndexNo framework integration documented.
With a caveatMCP servers to governHosted MCP server — governs nothing on your side.
FitsNothing — plain Node or PythonChange one base URL.

Reading this the other way round — pick what you already run and see every product scored against it.

The integration surfaces behind those answers

Vercel AI SDKOfficial provider package
Install the package, swap the model factory, done. Maintained by a party with a stake in it.
Named: @llmgateway/ai-sdk-provider
LLM Gateway provides the first-party npm package @llmgateway/ai-sdk-provider for the Vercel AI SDK, including generateText, streamText, and generateObject. The documentation also states that @ai-sdk/openai can use the gateway with a custom base URL. (LLM Gateway Vercel AI SDK documentation)
Cloudflare WorkersNot documented
No Workers guidance either way. If you are edge-first, verify fetch-only compatibility yourself.
n.a. (not documented)
KubernetesOfficial Helm chart
A named, published chart. You can read its values file before committing to anything.
Named: OCI chart on GHCR; deploys gateway, API, UI and worker, and needs PostgreSQL plus Redis
LLM Gateway is self-hostable on Kubernetes, and the Kubernetes documentation describes an official Helm chart published as an OCI artifact on GitHub Container Registry. The chart deploys the gateway, API, UI, and worker and requires PostgreSQL and Redis. (LLM Gateway Kubernetes self-hosting documentation)
TerraformOfficial modules
Vendor-published modules exist. Some are off-Registry Git references, so pin them explicitly.
Named: Enterprise-plan modules provisioning EKS, RDS, ElastiCache, networking and secrets; no Registry name
The Enterprise plan includes Terraform modules that provision AWS EKS, RDS, ElastiCache, networking, and secrets, and deploy LLM Gateway. The Terraform registry name is not stated. (LLM Gateway AWS self-hosting documentation)
Existing API gatewayNot documented
Nothing published about sitting behind an existing gateway. Treat it as a separate hop you route to yourself.
n.a. (not documented)
Cloud identityNot documented
No identity integration published. Expect API keys in a secret store.
n.a. (not documented)
MCPHosted MCP server
The vendor runs an MCP server you connect a client to. Useful for reaching this product from an agent, but it does not govern your other MCP servers.
LLM Gateway provides a hosted MCP server at https://api.llmgateway.io/mcp using streamable HTTP and bearer authentication. It is compatible with Claude Code, Codex, Cursor, and other MCP-compatible clients, and exposes chat, generate-image, generate-nano-banana, list-models, and list-image-models tools. (LLM Gateway MCP server)

Python frameworksNone documented

n.a. (not documented)

First-party client libraries

TypeScript

Documented integrations include the OpenAI SDK or other OpenAI-compatible SDKs, the Anthropic SDK, the Vercel AI SDK provider @llmgateway/ai-sdk-provider, and the TanStack AI adapter @tanstack/ai-llmgateway. The documented OpenAI SDK package is openai; languages are otherwise not stated, except that the Vercel AI SDK interface is TypeScript. (LLM Gateway developer documentation)

Agent features: LLM Gateway is described as working with AI agents and development tools and provides pre-built templates and agent configurations. Tool calling is supported through the Vercel AI SDK; sessions, threads, multi-step orchestration, and A2A are not documented. (LLM Gateway documentation home)

Hosted path works in "any language or framework," streaming via stream: true is proxied unchanged, every call appears in the dashboard with latency, cost and provider breakdown, and an Anthropic-compatible endpoint is available. Self-hosted setup requires creating an organization and project in the local UI before generating keys, and warns "Do not bind-mount a host directory directly to /var/lib/postgresql/data" (Quickstart | LLM Gateway Docs, Docker | LLM Gateway Docs)

LLM Gateway is available as a hosted service and can be self-hosted under AGPLv3 using Docker, Docker Compose, or Kubernetes; hosted usage is described as pay-as-you-go with a flat 5% platform fee, and BYOK is supported. It provides one API across 40+ providers and routes across 200+ models, with integrations including REST, CLI, MCP, OpenAI-compatible SDKs, the Anthropic SDK, the Vercel AI SDK provider, and the TanStack AI adapter. (LLM Gateway AI Gateway product page)

Silence in the docs: 3 of the integration questions on this card have no published answer either way. That is recorded as undocumented, not as a no — but it does mean you would be verifying it yourself.

What it does well

No token markup: you pay each provider's own rates, with a flat 5% only on hosted credit purchases
Entire platform self-hosts from a single Docker image under AGPLv3, free forever
Free plan has no seats or minimums and includes BYOK at no fee
SOC 2 Type II certified despite being a very small operation
Enterprise tier covers SSO/SAML/SCIM, guardrails, immutable SIEM-ready audit logs and provider compliance routing

Where it falls short

Tiny project by comparison (601 GitHub stars, latest tagged release v1.2.0 from September 2025) and operated by a single small Delaware LLC
Publishes no latency or throughput benchmarks, and no independent benchmark exists
Model and provider counts are inconsistent between its pricing and open-source pages
Advanced governance (guardrails, audit logs, region pinning, SLA) is gated behind an unpriced enterprise license plus the ee/ code license

Choose it when

Small teams that want an OpenRouter-style hosted gateway with a genuine zero-cost self-host escape hatch and no token markup.

Look elsewhere when

You need proven scale, published performance data, or vendor depth to support a large regulated deployment.

Open source: You run the gateway yourself. Full control over the data path and no per-request vendor fee, in exchange for operating it.

How hard is it to leave?

Derived from six published facts, not from an opinion. The weights are fixed and the same for every product — see the arithmetic.

72/88Some work to leave

Portability score breakdown for LLM Gateway
What helps you leave	Points	Source
Works with standard OpenAI codeSwitching away is a base-URL change rather than a rewrite of every call site.	22/22	vendor page
No vendor-specific SDK requiredA proprietary client library spreads through your codebase and has to be torn out again.	10/10	vendor page
Can use your own provider accountsYour keys and billing relationship stay yours, so removing the gateway does not cut off model access.	20/20	vendor page
Can be self-hostedYou can run it yourself instead of accepting a pricing or policy change.	20/20	—
Configuration lives in version controlRouting and budget rules are a file you keep, not dashboard state you would have to rebuild.	0/16	vendor page
Your request history can be exportedYou leave with your own logs instead of abandoning them.	not published	—

Read the fine print: Activity page is view-only; no documented export.

1 of the 6 inputs is not published, so the highest reachable score here is 88 rather than 100. That is a gap in the public documentation, not a mark against the product — no points are deducted, they simply cannot be claimed. This measures technical switching cost only. It does not price the engineering time to re-test prompts against a different routing stack.

Full specification

Every field we track. Blank fields say "Not published" rather than "No" — we do not infer an absence from silence. Switch to Technical in the header for the precise field names and the low-level details.

Overview

What kind of product Category: Open source Not verified; Marketplaces resell many providers behind one key. Gateways add governance on top. Open-source projects you run yourself. Cloud platforms are hyperscaler surfaces. Inference providers host models on their own hardware.
Who runs it Deployment model: Managed or self-host Not verified; Managed means the vendor operates it. Self-host means you run it on your own infrastructure. Both means you can choose.
Licence Licence: AGPL-3.0 Verified 2 days ago; Proprietary products cannot be inspected or forked. Open licences such as MIT and Apache-2.0 let you audit, modify, and run the code without permission.
Company Company: Polar Lights LLC Verified 2 days ago; The organisation that maintains the product.
Who you would be signing with Vendor status: Independent company Not verified; Whether the product is still an independent company, has been acquired, is a large cloud vendor’s product line, is run by a software foundation, or has been put into maintenance mode. Maintenance mode means bug fixes and security patches only — no new features.
Last shipped an update Latest release: 2026-08-24 Not verified; The date of the most recent release or version tag. A product that has not shipped in a year is a different risk from one that shipped last week, regardless of what its marketing site says.
GitHub stars GitHub stars: 601 Verified 2 days ago; A rough proxy for community size on open-source projects. Not a quality measure.

Cost

Markup on model prices Token markup: None Verified 2 days ago; How much the product adds on top of what the underlying model provider charges. Zero means you pay the same per-token price you would pay the model provider directly.
Fee to add funds Credit purchase fee: 5% Verified 2 days ago; A percentage charged when you top up your balance, separate from token prices. It is easy to miss because it does not appear on the per-token price list.
Monthly cost per person Seat fee: None Verified 2 days ago; A recurring per-user platform charge that applies regardless of how much you use the models.
Can use your own provider accounts BYOK supported: Yes Verified 2 days ago; Bring Your Own Key: you keep direct contracts with OpenAI, Anthropic and others, and the gateway only routes traffic. This preserves negotiated rates and committed-spend discounts.
Cost of using your own accounts BYOK terms: BYOK included on the free plan at no fee; self-hosted requests carry no credit fee and no BYOK metering. Verified 2 days ago; What the product charges to route traffic through your own provider keys.
Free tier Free tier: $0 forever: no seats or minimums, BYOK free, 3 free models limited to 20 req/min, 30-day data retention; hosted credits carry a 5% platform fee. Verified 2 days ago; What you can do without paying, useful for evaluation.
Enterprise plan from Enterprise plan from: Not published Not verified; Annual entry price for the enterprise tier, where one is published or credibly reported.
Cost to run it yourself Self-host cost: The whole platform (gateway, API, dashboard, worker) ships as one Docker image, ghcr.io/theopenco/llmgateway-unified:latest, free forever under AGPLv3 with no credit fee or BYOK metering; the same codebase powers the managed cloud. Code in the repo's ee/ directory requires a separate commercial enterprise license (contact@llmgateway.io). Verified 2 days ago; What self-hosting actually costs once you account for infrastructure and any paid tier.
How the vendor makes money Pricing model: Platform fee plus usage meters Not verified; The shape of the vendor’s bill: does the routing layer charge a percentage on top of tokens, a flat monthly fee, both, neither (because inference is the product), or nothing at all (open source with no paid tier).
How pricing works, briefly Pricing model detail: Free OSS self-host (AGPLv3) + credit top-up fee on the hosted platform (5% flat, +1.5% international card fee) + optional storage charge + Enterprise quote. No seats, no minimums, no subscription. Provider rates passed through 0% markup. BYOK 0% fee. Not verified; A one-paragraph description that covers the caveats a pricing category cannot: introductory rates, per-feature meters, tier gating, and pricing that resets on a specific date.
Minimum commitment Minimum commitment: None — 'no minimums, no subscription'. Enterprise is custom with volume discounts and a 30-day trial. Not verified; Whether the vendor requires a minimum contract term, a minimum spend, or a provisioned-capacity purchase to get its published rate.
Charges that fire after you go over an allowance Overage terms: The only after-the-fact charge is storage: full data retention bills stored requests at $0.01 per 1M tokens; metadata retention is free. Free-plan retention is 30 days; Enterprise is unlimited. No request-volume overage or export fee stated. Not verified; The line items that scale with usage after an included allowance is exhausted — log storage, extra requests, per-feature meters, data export — which is where cost estimates usually go wrong.
Prompt cache offered Cache mechanism: No gateway-owned cache Not verified; Whether the gateway offers its own response cache, what kind of match it does (exact request, prefix, semantic), or simply passes provider caching through unchanged.
Discount on cached input Cache-read discount: Not published Not verified; How much cheaper cached tokens are than fresh input, when the vendor publishes a single figure. Bundled-inference clouds usually price this per model instead of as one number.
Premium on cache writes Cache-write premium: Not published Not verified; How much more the first write of a cached prefix costs versus a plain input token. A high write premium and a low hit rate can leave you paying more than you save, so this matters as much as the read discount.
Who captures the cache saving Cache economics: Prompt caching is offered ('cache prompts for faster responses'); BYOK users 'still get unified analytics, caching, and failover'. Mechanism (exact-match / semantic / provider passthrough) not stated. LLM Gateway's only fee is 5% on credit purchases (0% on BYOK), so cache savings flow to the customer. Not verified; Whether the customer keeps the full saving from caching or the vendor captures part of it — and any conditions attached (write premium, storage fees, best-effort hits).
What you can split spend by Cost attribution: Per project (cost ceilings and per-project routing overrides) with 'Advanced analytics'. Per key/user/team/tag/customer not stated. Not verified; The dimensions the vendor documents for splitting spend — per key, per user, per team, per tag, per customer. Matters if you need to chargeback internally or bill an end customer.
How you get cost data out Cost export: No CSV/API/webhook/S3/warehouse cost export stated; only alerting webhooks and enterprise audit logs. Not verified; The mechanisms the vendor publishes for exporting cost and usage data: CSV, an API, webhooks, S3, a data warehouse, or nothing at all. Any per-unit price is included.
Who pays the model bill BYOK mode: Your keys or their credits Not verified; Whether you bring your own provider accounts (BYOK), buy inference from this vendor, or can do either. This is the single biggest commercial difference between these products: it decides who holds the contract with the model provider and who carries the spend.

Spend governance in detail

Seven signals matter when a bill starts to hurt: who can spend, how much, on what, and who gets paged when it goes wrong. Everything below is drawn from the vendor’s own pricing and docs pages — how we read these.

Virtual or scoped keysNot published
Keys that carry their own budget and rate-limit policy, so an intern experiment cannot spend against a production budget.
Not stated.
Budget caps per keyYes
A dollar or token ceiling attached to an individual key. Where enforcement is soft, one over-limit request still completes before the block kicks in.
Not stated as per-key; the published equivalent is 'cost ceilings per project' plus 'Budgets and spend controls'.
Budget caps per team or workspaceYes
A ceiling applied at a higher scope than one key — a team, a workspace, a customer, or an entire environment.
'Team management' and 'Admin controls' listed without budget specifics.
Rate limiting as a cost controlYes
Configurable request-per-time-window caps. Platform-set rate limits do not count as spend controls; user-configurable ones do.
20 req/min on 3 free models; paid models unlimited; Enterprise custom. Pre-request.
Model allowlistsNot published
A policy that constrains which models a key or team can call, keeping expensive frontier models out of the wrong hands.
Not stated. Closest are provider compliance policies ('route only to SOC 2 / ISO 27001 / GDPR providers') and region pinning.
Spend alertsYes
Alerts fired as spend approaches a threshold. Alerts that only fire after the meter has rolled over are marked as such.
'Real-time webhook alerts to your channels' (Discord and Slack).
Webhook notificationsYes
Programmatic notifications on spend events, so budget breaches can page an on-call or open a ticket.
Real-time webhook alerts.

Enforcement:Enforced before each request

Catalog

Models available Models available: ~200 Verified 2 days ago; How many models you can call, shown as a range because several vendors publish different totals on different pages. Vendor-reported either way, so counts are not directly comparable — some count every provider variant of the same model separately.
Model providers reachable Upstream providers: ~40 Verified 2 days ago; How many distinct model providers or labs you can reach, shown as a range where the vendor’s own pages disagree. More providers usually means better redundancy when one has an outage.
Works with standard OpenAI code OpenAI-compatible API: Yes Verified 2 days ago; If yes, you can usually switch to it by changing one base URL, and switch away just as easily. This is the main defence against lock-in.
OpenAI chat endpoint POST /v1/chat/completions: Yes Not verified; The endpoint almost every application ports first. "Not documented" means the vendor never states it, which is different from a documented no.
Anthropic messages endpoint POST /v1/messages: Yes Not verified; Whether Anthropic-shaped calls work without rewriting them. Several products support this only as SDK compatibility or provider passthrough rather than a native endpoint — the detail page says which.
OpenAI Responses endpoint POST /v1/responses: Not documented Not verified; The newer stateful OpenAI surface. Support is much thinner across this market than chat completions.
Embeddings endpoint POST /v1/embeddings: Not documented Not verified; Whether you can generate vectors through the same gateway, or need a second integration for retrieval workloads.
Image generation endpoint POST /v1/images/generations: Partly Not verified; Whether image models are reachable through the same surface as text.
Audio endpoints POST /v1/audio/*: Not documented Not verified; Speech-to-text and text-to-speech. Frequently the first gap in an otherwise complete gateway.
Batch jobs endpoint POST /v1/batches: Not documented Not verified; Asynchronous bulk processing, usually at a discount. Commonly undocumented, and commonly the reason a migration stalls late.
Needs the vendor’s own code library Requires a vendor-specific SDK: No Verified 2 days ago; A proprietary client library spreads through your codebase and has to be torn out again if you leave. “No” is the better answer here, and it means the standard OpenAI client works.
You can export your request history Logs / usage data export: Not published Not verified; Whether you can get your own request logs, traces, or usage records back out — through an API, a bulk export, or a download. Decides whether you leave with your history or abandon it.
Settings can live in version control Declarative config-as-code: No Verified 2 days ago; Whether routing, fallback, and budget rules can be declared in a file you keep in Git, rather than existing only as settings clicked into a hosted dashboard.
Embeddings Embeddings: Not published Not verified; Text-to-vector models, needed for search and retrieval features.
Image generation Image generation: Not published Not verified; Whether image models are reachable through the same interface.
Speech and audio Speech and audio: Not published Not verified; Text-to-speech or transcription models through the same interface.
Video generation Video generation: Not published Not verified; Whether video models are reachable through the same interface.
Batch processing Batch processing: Not published Not verified; Submitting large jobs for cheaper, slower processing. Often 50% off for work that is not time-sensitive.

Routing & reliability

Uptime it promises in writing Contractual SLA uptime: Not published Not verified; The uptime percentage in a published, contractual service level agreement. A public status page is not an SLA — it reports what happened, it does not promise anything or pay you back when it breaks.
Automatic failover Automatic failover: Yes Verified 2 days ago; When a model provider goes down or rate-limits you, traffic moves to a backup automatically instead of returning errors to your users.
Load balancing Load balancing: Yes Verified 2 days ago; Spreads requests across several providers or keys to raise your effective rate limit.
Rule-based routing Conditional routing: Yes Verified 2 days ago; Send different requests to different models based on rules — for example a cheap model for free users and a strong model for paying ones.
Response caching Response caching: Yes Verified 2 days ago; Reuses the answer when the exact same request comes in again, which cuts both cost and latency.
Similar-question caching Semantic cache: Not published Not verified; Reuses an answer when a new question means roughly the same thing as an earlier one. Saves far more than exact-match caching but can return subtly wrong answers if tuned loosely.
Where you set the timeout Request timeout surface: Not documented Not verified
`not_documented` across all four fetched pages (docs home, self-host, self-host/docker, repo) ([LLM Gateway docs](https://docs.llmgateway.io/)).; Where a request timeout can be set: per request, in a config file, in the vendor dashboard, or nowhere. Nine of the twenty products documented here do not describe a request timeout at all, so the worst case of a hung upstream call is unknowable from the docs.
Where you set retries Retry policy surface: Not documented Not verified
`not_documented`; default count and backoff: `n.a.` ([LLM Gateway docs](https://docs.llmgateway.io/)).; Where retry count and backoff are configured. Worth knowing alongside billing: a retried streaming call can be charged more than once.
Where you set fallbacks Fallback surface: Not documented Not verified
`not_documented` as a configurable key. Behavior is asserted without configuration: the gateway "fails over to a healthy provider when one errors" — ordering, weighting and key name are all unstated ([LLM Gateway docs](https://docs.llmgateway.io/)).; Where the fallback chain is defined. Almost every product claims fallback; the useful question is whether you can change it from code or only by hand in a dashboard.
Shape of the fallback chain Fallback shape: Shape not documented Not verified; An ordered list tries targets in sequence; a weighted split sends a percentage of traffic to each, which is what you need to trial a new model on 5% of requests. Weighted splits are much rarer than the marketing implies.
Upstream health tracking Health checks / circuit breaking: Not documented Not verified
Only the same "fails over to a healthy provider when one errors" statement; no health check, breaker or ejection key ([LLM Gateway docs](https://docs.llmgateway.io/)).; Whether the product notices a failing upstream and stops sending traffic to it, and whether you can tune the thresholds. This is what turns a provider outage into a blip rather than a sustained error rate, and only four of the twenty expose it.
Cross-region failover you control Multi-region failover surface: Not documented Not verified
Self-hosting on AWS/GCP/Azure and any Kubernetes cluster is documented, but no cross-region failover ([LLM Gateway self-host](https://docs.llmgateway.io/self-host)).; Whether you can define what happens when a region degrades. A vendor running many regions is not the same as a vendor letting you configure failover between them; only three document a user-controlled mechanism.
Where you set load balancing Load balancing surface: In config Not verified
`config_file` at the credential level only: "LLM Gateway supports multiple API keys per provider for load balancing — provide comma-separated values" via environment variables. No key name for the policy and no weights ([LLM Gateway Docker self-host](https://docs.llmgateway.io/self-host/docker)).; Where traffic distribution across upstreams or keys is configured.

Operations

Usage dashboards and logs Observability: Yes Verified 2 days ago; Built-in visibility into what was sent, what came back, what it cost, and how long it took.
Spending limits Budget controls: Yes Verified 2 days ago; Hard caps that stop spend before it becomes a surprise invoice. The single most valuable control for a small team.
Rate limits Rate limits: Yes Verified 2 days ago; Caps on request volume per key or per user, useful for protecting against abuse and runaway loops.
Separate keys per team or app Virtual keys: Not published Not verified; Issue scoped keys with their own budgets and permissions so you can attribute cost and revoke access without rotating everything.
Prompt versioning Prompt management: Not published Not verified; Store and version prompts outside your code so they can be changed without a deploy.
Quality testing Evals: Not published Not verified; Built-in tooling to score model output against test cases, so you can tell whether a model swap made things better or worse.
MCP support MCP support: Not published Not verified; Native support for the Model Context Protocol, the emerging standard for connecting models to external tools.
What gets logged Logged content: Metadata only Not verified
"Metadata Only" is the default retention level, storing timestamps, model, tokens, and costs "without full payloads"; "Retain All Data" stores complete request and response payloads including messages, tool calls, and attachments. The log table has `content`, `messages`, and `reasoningContent` columns used only when payload retention is on ([Data retention](https://docs.llmgateway.io/features/data-retention), [Logging and analytics](https://deepwiki.com/theopenco/llmgateway/5.3-logging-and-analytics)); Whether full prompts and responses are stored, only metadata, or nothing. Full-body logging is the most useful debugging feature here and the one most likely to need a conversation with your compliance team.
You can turn logging off Body-logging opt-out: Yes Not verified
Organization Settings → Policies → Data Retention Level; set to metadata-only or `none`. DevPass and chat subscriptions are always metadata-only with no way to enable payload storage. Responses API records are the exception: kept 30 days for `previous_response_id` chaining regardless of policy, unless `store: false` is sent ([Data retention](https://docs.llmgateway.io/features/data-retention)); Whether prompt and response bodies can be suppressed while still keeping usage metrics. Nineteen of the twenty document a way to do this; the mechanisms range from a per-request header to an organisation-wide setting.
Traces you can take elsewhere Distributed tracing: Not documented Not verified
`n.a.` — no OpenTelemetry or agent tracing documented ([Overview](https://docs.llmgateway.io/overview)); Whether the product emits OpenTelemetry, a proprietary format, or nothing. OpenTelemetry means the traces land in the tooling you already run instead of only in the vendor’s dashboard.
Where telemetry can go Export destinations: Not published Not verified
`n.a.` — activity and logs are exposed through LLMGateway's own management API rather than shipped to external sinks ([Activity API](https://theopenco-llmgateway-23.mintlify.app/api-reference/management/activity)); Documented sinks for logs and metrics. This is a good proxy for how replaceable the vendor’s own dashboard is: around twenty destinations means you never have to depend on it, while a CSV download means you do.
Can record user feedback Feedback capture API: No Not verified
`n.a.` ([Activity API](https://theopenco-llmgateway-23.mintlify.app/api-reference/management/activity)); Whether there is an API to attach a rating or score to a logged request, which is what lets production traffic feed quality work later.
Scores live traffic Online eval hooks: No Not verified
`n.a.` ([Overview](https://docs.llmgateway.io/overview)); Whether automated scorers can run against real production requests, rather than only against a test set you assemble yourself.

Performance

Delay it adds Proxy overhead: Not published Not verified; Extra time the product itself adds to each request, on top of however long the model takes. Usually irrelevant next to multi-second model latency, but it matters for high-volume or streaming-sensitive workloads.
Requests per second ceiling Throughput: Not published Not verified; Published sustained request rate before the product becomes the bottleneck. Only relevant at genuinely high volume.
What the request path runs on Architecture class: Interpreted proxy Not verified
`interpreted_proxy` (JS runtime). Repo is TypeScript 95.7%; components are `apps/ui` (Vite + React), `apps/api` (Hono backend) and `apps/gateway` ([theopenco/llmgateway](https://github.com/theopenco/llmgateway)).; The comparable way to talk about latency here. An edge worker, a compiled Go or Rust binary, and a Python proxy have different overhead floors no matter which figures each vendor publishes. Products that never disclose their runtime are recorded as undisclosed rather than assumed.
You can run the request path yourself Self-hostable data plane: Yes Not verified
Single unified Docker container bundling UI, API, gateway, PostgreSQL and Redis; Docker Compose for per-service scaling; an "official Helm chart" for EKS/GKE/AKS. Image name and chart repo are not stated ([LLM Gateway self-host](https://docs.llmgateway.io/self-host), [LLM Gateway Docker self-host](https://docs.llmgateway.io/self-host/docker)).; Whether the component that actually carries your prompts can run on your own infrastructure. Distinct from a vendor offering a self-hosted control plane while still proxying traffic through their network.
Streaming responses Streaming support: Not documented Not verified
`not_documented` on the fetched pages; no streaming statement or caveat found ([LLM Gateway docs](https://docs.llmgateway.io/)).; Whether token-by-token streaming is documented. The caveats matter more than the yes: some products cannot cancel a stream without still being billed, and several timeout and fallback mechanisms stop applying once the first token has been sent.

Security & compliance

Does your prompt reach their servers Prompt transits vendor: Depends how you deploy it Not verified
Hosted, where it sits between your application and every provider, or self-hosted with data in your own PostgreSQL. One disclosure worth reading: the hosted service may route some requests through stealth providers whose identity is not publicly disclosed.; Whether the text you send passes through this company’s own infrastructure. If it does, every other promise on this page is a policy commitment rather than a physical impossibility. Self-hosted products can answer no outright.
What they keep if you change nothing Logging default: Metadata only, not content Not verified
Metadata-only retention is the default and gives usage analytics without storing payloads.; Defaults matter more than options. A product that stores full prompts and replies unless you find the right header will have stored them by the time you read the docs.
How long they keep it Default content retention (days): 30 days Not verified
Thirty days for all users, custom on enterprise. Stored Responses-API responses are also held 30 days unless you set store to false.; Default retention for request content, in days. Zero means nothing is kept. Read the note: several products keep nothing as a rule but make timed exceptions for abuse review or specific models.
Could they train on your prompts Training on customer data: No Not verified
Customer prompts, inputs and returned responses are not used to train any models.; Whether the vendor may use your prompts and outputs to train models. “Not published” means we could not find any position, which is not the same as a no — ask for it in writing.
Where it runs, and what you can pin Region and residency control: Processed and stored in the EU or the US, with per-project routing overrides to pin regulated workloads to specific regions and providers. Self-hosting supported. Not verified; Which regions are offered and whether you can force processing to stay in one. A global endpoint that silently picks a region is a different compliance story from an endpoint you pin yourself.
Where safety filters run Guardrail execution location: Either, depending on deployment Not verified
Prompt-injection detection, PII redaction and content moderation are enforced at the gateway before any prompt leaves your boundary — which in a self-hosted install is literally your boundary.; A filter that strips personal data only helps if it runs before the data leaves your boundary. If guardrails execute in the vendor’s cloud, the vendor has already received whatever you wanted redacted.
Who else touches the data Subprocessor list: https://security.llmgateway.io/ Not verified; The published list of third parties the vendor passes your data to. No list means you cannot know the full chain, which most data-protection agreements require you to.
SOC 2 audited SOC 2 audited: Yes Verified 2 days ago; An independent audit of security controls. Enterprise buyers and their procurement teams routinely require it.
Will sign a HIPAA agreement HIPAA BAA: Not published Not verified; Required before you may send protected health information through the service. Without a signed BAA, healthcare data is off limits.
GDPR commitments GDPR commitments: Not published Not verified; Published data processing terms for handling personal data of people in the EU and UK.
Can keep data in the EU EU data residency: Not published Not verified; Requests can be processed inside the EU rather than routed to US infrastructure. Often the deciding constraint for European customers.
Does not retain your data Zero data retention: Not published Not verified; Prompts and responses are not stored after the request completes. Sometimes a paid add-on rather than the default.
Strips personal data PII redaction: Yes Verified 2 days ago; Detects and removes identifiers such as names, emails, and card numbers before the request reaches the model provider.
Content guardrails Content guardrails: Yes Verified 2 days ago; Policy checks on inputs and outputs — blocking unsafe content, enforcing formats, or catching prompt-injection attempts.
Runs fully disconnected Air-gapped deployment: Not published Not verified; Can be deployed in a network with no internet access, which some regulated and defence environments require.
Blocks personal data in prompts PII / DLP enforcement: Not documented Not verified
Data handling is a retention setting (`retentionLevel: none` avoids storing sensitive content), not a request-path scanner ([Data retention](https://docs.llmgateway.io/features/data-retention), [Logging and analytics](https://deepwiki.com/theopenco/llmgateway/5.3-logging-and-analytics)); Whether personal data detection sits on the request path and can stop the call, merely inspects and forwards it, or is not documented. A control that only reports is a logging feature, not a policy control.
Blocks prompt injection Injection / jailbreak enforcement: Not documented Not verified
`not_documented` ([Overview](https://docs.llmgateway.io/overview)); Whether injection and jailbreak detection can stop a request. Most products offering this call a partner classifier rather than shipping their own.
Blocks harmful content Toxicity / moderation enforcement: Not documented Not verified
`not_documented` ([Overview](https://docs.llmgateway.io/overview)); Whether hate, violence, sexual and self-harm categories are checked inline and can stop a request, in either direction.
Your own policy rules Custom policy hooks: Not documented Not verified
Organisation Policies cover data handling and provider routing rather than content rules ([Policies](https://docs.llmgateway.io/learn/policies), [Audit logs](https://docs.llmgateway.io/features/audit-logs)); Whether you can add your own rule — a regex, a webhook, or your own classifier — rather than choosing from the vendor library.
Where guardrails run Guardrail execution location: Either, your choice Not verified; Whether guardrail evaluation happens inside your infrastructure or on the vendor’s servers. This decides whether the prompt you are trying to protect leaves your network in order to be checked.
If the guardrail itself fails Guardrail failure mode: Not documented Not verified
`not_documented` ([Overview](https://docs.llmgateway.io/overview)); What happens when the guardrail service times out or errors: does the request proceed unchecked, or is it blocked? This is the worst-documented field in the entire catalogue — only two vendors state it plainly, which means most teams are running a control whose failure behaviour they cannot know.
Third-party guardrail vendors Guardrail integrations: Not published Not verified; Named external guardrail services the product can call. A long list means the product is a router for policy engines rather than a policy engine itself — which also means another vendor bill and another hop.

Compliance evidence

Graded by how strong the evidence is, not whether the word appears on the vendor’s website. An audited report and a marketing claim are different things, and only one of them will satisfy your own auditor.

SOC 2 Certified Type II, independently examined
ISO 27001 Not published
GDPR DPA Available on request DPA for enterprise
HIPAA BAA Not published
FedRAMP Not published
ITAR Not published

Vendor source

Fit & integration

Work to try it Evaluation work shape: Change one base URL Not verified; The shape of the work on the vendor’s own quickstart, from swapping one base URL through to deploying infrastructure. An ordinal class rather than a duration, because elapsed time depends on accounts and quota we cannot see.
Work to run it Production work shape: Change one base URL Not verified; The same scale applied to the vendor’s recommended production path. For several products this is much heavier than the quickstart, which is exactly why both are recorded.
Steps on the quickstart Numbered quickstart steps: 6 Not verified; A literal count of numbered steps on the vendor’s quickstart, recorded as evidence beside the work shape. Zero means the page publishes no numbered procedure at all. Large counts usually mean interleaved language tracks rather than more work.
Can you self-host it today Self-host install documentation: Offered, but no command published Not verified
Offered via Docker (all services in one container), Docker Compose, or the official Helm chart on Kubernetes, but no literal command string is printed on the self-host pages; the Docker page's post-install procedure is 4 steps ending "Test the gateway by making API calls to http://localhost:4001" (web UI at http://localhost:3002) ([Self Host LLM Gateway](https://docs.llmgateway.io/self-host), [Docker | LLM Gateway Docs](https://docs.llmgateway.io/self-host/docker)); Whether an install command is actually published. Several products advertise self-hosting while publishing no command to start from, which a plain yes/no would hide.
Works with the OpenAI SDK OpenAI SDK drop-in: Yes Not verified
Yes. The existing OpenAI SDK can be kept while changing the base URL to `https://api.llmgateway.io/v1`; the documented example uses `apiKey: process.env.LLM_GATEWAY_API_KEY`. ([LLM Gateway AI Gateway product page](https://llmgateway.io/products/ai-gateway)); Whether an existing OpenAI-compatible client can be pointed at it by changing the base URL and key.
Vercel AI SDK support AI SDK provider package: Official provider package Not verified
LLM Gateway provides the first-party npm package `@llmgateway/ai-sdk-provider` for the Vercel AI SDK, including `generateText`, `streamText`, and `generateObject`. The documentation also states that `@ai-sdk/openai` can use the gateway with a custom base URL. ([LLM Gateway Vercel AI SDK documentation](https://docs.llmgateway.io/developers/ai-sdk)); Whether a first-party AI SDK provider package exists, or only a community package, a documented workaround, or the generic OpenAI provider pointed at a custom base URL.
Python framework integrations Documented Python frameworks: Not published Not verified
n.a. (not documented); LangChain, LangGraph, LlamaIndex and similar orchestration frameworks with a documented integration.
Callable from Cloudflare Workers Cloudflare Workers support: Not documented Not verified
n.a. (not documented); Whether the docs show calling this product from your own Worker. Deliberately separated from the several products whose own gateway runs on Workers, which is a fact about their infrastructure and not about your edge compatibility.
Kubernetes install Helm chart availability: Official Helm chart Not verified
LLM Gateway is self-hostable on Kubernetes, and the Kubernetes documentation describes an official Helm chart published as an OCI artifact on GitHub Container Registry. The chart deploys the gateway, API, UI, and worker and requires PostgreSQL and Redis. ([LLM Gateway Kubernetes self-hosting documentation](https://docs.llmgateway.io/self-host/kubernetes)); Whether a named, published Helm chart exists, versus Helm being referenced with no chart named, versus generic cluster documentation that has nothing to do with this product.
Terraform support Terraform provider or modules: Official modules Not verified
The Enterprise plan includes Terraform modules that provision AWS EKS, RDS, ElastiCache, networking, and secrets, and deploy LLM Gateway. The Terraform registry name is not stated. ([LLM Gateway AWS self-hosting documentation](https://docs.llmgateway.io/self-host/aws)); Whether you can declare this in code: an official provider, official modules, resources inside a hyperscaler’s provider, a community provider, or only Terraform code shipped in a repo.
Reuses your cloud identity Cloud IAM reuse: Not documented Not verified
n.a. (not documented); Whether you can authenticate with IAM roles, workload identity or managed identities instead of another long-lived API key. Distinguished from products that only accept static upstream provider credentials.
Fits behind your API gateway API gateway integration: Not documented Not verified
n.a. (not documented); Whether AI traffic can go through a gateway you already run, and who documented that — the product’s vendor or the gateway’s.
MCP support MCP surface shape: Hosted MCP server Not verified
LLM Gateway provides a hosted MCP server at `https://api.llmgateway.io/mcp` using streamable HTTP and bearer authentication. It is compatible with Claude Code, Codex, Cursor, and other MCP-compatible clients, and exposes `chat`, `generate-image`, `generate-nano-banana`, `list-models`, and `list-image-models` tools. ([LLM Gateway MCP server](https://llmgateway.io/mcp)); Which kind of MCP support this is: a gateway that governs many MCP servers, a hosted MCP server you connect to, MCP tools accepted inside the completion API, or client tooling. These are different products behind one acronym.
Needs your own provider key Upstream provider key required: Not hosted, yes self-hosted Not verified
No for the hosted platform: "With your own provider keys (BYOK), routing through LLM Gateway is free" and you may instead "use the hosted platform and pay only the credit fee" ([Pricing - LLM Gateway](https://llmgateway.io/pricing)). Yes for self-hosting: prerequisites include "API keys for the LLM providers you want to use (OpenAI, Anthropic, etc.)" ([Docker | LLM Gateway Docs](https://docs.llmgateway.io/self-host/docker)); Whether an upstream provider account and key must exist before your first call works. A prerequisite rather than a step, and it can differ between the hosted and self-hosted forms of the same product.
Gate before models work Model access gate: No gate Not verified
No model gate documented; the licensing gate applies to enterprise features only: "Enterprise features require a signed license in production," with "An expired license has a seven-day grace period." ([Docker | LLM Gateway Docs](https://docs.llmgateway.io/self-host/docker), [Self Host LLM Gateway](https://docs.llmgateway.io/self-host)); Whether an enablement click, a quota grant, a paid tier or an approval form stands between a valid key and a working model call.
Official client languages First-party client SDK languages: TypeScript Not verified
Documented integrations include the OpenAI SDK or other OpenAI-compatible SDKs, the Anthropic SDK, the Vercel AI SDK provider `@llmgateway/ai-sdk-provider`, and the TanStack AI adapter `@tanstack/ai-llmgateway`. The documented OpenAI SDK package is `openai`; languages are otherwise not stated, except that the Vercel AI SDK interface is TypeScript. ([LLM Gateway developer documentation](https://docs.llmgateway.io/developers)); Languages with a first-party client library. An empty list can still mean the product is usable from any language via an OpenAI-compatible SDK.

Additional charges

These are the fees that do not appear on a per-token price list, and they are where estimates usually go wrong.

Full request/response payload retention $0.01 per 1M tokens
Credit purchases on the hosted service 5% platform fee

How pricing actually works

The whole platform (gateway, API, dashboard, worker) ships as one Docker image, ghcr.io/theopenco/llmgateway-unified:latest, free forever under AGPLv3 with no credit fee or BYOK metering; the same codebase powers the managed cloud. Code in the repo's ee/ directory requires a separate commercial enterprise license (contact@llmgateway.io).

Official links

601 GitHub stars — a proxy for community size, not for quality.

Independent coverage

Third-party analysis, walkthroughs and operator threads. We link criticism as readily as praise. Nothing here is written or published by the vendor, by a competitor listed on this site, or by an SEO content farm — how we vet these.

Video 1

DevNTell - Unifying AI Model Access with LLM Gateway feat. Co-Founders Ismail & Luca YouTube - DevNTell (Developer DAO) · Nov 2025 Independent developer show demos llmgateway.io live with its founders: self-hosting, provider keys, base-URL swap, local models and Docker deployment.

What has changed here

No tracked value on this page has moved since it was first researched. Every figure still carries its original citation and check date.

Usually weighed against

LLM Gateway vs Apache APISIX AI Gateway Compare → LLM Gateway vs Bifrost Compare → LLM Gateway vs LiteLLM Compare →