Guide
Best LLM gateways in 2026
Short answer
There is no single best LLM gateway, because the five common reasons for wanting one have five different answers. Self-hosting on your own infrastructure, getting to many models fastest, satisfying a security review, staying inside the platform you already deploy to, and staying inside one cloud account each point at a different product. Pick the job, then the product.
The five jobs at a glance
| If your job is | Our pick | Runner-up | When this is you |
|---|---|---|---|
| You need it on your own infrastructure | LiteLLM | Bifrost | Regulated data that cannot leave your network, an existing Kubernetes platform, or a flat refusal to put a third party in the request path. |
| You want to be calling many models this afternoon | OpenRouter | Requesty | Prototyping, evaluating models against each other, or a product whose model choice is not settled yet. |
| You need a signed BAA and an audit trail | Portkey | Orq.ai Router | Healthcare, or any organisation whose security review will ask for a SOC 2 report and a data-processing agreement before a line of code ships. |
| Your application already lives on Vercel | Vercel AI Gateway | Cloudflare AI Gateway | A Next.js application deployed on Vercel, using the AI SDK, where the gateway should be one less thing to think about. |
| Everything must stay inside one cloud account | Amazon Bedrock | Azure AI Foundry | Procurement, existing committed spend, or a security model built on cloud IAM rather than on API keys. |
How to use this shortlist
Before deciding, check deployment ownership, compliance evidence, required-model coverage and budget enforcement.
The picks below are editorial judgement and are labelled as such. There is no composite score on this page and none anywhere on this site, because a single number would have to decide for you how much compliance is worth against how much cost is worth, and that weighting is the decision you came here to make.
The evidence tables use catalogue values, source links and recorded check dates. The recommendation is a separate editorial decision and must be re-evaluated when a deciding field changes. Missing values remain questions to resolve before selection; a synchronized table alone does not validate a pick.
Nobody pays to appear here. If you would rather answer questions than read five arguments, the six-question wizard filters the same catalogue on your constraints and shows why each product was ruled in or out.
Job 1 of 5
You need it on your own infrastructure
Regulated data that cannot leave your network, an existing Kubernetes platform, or a flat refusal to put a third party in the request path.
Our pick
LiteLLM is our starting point for a self-hosted proxy because its recorded deployment and licence let a team operate the gateway itself. This is not a claim of the widest provider coverage. Verify your exact models and the edition required for governance; model hosting, commercial support and operational work remain separate costs.
What you are accepting in exchange is that this is now a service you operate, monitor and upgrade. Teams that adopt it without owning that end up with a single-replica gateway in front of production, which introduces a new single point of failure.
Rule this pick out if its required model, deployment, contract or operational acceptance test fails. Do not trade a mandatory requirement for the convenience of the recommendation.
Evidence to check
| Field | LiteLLM | Bifrost |
|---|---|---|
| Licence | MIT Checked 2026-08-29 | Apache-2.0 Checked 2026-08-29 |
| Who runs it | Self-host only Check date not recorded | Self-host only Check date not recorded |
| Model providers reachable | Not published Low: Check date not recorded · High: Check date not recorded | 20–23 Low: Check date not recorded · High: Check date not recorded |
| Runs fully disconnected | Yes Checked 2026-08-29 | Yes Checked 2026-08-29 |
| Markup on model prices | None Checked 2026-08-29 | None Checked 2026-08-29 |
Choose Bifrost instead if
- You want the same self-hosted, permissive-licence position but care more about the gateway being lean than about breadth of providers.
- Bifrost supports every model and endpoint in your acceptance suite, and its operational model suits your team.
Side by side: LiteLLM vs Bifrost LiteLLM vs Portkey
Job 2 of 5
You want to be calling many models this afternoon
Prototyping, evaluating models against each other, or a product whose model choice is not settled yet.
Our pick
OpenRouter is our starting point for managed model evaluation: a unified endpoint and purchased credits remove the need to deploy a proxy. This is an integration preference, not a measured fastest setup or largest catalogue claim. Compare the actual models and provider routes you need, not headline counts.
The cost of that convenience is a cut on the money you load rather than a markup on tokens — an easy fee to miss precisely because it is not on the per-token price. At prototype spend the minimum top-up fee can matter. At production spend it is the reason to re-read the pricing guide, and BYOK is the escape hatch.
Rule this pick out if its required model, deployment, contract or operational acceptance test fails. Do not trade a mandatory requirement for the convenience of the recommendation.
Evidence to check
| Field | OpenRouter | Requesty |
|---|---|---|
| Models available | 458 Low: Checked 2026-09-25 · High: Checked 2026-09-25 | 759 Low: Checked 2026-10-07 · High: Checked 2026-10-07 |
| Model providers reachable | ~83 Low: Check date not recorded · High: Check date not recorded | Not published Low: Check date not recorded · High: Check date not recorded |
| Markup on model prices | None Checked 2026-08-29 | 5% Checked 2026-08-29 |
| Fee to add funds | 5.5% Checked 2026-08-29 | Not published Check date not recorded |
| Can use your own provider accounts | Yes Checked 2026-08-29 | Yes Checked 2026-08-29 |
Choose Requesty instead if
- You want the same managed, wide-catalogue position with a per-token markup instead of a top-up cut, and you have read what that means at your volume.
- EU data residency on the free plan matters more to you than the size of the model catalogue.
- You are prototyping rather than shipping regulated work: it has no completed SOC 2 report evidenced in this catalogue; request the current report and assess whether your policy requires it.
Side by side: OpenRouter vs LiteLLM OpenRouter vs Requesty
Job 3 of 5
You need a signed BAA and an audit trail
Healthcare, or any organisation whose security review will ask for a SOC 2 report and a data-processing agreement before a line of code ships.
Our pick
A signed BAA and a SOC 2 report are table stakes here and several products have both. The reason this is our pick is the deployment column: you can start on the managed service to get moving and assess its self-hosting offering if your requirements later change. Confirm commercial edition, control-plane dependencies and migration effort before assuming the configurations are portable.
Read the compliance claims narrowly. A vendor being willing to sign a BAA is not the same as your configuration being compliant, and the retention and logging defaults are where that goes wrong. Check what is logged by default before the security review does.
Rule this pick out if its required model, deployment, contract or operational acceptance test fails. Do not trade a mandatory requirement for the convenience of the recommendation.
Evidence to check
| Field | Portkey | Orq.ai Router |
|---|---|---|
| Will sign a HIPAA agreement | Yes Checked 2026-08-29 | Yes Checked 2026-08-29 |
| SOC 2 audited | claimed Check date not recorded | Completed audit evidence Check date not recorded |
| Who runs it | Managed or self-host Checked 2026-08-29 | Managed or self-host Check date not recorded |
| Licence | MIT Checked 2026-09-14 | Proprietary Check date not recorded |
| Does not retain your data | Yes Check date not recorded | Yes Check date not recorded |
Choose Orq.ai Router instead if
- You need an air-gapped deployment as well as the compliance paperwork, which narrows the field sharply.
- You want the evaluation and experiment tooling in the same product rather than bolted alongside it.
Side by side: LiteLLM vs Portkey Portkey vs Helicone
Job 4 of 5
Your application already lives on Vercel
A Next.js application deployed on Vercel, using the AI SDK, where the gateway should be one less thing to think about.
Our pick
This is the case where the platform-native option genuinely wins, and it wins on integration rather than on price. No markup on tokens and no cut on credits, first-class support in the SDK you are already using, and billing that arrives on an invoice you already pay. For a team whose deployment target is settled, that removes an entire vendor relationship.
The counter-argument is concentration. Your hosting, your framework and now your model access all terminate at one company, and optional features may require a paid platform plan as well as usage fees. Basic gateway credits do not require a per-developer gateway subscription. If you are already uneasy about that dependency, this pick makes it worse rather than better.
Rule this pick out if its required model, deployment, contract or operational acceptance test fails. Do not trade a mandatory requirement for the convenience of the recommendation.
Evidence to check
| Field | Vercel AI Gateway | Cloudflare AI Gateway |
|---|---|---|
| Markup on model prices | None Checked 2026-08-29 | None Checked 2026-08-29 |
| Fee to add funds | None Checked 2026-08-29 | 5% Checked 2026-08-29 |
| Monthly cost per person | None Checked 2026-09-16 | None Checked 2026-08-29 |
| Vercel AI SDK support | Official provider package Check date not recorded | Official provider package Check date not recorded |
| Can use your own provider accounts | Yes Checked 2026-08-29 | Yes Checked 2026-08-29 |
Choose Cloudflare AI Gateway instead if
- Your edge and hosting story is Cloudflare rather than Vercel, and its specific caching, routing and export capabilities meet your requirements.
- You prefer Cloudflare integration after comparing its feature scope and total cost with the Vercel features you would enable.
Side by side: OpenRouter vs Vercel AI Gateway Cloudflare AI Gateway vs Vercel AI Gateway Vercel AI Gateway vs LiteLLM
Job 5 of 5
Everything must stay inside one cloud account
Procurement, existing committed spend, or a security model built on cloud IAM rather than on API keys.
Our pick
When the constraint is the cloud account, the choice is made for you: use the gateway belonging to the cloud you are already committed to. We name one here for concreteness rather than because it beats its two peers on the merits — the deciding factor is which committed-spend agreement and IAM model you already have, and that is not a fact about the product.
Cloud procurement and IAM integration can simplify operations, but an account is not a guarantee about data residency or processing boundaries. Check model-specific regions, service terms and inference routes. Compare an approved-model inventory rather than assuming all cloud catalogues are smaller or equivalent.
Rule this pick out if its required model, deployment, contract or operational acceptance test fails. Do not trade a mandatory requirement for the convenience of the recommendation.
Evidence to check
| Field | Amazon Bedrock | Azure AI Foundry |
|---|---|---|
| Models available | ~100 Low: Check date not recorded · High: Check date not recorded | ~10,000 Low: Check date not recorded · High: Check date not recorded |
| Will sign a HIPAA agreement | Yes Checked 2026-08-29 | Yes Checked 2026-08-29 |
| SOC 2 audited | Completed audit evidence Check date not recorded | not published Check date not recorded |
| Reuses your cloud identity | Reuses your cloud identity Check date not recorded | Reuses your cloud identity Check date not recorded |
| Who runs it | Managed only Check date not recorded | Managed or self-host Checked 2026-08-29 |
Choose Azure AI Foundry instead if
- Your organisation is an Azure shop, or the models you need are the ones Foundry carries and Bedrock does not.
- You want the option of a self-managed deployment alongside the managed platform.
Side by side: Amazon Bedrock vs Azure AI Foundry Amazon Bedrock vs Google Vertex AI Amazon Bedrock vs OpenRouter
Where this page is weakest
Three limits. First, performance: no neutral benchmark covers this category, so every latency and throughput figure in the catalogue is the vendor's own, measured on their hardware to make their point. Those figures are marked as such and none of the picks above rests on one. If a vendor's speed claim is the reason you are choosing, test it yourself on your workload.
Second, coverage of the fields themselves. Some products publish far more than others, and a product that says nothing about retention or air-gapped deployment gets "not published" rather than a negative. That is the honest rendering, but it does mean the more transparent vendors can look more constrained than the quiet ones. Read a blank as a question to ask on a sales call, not as a point in anyone's favour.
Third, the field itself is not fully covered. Two products have been checked against the inclusion criteria, qualify, and are not rated here yet: Martian Gateway and the AI gateway capabilities in Azure API Management. Neither could have displaced a pick above without being rated first, so treat the shortlist as the shortlist within the products reviewed here rather than of everything that exists. What is in scope, and what is missing keeps that list current.
Common questions
Which LLM gateway is the best one overall?
There is no universal winner. A useful recommendation states the requirements and tradeoffs it prioritizes. The five jobs on this page pull in genuinely opposite directions: the answer that is right when your data cannot leave your network is wrong when you want to be calling twenty models this afternoon. Pick the job first and the product follows quickly.
How are these picks decided? Is there a score?
No score, deliberately. A single number would have to weight cost against compliance against model coverage on your behalf, and the weighting is the actual decision. These are hand-written editorial picks; what is not hand-written is the evidence, because every figure quoted under a pick is rendered live from the catalogue with its source link and check date. You can disagree with a pick while still trusting the numbers under it.
Does anyone pay to be listed or recommended here?
No. There are no paid placements, no affiliate links and no sponsored entries. Products are added because they meet the definition of a gateway, and removed only if they shut down. The methodology page states what is scored and what is deliberately not.
Is a self-hosted gateway cheaper than a managed one?
Cheaper in fees, not necessarily in cost. An open-source edition may avoid a software licence charge; enterprise features, infrastructure and operations can still cost money. Against that you are running a high-availability service in your request path: the engineer time to deploy, monitor, upgrade and be paged for it is the price, and below a certain spend it is more than the fees you avoided.
Should I use a gateway or just call the provider directly?
Call the provider directly while you have one provider, one application and no need to attribute spend. Adopt a gateway when a second provider appears, when you cannot answer which team or feature spent the money, or when a provider outage starts taking a user-facing feature down with it. The explainer guide has a fuller checklist for both directions.
How current is this page?
The prose carries a published and updated date from the revision history of this page specifically, not from the last time anything on the site was deployed. The figures are separate: each one shows the date that individual value was last checked against the vendor, so a stale number is visible as a stale number rather than hidden behind a fresh page date.