Guide
Best LLM gateways in 2026
Short answer
There is no single best LLM gateway, because the five common reasons for wanting one have five different answers. Self-hosting on your own infrastructure, getting to many models fastest, satisfying a security review, staying inside the platform you already deploy to, and staying inside one cloud account each point at a different product. Pick the job, then the product.
How to read this
The picks below are editorial judgement and are labelled as such. There is no composite score on this page and none anywhere on this site, because a single number would have to decide for you how much compliance is worth against how much cost is worth, and that weighting is the decision you came here to make.
What is not editorial is the evidence. Every figure under every pick is rendered from the catalogue at build time with its source link and the date it was last checked, so the numbers cannot drift out of step with the argument they support. Where a value has not been published by the vendor it says so rather than showing a blank or, worse, a no. If a pick ever contradicts a figure beneath it, the figure is right.
Nobody pays to appear here. If you would rather answer questions than read five arguments, the six-question wizard filters the same catalogue on your constraints and shows why each product was ruled in or out.
The five jobs at a glance
| If your job is | Our pick | Runner-up | When this is you |
|---|---|---|---|
| You need it on your own infrastructure | LiteLLM | Bifrost | Regulated data that cannot leave your network, an existing Kubernetes platform, or a flat refusal to put a third party in the request path. |
| You want to be calling many models this afternoon | OpenRouter | Requesty | Prototyping, evaluating models against each other, or a product whose model choice is not settled yet. |
| You need a signed BAA and an audit trail | Portkey | Orq.ai Router | Healthcare, or any organisation whose security review will ask for a SOC 2 report and a data-processing agreement before a line of code ships. |
| Your application already lives on Vercel | Vercel AI Gateway | Cloudflare AI Gateway | A Next.js application deployed on Vercel, using the AI SDK, where the gateway should be one less thing to think about. |
| Everything must stay inside one cloud account | Amazon Bedrock | Azure AI Foundry | Procurement, existing committed spend, or a security model built on cloud IAM rather than on API keys. |
Job 1 of 5
You need it on your own infrastructure
Regulated data that cannot leave your network, an existing Kubernetes platform, or a flat refusal to put a third party in the request path.
Our pick
The licence is the whole argument. A permissively licensed gateway you run yourself has no vendor in the request path, no per-seat meter, and no commercial event that can change your economics next quarter. It also covers by far the most upstream providers of anything self-hostable here, which matters because the reason to self-host is usually that you already have an awkward mix of models to serve.
What you are accepting in exchange is that this is now a service you operate, monitor and upgrade. Teams that adopt it without owning that end up with a single-replica gateway in front of production, which is strictly worse than no gateway at all.
What decided it
| Field | LiteLLM | Bifrost |
|---|---|---|
| Licence | MIT | Apache-2.0 |
| Who runs it | Self-host only | Self-host only |
| Model providers reachable | Not published | 20–23 |
| Runs fully disconnected | Yes | Yes |
| Markup on model prices | None | None |
Choose Bifrost instead if
- You want the same self-hosted, permissive-licence position but care more about the gateway being lean than about breadth of providers.
- Your provider list is short and stable, so the coverage gap does not cost you anything.
Side by side: LiteLLM vs Bifrost LiteLLM vs Portkey
Job 2 of 5
You want to be calling many models this afternoon
Prototyping, evaluating models against each other, or a product whose model choice is not settled yet.
Our pick
Nothing else here gets you access to this many models this quickly. One key, one endpoint, no cloud account to provision and no infrastructure to stand up, which is exactly the right trade while your question is still "which model is good at this" rather than "how do we run this at scale".
The cost of that convenience is a cut on the money you load rather than a markup on tokens — an easy fee to miss precisely because it is not on the per-token price. At prototype spend it is irrelevant. At production spend it is the reason to re-read the pricing guide, and BYOK is the escape hatch.
What decided it
| Field | OpenRouter | Requesty |
|---|---|---|
| Models available | 400–500 | 160–600 |
| Model providers reachable | ~83 | Not published |
| Markup on model prices | None | 5% |
| Fee to add funds | 5.5% | Not published |
| Can use your own provider accounts | Yes | Yes |
Choose Requesty instead if
- You want the same managed, wide-catalogue position with a per-token markup instead of a top-up cut, and you have read what that means at your volume.
- EU data residency on the free plan matters more to you than the size of the model catalogue.
- You are prototyping rather than shipping regulated work: it publishes no SOC 2 report, which is fine for an evaluation and disqualifying for a security review.
Side by side: OpenRouter vs LiteLLM OpenRouter vs Requesty
Job 3 of 5
You need a signed BAA and an audit trail
Healthcare, or any organisation whose security review will ask for a SOC 2 report and a data-processing agreement before a line of code ships.
Our pick
A signed BAA and a SOC 2 report are table stakes here and several products have both. The reason this is our pick is the deployment column: you can start on the managed service to get moving and later move the same gateway inside your own perimeter without changing product, which is the migration most regulated teams discover they need about a year in.
Read the compliance claims narrowly. A vendor being willing to sign a BAA is not the same as your configuration being compliant, and the retention and logging defaults are where that goes wrong. Check what is logged by default before the security review does.
What decided it
| Field | Portkey | Orq.ai Router |
|---|---|---|
| Will sign a HIPAA agreement | Yes | Yes |
| SOC 2 audited | Yes | Yes |
| Who runs it | Managed or self-host | Managed or self-host |
| Licence | Open core | Proprietary |
| Does not retain your data | Yes | Yes |
Choose Orq.ai Router instead if
- You need an air-gapped deployment as well as the compliance paperwork, which narrows the field sharply.
- You want the evaluation and experiment tooling in the same product rather than bolted alongside it.
Side by side: LiteLLM vs Portkey Portkey vs Helicone
Job 4 of 5
Your application already lives on Vercel
A Next.js application deployed on Vercel, using the AI SDK, where the gateway should be one less thing to think about.
Our pick
This is the case where the platform-native option genuinely wins, and it wins on integration rather than on price. No markup on tokens and no cut on credits, first-class support in the SDK you are already using, and billing that arrives on an invoice you already pay. For a team whose deployment target is settled, that removes an entire vendor relationship.
The counter-argument is concentration. Your hosting, your framework and now your model access all terminate at one company, and the seat charge means the cost scales with headcount rather than usage. If you are already uneasy about that dependency, this pick makes it worse rather than better.
What decided it
| Field | Vercel AI Gateway | Cloudflare AI Gateway |
|---|---|---|
| Markup on model prices | None | None |
| Fee to add funds | None | 5% |
| Monthly cost per person | $20/user/mo | None |
| Vercel AI SDK support | Official provider package | Official provider package |
| Can use your own provider accounts | Yes | Yes |
Choose Cloudflare AI Gateway instead if
- Your edge and hosting story is Cloudflare rather than Vercel, in which case the identical argument applies to the identical product on the other platform.
- You want the gateway analytics and caching without adding a per-seat line item.
Side by side: OpenRouter vs Vercel AI Gateway Cloudflare AI Gateway vs Vercel AI Gateway Vercel AI Gateway vs LiteLLM
Job 5 of 5
Everything must stay inside one cloud account
Procurement, existing committed spend, or a security model built on cloud IAM rather than on API keys.
Our pick
When the constraint is the cloud account, the choice is made for you: use the gateway belonging to the cloud you are already committed to. We name one here for concreteness rather than because it beats its two peers on the merits — the deciding factor is which committed-spend agreement and IAM model you already have, and that is not a fact about the product.
What you get is IAM instead of API keys, one invoice, and data that never leaves the account boundary. What you give up is the model catalogue: all three hyperscalers carry a fraction of what the marketplaces do, and the model you want may simply not be there. That is the trade to check first, before any of the rest of it.
What decided it
| Field | Amazon Bedrock | Azure AI Foundry |
|---|---|---|
| Models available | ~100 | ~10,000 |
| Will sign a HIPAA agreement | Yes | Yes |
| SOC 2 audited | Yes | Yes |
| Reuses your cloud identity | Reuses your cloud identity | Reuses your cloud identity |
| Who runs it | Managed only | Managed or self-host |
Choose Azure AI Foundry instead if
- Your organisation is an Azure shop, or the models you need are the ones Foundry carries and Bedrock does not.
- You want the option of a self-managed deployment alongside the managed platform.
Side by side: Amazon Bedrock vs Azure AI Foundry Amazon Bedrock vs Google Vertex AI Amazon Bedrock vs OpenRouter
Where this page is weakest
Two honest limits. First, performance: no neutral benchmark covers this category, so every latency and throughput figure in the catalogue is the vendor's own, measured on their hardware to make their point. Those figures are marked as such and none of the picks above rests on one. If a vendor's speed claim is the reason you are choosing, test it yourself on your workload.
Second, coverage of the fields themselves. Some products publish far more than others, and a product that says nothing about retention or air-gapped deployment gets "not published" rather than a negative. That is the honest rendering, but it does mean the more transparent vendors can look more constrained than the quiet ones. Read a blank as a question to ask on a sales call, not as a point in anyone's favour.
Common questions
Which LLM gateway is the best one overall?
There is not one, and any page that names a single winner is either selling something or has picked one job and not told you which. The five jobs on this page pull in genuinely opposite directions: the answer that is right when your data cannot leave your network is wrong when you want to be calling twenty models this afternoon. Pick the job first and the product follows quickly.
How are these picks decided? Is there a score?
No score, deliberately. A single number would have to weight cost against compliance against model coverage on your behalf, and the weighting is the actual decision. These are hand-written editorial picks; what is not hand-written is the evidence, because every figure quoted under a pick is rendered live from the catalogue with its source link and check date. You can disagree with a pick while still trusting the numbers under it.
Does anyone pay to be listed or recommended here?
No. There are no paid placements, no affiliate links and no sponsored entries. Products are added because they meet the definition of a gateway, and removed only if they shut down. The methodology page states what is scored and what is deliberately not.
Is a self-hosted gateway cheaper than a managed one?
Cheaper in fees, not necessarily in cost. The open-source gateways here charge nothing and take no cut, so at high spend the fee saving is real and large. Against that you are running a high-availability service in your request path: the engineer time to deploy, monitor, upgrade and be paged for it is the price, and below a certain spend it is more than the fees you avoided.
Should I use a gateway or just call the provider directly?
Call the provider directly while you have one provider, one application and no need to attribute spend. Adopt a gateway when a second provider appears, when you cannot answer which team or feature spent the money, or when a provider outage starts taking a user-facing feature down with it. The explainer guide has a fuller checklist for both directions.
How current is this page?
The prose carries a published and updated date from the revision history of this page specifically, not from the last time anything on the site was deployed. The figures are separate: each one shows the date that individual value was last checked against the vendor, so a stale number is visible as a stale number rather than hidden behind a fresh page date.