Category
6 of 20 products we track
A managed gateway is a hosted control plane between your application and the model providers. You send it one request format and it handles routing, retries, caching, rate limits, guardrails and audit logging, then bills you for the service on top of whatever the tokens cost. You are buying governance, not model access.
This is the category most people mean when a security review asks "what is in front of the models?" The distinguishing feature is not the routing — a marketplace routes too — but that the product is built to be answerable to someone: per-team budgets, per-key limits, a log you can hand to an auditor, and an owner who will sign a data-processing agreement.
The trade is operational. You are adding a hop in the request path that you do not run, which means its uptime becomes your uptime and its data handling becomes part of your compliance story. Several products in this category answer that objection by also shipping a self-hostable build, so the same control plane can run inside your own network. Where that option exists it is worth taking seriously, because it collapses the vendor-risk conversation entirely.
Products here charge for the control plane, not for tokens, so a headline figure is usually a seat fee, a platform subscription, or a request-volume tier. That number is not comparable with a marketplace percentage: one is a fixed cost that dilutes as you grow, the other scales with spend forever. Compare them at your actual volume, not by which figure looks smaller.
The pricing guide works through all four charging mechanisms — token markupMarkup: A percentage the gateway adds on top of what the model actually costs. Some charge none at all and make money elsewhere., credit feesCredit or top-up fee: A cut taken when you add money to a prepaid balance, typically around 5%. Easy to miss because it is not a markup on tokens — but you pay it on every dollar you load., per-seat, and self-hostedSelf-hosted: You run the software on your own infrastructure. No third party sees your traffic, and there is no vendor fee — but you own the uptime, the patching, and the upgrades. — with figures computed at four real workloads, and the cost estimator runs the same model against your own volumes.
If you want one key that reaches hundreds of models tomorrow morning and nobody is going to ask you for a SOC 2 report, a managed marketplace will get you there faster and cheaper.
Multi-provider gateway inside Braintrust's eval and observability platform, with caching and span-level tracing.
US company · EU region available
Best for Teams that already run evals and tracing in Braintrust and want their model traffic to flow through the same platform.
Open-source LLM observability platform with an OpenAI-compatible AI gateway attached.
US company · EU region available
Maintenance mode
Best for Teams whose main need is per-request, per-user LLM observability with a thin gateway bolted on, ideally self-hosted.
AI plugins on the Kong API gateway, adding LLM routing, guardrails and token limits to existing API infrastructure.
US company
Best for Enterprises already standardized on Kong for API management that want AI traffic governed by the same gateway, plugins and ops tooling.
Managed EU-hosted AI gateway and router bundled with evaluation, observability and governance tooling.
Netherlands company · EU region available
Best for European teams that want an EU-resident managed gateway with governance, evaluations and observability in one platform rather than a bare proxy.
Open-core AI gateway with a hosted control plane for observability, prompt management and governance.
US company
Acquired
Best for Teams that want governance, guardrails and prompt management in one control plane, with the option to self-host the routing engine.
Vercel-operated gateway that routes AI SDK and OpenAI-format requests to many providers with zero token markup.
US company
Best for Teams already shipping on Vercel with the AI SDK who want zero token markup and no credit-purchase fee.
Products are sorted by what they actually are, not by what they are marketed as. If none of the above is the shape of your problem, one of these probably is.
A marketplace sells you access to models it aggregates and resells. A managed gateway sells you control over calls you are already entitled to make. The clearest test is billing: if the product resells tokens and takes a cut, it is a marketplace; if it charges a platform or seat fee and you keep your own provider accounts, it is a gateway.
Some can. Several products in this category publish a self-hostable or air-gapped build alongside the hosted one, which lets the same control plane run inside your network. Others are hosted-only. The deployment field on each product page states which, and links to the vendor page it was read from.
Usually not. The common pattern is that you bring your own provider keys and pay the model vendors directly, with the gateway charging separately for the platform. Where a product does add a percentage, it is recorded in the token markup field with a link to the pricing page stating it.