Self-hosted vs managed LLM gateways

Deployment decision worksheet

Illustrative monthly TCO: $120 compute + $40 storage/backups + 8 operating hours × $100/hour = $960, before commercial licences, inference and incident time. These are worksheet inputs, not vendor prices or a benchmark. Compare them with managed fees and the operational work that remains on your side.

Where the 31 stand

Deployment is recorded as managed-only, self-host-only or both. “Both” identifies a potential migration path, not guaranteed feature parity. Compare editions, control-plane requirements, exported configuration and data before planning a move.

14 Managed only

Someone else runs it. There is no self-host path to move to later, so the decision to adopt one is also a decision about how you would leave.

11 Both

A hosted service and a documented way to run the same gateway yourself. The group where starting managed and moving later is realistic — though what you can run for free varies enormously inside it.

6 Self-host only

No hosted option at all. You are taking on the operations from day one, and in exchange there is no vendor between you and the configuration.

The recorded licence and deployment fields describe different constraints: permission to run or modify software, and the supported operating arrangement. They may correlate within this catalogue, but an open licence does not establish complete offline operation or access to every commercial feature.

“Self-hostable” is four different things

The 17 products you can run yourself fall into four groups that get discussed as if they were one. The difference between them is not where the process runs — that is identical — but who decides what it may do, and whether you can answer that question without talking to a salesperson.

Permissive, and that is the whole product — 11

Apache-2.0 or MIT, with the feature set in the open repository. You can read the licence and know what you are getting. Note that this group still contains products which keep an enterprise edition alongside the free one, so the licence is a floor rather than a guarantee.

Copyleft — 2

AGPL. Free to run, with an obligation attached if you offer it onward as a service, and a paid exemption available for organisations that cannot accept that. The licence is the thing to check with counsel, not the price.

Open core — 0

A free build exists and is genuinely useful, but the governance features are in a commercial edition. This is the group most likely to surprise you, because the repository looks complete until the security review asks for SSO.

Proprietary, self-hosted — 3

No open-source edition at all. You run the vendor’s closed data plane inside your own network, under a commercial agreement. A legitimate answer to a data residency requirement, and not in any sense an open-source one.

The catalogue records the licence and the self-host cost note for every one of these, with the vendor page each was read from. Where a vendor gates deployment shapes behind an enterprise agreement, that appears in the note on its own page rather than being averaged into a score.

A blank reads as not published rather than as no. Silence from a vendor on a compliance question is a different thing from a documented refusal, and collapsing the two would flatter the products that say nothing.

The air-gapAir-gapped: Runs with no outbound internet access at all. A hard requirement in some defence and regulated environments, and a genuine constraint on which products are even eligible.Air-gappedRuns with no outbound internet access at all. A hard requirement in some defence and regulated environments, and a genuine constraint on which products are even eligible. question

Air-gapped installation is the requirement people most often mean when they say they have to self-host, and it is the field where this catalogue is most silent. 4 products document that they support it. 3 state that they do not. 24 of 31 publish nothing either way.

That silence is the finding. It is recorded as unpublished rather than as a no, because a vendor who has not written it down may well support it and simply never had the question asked in public. But it means the honest answer to “does this gateway run air-gapped” is unavailable from the documentation for much of this catalogue, and you will be asking a salesperson under a deadline.

What the gated tier costs

1 of 31 products publish a starting price for the commercial edition that unlocks the self-hosting features above. The field exists on every row and is empty on 30 of them.

This is worth planning around rather than complaining about. It means the moment your requirements cross from the free build into the paid one — and SSO or an audit log is usually the crossing point — you enter a quote cycle with no public anchor, on the vendor’s timetable, typically while a security review is already open. The practical defence is to find out which side of the line your requirements sit on before you deploy, not after. The head-to-head pages put two products’ licence and fee rows side by side, and the cost estimator shows what the fee-bearing options charge at your volume.

What running it yourself actually costs

No licence fee is not no cost, and the gap between the two is where self-hosting decisions go wrong. Four lines, none of which appear on a pricing page.

The gateway process

The cheapest line and the one people focus on. A compiled single-binary gateway runs in a small container; a proxy built on an interpreted runtime, or one that ships a console and a worker alongside the router, needs more. Each product’s own page records the install shape and what the vendor says about sizing.

Its dependencies

Most gateways need a database once you want budgets, virtual keys or usage accounting, and several want a cache as well. These are not optional extras; they are the difference between a demo and something you can bill against.

ObservabilityObservability: Logs, traces, and dashboards showing what was sent, what came back, what it cost, and how long it took. The thing you desperately want the first time a bill surprises you.ObservabilityLogs, traces, and dashboards showing what was sent, what came back, what it cost, and how long it took. The thing you desperately want the first time a bill surprises you.

A managed gateway comes with a dashboard. A self-hosted one usually comes with metrics and an expectation that you already run somewhere to put them. Budget for the stack, or for the vendor’s hosted control plane if it offers one.

Being paged

The largest line and the only one that is a person. You have put a new service in the request path of everything your product does with a model. Somebody now owns its upgrades, its certificate rotation and its 3am failures.

11 of the 17 self-hostable products document configuration as code, which can make configuration changes easier to review and reproduce. A gateway configured by clicking is a gateway whose production state exists only in its own database.

Common questions