Open source

9 of 31 products we track

An open-source gateway is one you run on your own infrastructure. There is no vendor in the request path, no per-request fee, and no third party holding your prompts. You get the source under a named licence and take on operating it: deployment, upgrades, availability and whoever gets paged at three in the morning.

What this category is

This category exists because for some organisations every other answer is disqualified before price is discussed. Regulated data that cannot leave the network, an air-gapped environment, or a flat institutional refusal to add a third party to the request path all lead here. The absence of a vendor is the feature.

The licence matters more than it usually does. These products range from permissive MIT and Apache-2.0 through copyleft AGPL-3.0 to open-core arrangements where the repository is real but the features you actually wanted are in the commercial tier. Each is a different long-term commitment, and the one that bites later is discovering the governance feature your auditor asked for sits behind the licence boundary.

Why you cannot compare these prices with the others

The catalogue records these as no fee, and every cost model on this site says so. That figure is honest about the vendor and silent about you: it excludes the compute you provision, the availability you engineer, and the engineer-time you spend, which is usually the largest number of the four. Treat a zero here as "no vendor invoice", never as "free".

The pricing guide works through all four charging mechanisms — token markupMarkup: A percentage the gateway adds on top of what the model actually costs. Some charge none at all and make money elsewhere., credit feesCredit or top-up fee: A cut taken when you add money to a prepaid balance, typically around 5%. Easy to miss because it is not a markup on tokens — but you pay it on every dollar you load., per-seat, and self-hostedSelf-hosted: You operate the software on your infrastructure and own its uptime, patching and upgrades. Model hosting, telemetry and control planes may still involve third parties; commercial editions can have licence fees. — with figures computed at four real workloads, and the cost estimator runs the same model against your own volumes.

When this is the wrong category

If you have no platform team and no appetite to acquire one, a hosted product is genuinely cheaper. The self-hosted option only wins when running infrastructure is already something you do well.

Open source products (9)

agentgateway

Open source Self-host only Apache-2.0 3 security incidents on record

Rust single-binary gateway for LLM, MCP, A2A and ordinary HTTP traffic, run entirely on your own infrastructure.

Cost above the model bill: Self-hosted; Self-hosting is the only mode and it is free of vendor charges. The binary is a single Rust process (13-22 MB resident in the project's own benchmark runs), so the practical floor is one small container plus, optionally, SQLite or PostgreSQL if you want USD budgets, the cost dashboard or database-backed request logs ([Database](https://agentgateway.dev/docs/standalone/latest/setup/database/), [benchmark](https://agentgateway.dev/blog/2026-06-26-benchmarking-agentgateway-vs-litellm-part-2/)).
Models: ~1,002; across 20–44 providers

Failover
Spend limits
Logs
Guardrails

Best for Platform teams that already run Kubernetes or Envoy-style infrastructure and want one Rust data plane in front of LLM, MCP, A2A and ordinary HTTP traffic, with per-key USD budgets and guardrails defined in version-controlled YAML.

Apache APISIX AI Gateway

Open source Self-host only Apache-2.0

AI proxy plugins for the Apache APISIX API gateway, adding LLM routing and token limits to an OpenResty data plane.

US company

Cost above the model bill: Self-hosted; No license cost at all; you run APISIX and its etcd control store on your own infrastructure. The docs are co-branded with API7, a commercial vendor offering a supported distribution, whose pricing is not published on these pages.
Models: —; across 10–20 providers

Failover
Logs
Guardrails

Best for Teams already running APISIX that want basic multi-provider LLM proxying, retries and token-based rate limiting without adding another gateway.

Bifrost

Open source Self-host only Apache-2.0

Go-based open-source AI gateway focused on low proxy overhead, with an enterprise tier for clustering and SSO.

US company

Cost above the model bill: Self-hosted; OSS is free (Apache-2.0); infra cost only. Enterprise adds guardrails, cluster mode, adaptive load balancing, SAML/OIDC SSO, vault integration, log exports, audit logs, RBAC and SLAs at custom pricing after a 14-day trial; VPC, on-prem and air-gapped installs are enterprise options.
Models: —; across 20–23 providers

Failover
Spend limits
Logs
Semantic cache
Guardrails

Best for Teams that want a fast Go gateway they can self-host for free and later buy clustering, SSO and guardrails from a single vendor.

Envoy AI Gateway

Open source Self-host only Apache-2.0

Kubernetes-native open-source AI gateway that adds a unified LLM and MCP data plane on top of CNCF Envoy Gateway, with a stable v1beta1 CRD API and no vendor-hosted tier.

Cost above the model bill: Self-hosted; Self-hosting is the only mode and carries no licence fee: `helm upgrade -i aieg-crd oci://docker.io/envoyproxy/ai-gateway-crds-helm` followed by `helm upgrade -i aieg oci://docker.io/envoyproxy/ai-gateway-helm` ([installation](https://aigateway.envoyproxy.io/docs/getting-started/installation)). Real cost drivers are the Envoy Gateway control plane and Envoy data plane it requires, the AI Gateway controller (documented starting requests 100m CPU / 128Mi, limits 500m / 256Mi, HPA example min 2 max 10 replicas at 70% CPU) and a Redis instance if you enable token rate limiting or quotas ([scaling](https://aigateway.envoyproxy.io/docs/capabilities/scaling); [rate limiting](https://aigateway.envoyproxy.io/docs/capabilities/traffic/usage-based-ratelimiting), 2026-09-02).
Models: —; across 16–19 providers

Failover
Spend limits
Logs

Best for Platform teams already running Envoy Gateway or Gateway API on Kubernetes who want provider-agnostic LLM and MCP routing, cloud-native upstream credential handling and OpenTelemetry GenAI telemetry as ordinary infrastructure, with a stable CRD API and no vendor in the request path.

Higress

Open source Managed or self-host Apache-2.0

Envoy/Istio-based Kubernetes and microservice gateway from Alibaba, now a CNCF Sandbox project, whose Wasm plugin set turns it into an LLM and MCP gateway.

Cost above the model bill: No markup; You pay the model provider’s own prices
Models: ~100; across 26–31 providers

Failover
Spend limits
Logs
Semantic cache
Guardrails

Best for Teams already running Kubernetes who want one Envoy-based gateway for north-south traffic, microservices and LLM/MCP calls, and who are happy to assemble reliability and guardrail behaviour from plugin fields themselves.

LiteLLM

Open source Self-host only MIT Security incident on record

Open-source AI gateway and Python SDK that puts one OpenAI-compatible API in front of many LLM providers.

US company

Cost above the model bill: Self-hosted; OSS is free (MIT); you pay only for your own containers plus Postgres and Redis. Enterprise features (SSO, RBAC, JWT auth, SCIM, audit logs, support SLAs) require a paid LiteLLM commercial license whose price is not published.
Models: —; count not published

Failover
Spend limits
Logs
Semantic cache
Guardrails

Best for Platform teams that want a free, self-hosted, maximally broad provider abstraction with virtual keys and budgets, at moderate request volumes.

LLM Gateway

Open source Managed or self-host AGPL-3.0

AGPL-licensed OpenAI-compatible gateway available as one self-hosted Docker image or a hosted service with credit fees.

US company

Cost above the model bill: 5% to top up; No markup on tokens, fee applies when you add funds
Models: ~200; across ~40 providers

Failover
Spend limits
Logs
Caching
Guardrails

Best for Small teams that want an OpenRouter-style hosted gateway with a genuine zero-cost self-host escape hatch and no token markup.

MLflow AI Gateway

Open source Managed or self-host Apache-2.0

AI gateway that ships inside the MLflow Tracking Server: OpenAI-compatible routing over encrypted provider connections, with guardrails, budgets and every request landing as an MLflow trace.

US company

Cost above the model bill: No markup; You pay the model provider’s own prices
Models: —; across 14–100 providers

Failover
Spend limits
Logs
Guardrails

Best for Teams already running MLflow for tracing and evaluation who want governed, OpenAI-compatible LLM access on the same server — no second system to deploy, and every gateway request lands as a trace they can evaluate.

New API

Open source Self-host only AGPL-3.0 5 security incidents on record

Self-hosted Go gateway that fans 30+ upstream model providers into one OpenAI-compatible endpoint, with a full user, token and quota billing system bolted on.

Cost above the model bill: Self-hosted; You pay only for your own infrastructure and upstream provider spend: the software is AGPLv3 and "can be used for free" so long as the licence is observed, with a paid commercial licence available by email for AGPL-exempt use ([Project Introduction](https://www.newapi.ai/en/docs/guide/wiki/basic-concepts/project-introduction)). Production sizing implies a MySQL and a Redis alongside the gateway ([Cluster Deployment](https://www.newapi.ai/en/docs/installation/deployment-methods/cluster-deployment)).
Models: 100+; across 30+ providers

Failover
Spend limits
Logs
Guardrails

Best for Teams that want to self-host one multi-dialect gateway and also *bill* the people behind it - per-token quotas, groups, ratios, top-ups and subscription plans - with no vendor in the request path and no vendor fee ([Project Introduction](https://www.newapi.ai/en/docs/guide/wiki/basic-concepts/project-introduction), [Token Management](https://www.newapi.ai/en/docs/guide/feature-guide/user/token), [System Settings - Detailed](https://www.newapi.ai/en/docs/guide/feature-guide/admin/system-setting-advanced)).

The other four categories

Products are sorted by what they actually are, not by what they are marketed as. If none of the above is the shape of your problem, one of these probably is.

Is a self-hosted LLM gateway actually free?

There is no vendor fee, which is not the same thing. You still pay for the compute it runs on, the redundancy that keeps it available, and the engineering time to operate and upgrade it. The cost estimator on this site shows an infrastructure assumption for these products and flags it as an assumption rather than a quoted price.

Which open-source gateways can run air-gapped?

Several, but not all, and the difference is material for classified or regulated deployments. Air-gap support is a recorded field on every product page, taken from vendor documentation and linked to the page stating it. A blank means it was not published, not that the answer is no.

Does an open-source licence mean every feature is free?

No. Open-core products publish a genuine repository while keeping some capabilities — often the governance, SSO or audit features — in a paid tier. The licence field names the actual licence so you can tell a fully permissive project from an open-core one before you build on it.