Self-host only Apache-2.0

Apache APISIX AI Gateway Open source

Apache APISIX AI Gateway is an open-source LLM gateway with an OpenAI-compatible API in front of 10–20 upstream providers; it publishes no model count. It charges no token markup, credit-purchase fee or per-seat fee. It runs only self-hosted under Apache-2.0. It does not publish a HIPAA BAA. You can point it at your own provider accounts. Beyond chat it also serves embeddings. Published throughput is 18,000 requests per second.

· 29 of 113 fields dated · 43 sources

Built by Apache Software Foundation · US company

Compare with others Estimate cost Visit site ↗

Access at a glance

Whether this product can work for you at all, before features matter: who pays the model bill, where it can run, and which of your existing API calls keep working. Every value is the vendor’s own claim, linked to the page it came from.

Apache calls it "APISIX AI Gateway", "a complete AI gateway product" built on "a modern API gateway", with ai-proxy / ai-proxy-multi plugins that "simplify access to large language models (LLMs) and embedding models" (Introducing APISIX AI Gateway, 2025-04-08). Its own AI page calls it "an open-source LLM gateway and proxy" (APISIX AI Gateway). It is a self-hosted routing proxy, not a model platform.

Who pays the model bill

Your keys only

You contract with each model provider directly and hold those accounts. The gateway never resells inference.

Every example configures the customer's own provider credentials (OPENAI_API_KEY, DEEPSEEK_API_KEY, GCP service-account JSON, AWS keys) in the plugin's auth block (ai-proxy plugin, ai-proxy-multi plugin).

Merchant of record: The upstream provider, implicitly: the gateway is software the customer runs with its own provider keys, and no APISIX-side billing for model usage exists (ai-proxy plugin). API7 invoices only the gateway license, per CPU core (API7 pricing). An explicit statement of who invoices model usage: n.a.

Key handling: Keys live in the customer's own APISIX configuration, per provider instance via instances.auth.header / auth.query, with GCP service-account JSON in auth.gcp.service_account_json or the GCP_SERVICE_ACCOUNT environment variable; AWS secret_access_key and session_token are "stored encrypted" (ai-proxy plugin, ai-proxy-multi plugin). There is no vendor plane: request/response payload logging is opt-in via logging.payloads (with logging.summaries for model, duration and token counts) and goes to the operator's own http-logger/kafka-logger sinks; incoming client headers including Authorization and Cookie are forwarded upstream by default unless stripped (ai-proxy plugin).

Where it can run

3 of 5 shapes documented
  • Vendor-hosted
  • Self-host
  • Your VPC
  • On-premise
  • Air-gapped

Dimmed shapes are not documented by the vendor, which is not the same as unsupported.

self-host (open-source data plane) (APISIX AI Gateway); on-prem commercial edition via API7 (API7 pricing). API7 also markets a managed option — "the open-source & managed AI gateway" (API7 pricing). Hybrid/VPC and air-gapped: n.a. — The managed/SaaS shape is offered by API7, the commercial steward, not by the Apache APISIX project itself.

Configuration is applied to a locally run gateway through the APISIX Admin API (http://127.0.0.1:9180/apisix/admin/routes) and a config.yaml, reloaded after changes (ai-proxy-multi plugin). Commercial on-prem licensing is per gateway CPU core, annually: "Our on-premises API gateway pricing is based on API Gateway CPU cores" (API7 pricing).

API surfaces your code can keep using

3 of 7 documented, 1 partial
  • OpenAI chatPOST /v1/chat/completionsYes

    Routes proxy to the upstream /v1/chat/completions path and requests use the OpenAI messages array format (ai-proxy plugin, ai-proxy-multi plugin).

  • Anthropic messagesPOST /v1/messagesYes

    Setting the route URI to /v1/messages triggers automatic conversion between Anthropic Messages format and OpenAI-compatible format, including SSE streaming (ai-proxy plugin).

  • OpenAI ResponsesPOST /v1/responsesPartly

    Partially documented: openai-responses appears as a target protocol and in the max_tokens mapping table as "OpenAI Responses API", but no /v1/responses path is given (ai-proxy plugin).

  • EmbeddingsPOST /v1/embeddingsYes

    Documented with an override to https://api.openai.com/v1/embeddings using text-embedding-3-small (ai-proxy-multi plugin).

  • ImagesPOST /v1/images/generationsNot documented

    n.a. (no image endpoint documented on the pages fetched).

  • AudioPOST /v1/audio/*Not documented

    n.a. (no STT/TTS endpoint documented on the pages fetched).

  • Batch jobsPOST /v1/batchesNot documented

    n.a.

An asterisk marks a qualified verdict: support that is indirect (SDK compatibility or provider passthrough rather than a native endpoint), or a gap that is narrower or wider than the label suggests. Read the note before porting.

Not a base-URL-swap SaaS: you configure APISIX routes whose override.endpoint points at a provider, so clients call your gateway's route paths (ai-proxy-multi plugin). No proprietary SDK. Streaming caveats are explicit: max_stream_duration_ms can truncate an SSE stream without a protocol terminator such as [DONE], message_stop or response.completed, and Bedrock streaming returns AWS EventStream binary framing (Content-Type: application/vnd.amazon.eventstream) that clients must parse themselves; Bedrock routes must end in /converse (/model/<model>/converse-stream for streaming) (ai-proxy plugin).

How much it reaches

ModelsNot published
Upstream providers10–20vendor pages disagree

Providers: The vendor publishes different totals on different pages; both bounds are shown.

n.a.; no model total is stated, only examples such as gpt-4, deepseek-chat, text-embedding-3-small (ai-proxy-multi plugin, undated).

"20+ model providers" on the AI gateway page (APISIX AI Gateway, undated). The plugin reference enumerates 10 provider values — openai, deepseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, bedrock, openai-compatible — plus "other OpenAI-compatible APIs" (ai-proxy plugin, undated).

Whose models: All third-party: the gateway forwards to upstream provider APIs (OpenAI, DeepSeek, Azure OpenAI, Anthropic, OpenRouter, Gemini, Vertex AI, Bedrock, AIMLAPI) and ships no models of its own (ai-proxy plugin).

Your own endpoints: Yes: any endpoint can be set with override.endpoint, and it is required for the openai-compatible provider — "Custom LLM provider endpoint, required when provider is openai-compatible" (ai-proxy plugin). vLLM/Ollama/SageMaker by name: n.a.

How it behaves in production

What happens when an upstream model is slow, wrong, or down — and what you can see and stop while it happens. Reliability features are recorded as where you configure them, not whether the vendor lists them, because almost every product here lists all of them.

5 of 6 reachable from code3 of 4 can block8 documented destinations

When something goes wrong

Each row says where the knob is, not whether the feature is on the marketing page. A control you can only reach by hand in someone else’s dashboard cannot be reviewed or version-controlled.

  • Request timeoutIn config

    Changing it means editing configuration and shipping it, so behaviour is uniform across traffic until you redeploy.

    ai-proxy / ai-proxy-multi key timeout, integer milliseconds, range 1–60000, default 30000. Connection reuse is tuned with keepalive, keepalive_timeout (60000) and keepalive_pool (30) (APISIX ai-proxy-multi, APISIX ai-proxy).

  • RetriesIn config

    Retry/fallback ordering is driven by provider.priority; no retry-count key, default count or backoff strategy is documented on the fetched plugin pages: n.a. (APISIX ai-proxy-multi).

  • Fallback to another modelIn config

    ORDERED via provider.priority (higher-priority providers are tried first, lower priorities act as fallbacks); WEIGHTED distribution is separately available via provider.weight (APISIX ai-proxy-multi).

  • Load balancingIn config

    Weights supported: provider.weight, balancer.algorithm (roundrobin default, or chash) with balancer.hash_on / balancer.key (APISIX ai-proxy-multi). The repo describes round-robin "with weight", consistent-hash balancing, and custom algorithms in the balancer phase (apache/apisix).

  • Upstream health trackingIn config

    config_file, and it is both an upstream health check and unhealthy-node ejection: APISIX will "Enable health check on the upstream node and will automatically filter unhealthy nodes during load balancing", plus "Intelligent tracking of unhealthy upstream services" (circuit breaking) (apache/apisix). The AI Gateway landing page also advertises "health checks and weighted balancing" without naming keys (APISIX AI Gateway).

  • Cross-region failoverNot documented

    The vendor does not document this, so any behaviour you observe today is unversioned and may change.

    not_documented for provider failover. The only HA statement is about the control store: "Support to configure multiple etcd addresses in the same cluster" (apache/apisix).

Fallback chain: Weighted split — Traffic splits by percentage across targets, so you can shift 5% to a new model and watch it before committing.

The published QPS/latency numbers describe APISIX as a general API gateway, not as an AI proxy carrying token streams; do not read them as LLM gateway overhead (apache/apisix).

How fast the hop is

Interpreted proxy

Runs on an interpreted or JIT runtime (Lua, Python, Node). Overhead is higher than a compiled binary and more sensitive to concurrency, though a Lua-on-nginx proxy and a Python one are far apart.

Repo language breakdown Lua 81.6% (OpenResty-style plugin runtime; Java/Go/Python/Node.js plugins run out-of-process via RPC, and Wasm is experimental) (apache/apisix).

You can run the request path yourselfYes
StreamingPartly

Runs "from bare-metal to Kubernetes", official Helm charts installed with helm repo add apisix https://apache.github.io/apisix-helm-chart then helm install apisix apisix/apisix (apache/apisix, APISIX Helm chart).

Streaming caveats: SSE handling is documented for the mcp-bridge plugin, which converts stdio MCP servers into "scalable HTTP SSE services"; no LLM-streaming caveat is stated on the fetched AI plugin pages (apache/apisix).

Published figures, grouped by what each one measured. Figures in different groups are different quantities and cannot be compared with one another — nor, in most cases, with another vendor’s figure in the same group.

Overhead added by the gateway

The only figures that speak to whether this product slows your application down. Still not comparable between vendors: each was measured on different hardware, at a different load, with a different payload.

  • 0.2 msmeanVendor-published

    18k QPS on a single core. General API-gateway benchmark, NOT measured on LLM proxying; payload and cache state not stated.

    Source
  • 0.2 msmeanVendor-published

    140,000 QPS on an eight-core AWS server. Again general gateway traffic, not LLM proxying.

    Source

Sustained capacity

Requests or queries per second sustained on the stated hardware.

  • 18,000 QPSsustainedVendor-published

    Single core, general gateway traffic rather than LLM proxying.

    Source
  • 140,000 QPSsustainedVendor-published

    Eight-core AWS server, general gateway traffic.

    Source

Vendor/project self-published (Apache APISIX repo). No independent third-party benchmark and no claims about competitors on the fetched pages (apache/apisix).

What it will stop

3 of 4 can block

Two separate questions per control: can it stop a request at all, and what does it do before you change any settings? A control that inspects and forwards is a logging feature, however it is named.

  • Personal data in promptsNot documented

    No PII/DLP plugin on the fetched AI plugin pages (AI Gateway, ai-proxy)

  • Prompt injection and jailbreaksCan block the request

    Out of the box: Blocks out of the box

    ai-prompt-guard matches allow/deny regex patterns against prompts and rejects with HTTP 400; detection is regex, not a classifier (ai-prompt-guard)

  • Harmful contentCan block the request

    Out of the box: Blocks out of the box

    ai-aws-content-moderation sends the request body to AWS Comprehend and rejects when a category or the overall toxicity threshold is exceeded, returning 400 Bad Request with e.g. "request body exceeds PROFANITY threshold" (ai-aws-content-moderation)

  • Your own policiesCan block the request

    Out of the box: You pick the action when configuring

    Policy form is PCRE-style allow/deny pattern lists in ai-prompt-guard, plus per-category numeric moderation thresholds (ai-prompt-guard, ai-aws-content-moderation)

Where checks runEither, your choice
If the guardrail itself failsNot documented

Almost no vendor in this catalogue documents what happens when the guardrail service itself times out. If the control matters to you, this is a question worth asking before you sign.

not_documented (ai-aws-content-moderation)

Calls out to: AWS Comprehend. Each is a separate vendor relationship and a separate hop on the request path.

APISIX is the most privacy-default of the group: with payload and summary logging both off by default, an operator must deliberately turn on observability, which is the inverse of most hosted gateways (ai-proxy).

What you can see

Exports widely
What gets loggedMetadata only

Token counts, latency and model names are stored, but not the text itself.

You can turn bodies offYes

logging.payloads: false (the default) disables body logging while logging.summaries: true can keep token statistics (ai-proxy)

TracesOpenTelemetry

OpenTelemetry natively via the opentelemetry plugin (binary OTLP over HTTP, configurable sampler, optional full request-lifecycle spans); the plugin is disabled by default. Multi-step agent traces are not modelled — spans are HTTP-request-scoped (OpenTelemetry plugin)

metadata_only — and in fact less: in ai-proxy, logging.summaries and logging.payloads both default to false, so neither token summaries nor request/response bodies are logged unless enabled (ai-proxy)

Where telemetry can go

  • OpenTelemetry
  • Prometheus
  • Kafka
  • HTTP logger
  • Loki
  • Datadog
  • Splunk
  • Google Cloud Logging

OTLP/HTTP collectors for traces (OpenTelemetry plugin); http-logger and kafka-logger for AI request logs, with the llm_content_risk_level variable available to loggers (ai-proxy)

Records user feedbackNo
Scores live trafficNo

n.a. (AI Gateway)

n.a. (AI Gateway)

Depends on the vendor’s SaaS: No — everything runs and exports from the customer's own deployment (OpenTelemetry plugin)

Retention: n.a. — determined by the customer's log sink (ai-proxy)

Whether it fits how you work

How much work stands between you and a first call, how different that is from running it in production, and whether it slots into the stack you already have. Recorded as the shape of the work rather than a number of minutes — how long it takes you depends on which accounts and quota you already hold, which no comparison can know.

to try: cli or containerto run: infrastructure rolloutfits 5 of 10 common stacks

Getting to a first call

No numbered procedure published
Shape of the workRun something locally first

Nothing works until you have a process running. Fine on a laptop, but it means there is no zero-install way to try it.

Read off: the vendor’s own quickstart — no numbered procedure published.

Why the count is not the work: The getting-started page is prose plus commands, and its verification call is not an LLM completion. The actual LLM call lives on the ai-proxy plugin reference page.

Before step one

  • Your own provider keyRequired

    You need an upstream provider account and key before anything works. That is a prerequisite, not a step.

    Effectively yes: the ai-proxy examples configure provider API keys or AWS credentials (Bedrock "requires AWS SigV4 credentials and a non-empty AWS region"), though the page does not state a general first-call rule (AI Proxy | APISIX)

  • Payment methodNot documented

    n.a. (not documented); open-source install pages state no payment requirement (Getting Started with Apache APISIX, Installation - Apache APISIX)

  • Gate before models answerNot documented

    The docs do not say, so budget for a surprise on the first model you actually want.

    n.a. (not documented) (AI Proxy | APISIX, Getting Started with Apache APISIX)

Everything you need first: Docker version 20.10 or later and curl; no account, credit card, cloud account, cluster, or licence. For an AI completion you additionally need a provider key — the plugin examples use OPENAI_API_KEY, DEEPSEEK_API_KEY, AZ_OPENAI_API_KEY, ANTHROPIC_API_KEY (Getting Started with Apache APISIX, AI Proxy | APISIX)

Copyable snippet: incomplete. Yes, but in two parts: the getting-started page verifies with curl "http://127.0.0.1:9080/get" (not a completion), while the ai-proxy page shows the LLM call curl "http://127.0.0.1:9080/anything" -X POST -H "Content-Type: application/json" -H "Host: api.openai.com" -d '{"messages":[{"role":"system","content":"You are a mathematician"},{"role":"user","content":"What is 1+1?"}]}' after you configure a Route with the ai-proxy plugin (Getting Started with Apache APISIX, AI Proxy | APISIX)

Running it in production

The same scale applied to the path the vendor recommends for production traffic. Kept separate from the quickstart because for several products here the two are barely related pieces of work.

Shape of the workDeploy it on your infrastructure

This is an infrastructure project, not an integration. Expect charts or Terraform, networking, secrets and someone who owns the deployment.

Getting to production is a step up in kind from the quickstart, not just more of the same.

What production needs: etcd as the configuration store (installed automatically by the Docker and Helm methods); Docker 20.10+ for the container path; "Production environments should enable Admin API authentication"; the quickstart containers use Docker host network mode. No database, Redis, ClickHouse, load balancer, control-plane account, licence, or S3 requirement is stated (Installation - Apache APISIX, Getting Started with Apache APISIX)

Can you run it yourself

Install command publishedInstall command published

There is a command you can copy and run, so you can evaluate the self-hosted path yourself today.

git clone https://github.com/apache/apisix-docker.git, cd apisix-docker/example, docker-compose -p docker-apisix up -d; ARM: docker-compose -p docker-apisix -f docker-compose-arm64.yml up -d; Helm: helm repo add apisix https://charts.apiseven.com, helm repo update, helm install apisix apisix/apisix --create-namespace --namespace apisix; RPM: sudo yum install apisix; DEB: sudo apt install -y apisix (Installation - Apache APISIX)

How it fits your stack

5 of 10

Each row is a thing you might already run. “With a caveat” means it works but not the way the vendor’s marketing implies — read the reason, because that is usually where the surprise lives.

  • With a caveatThe OpenAI SDKOpenAI-compatible paths are documented, but not as a general drop-in.
  • With a caveatThe Vercel AI SDKVia the OpenAI provider
  • NoCloudflare WorkersNo Workers guidance published.
  • FitsKubernetesapache/apisix-helm-chart; API7 ships api7/gateway from https://charts.api7.ai
  • NoTerraform or OpenTofuNothing published for Terraform.
  • FitsAn existing API gatewayThis is that gateway — AI traffic becomes a plugin, not a new hop.
  • FitsCloud IAM I already runReuses IAM roles, workload identity or managed identities.
  • FitsLangChain or LlamaIndexLangChain, LangGraph, LlamaIndex
  • FitsMCP servers to governActs as an MCP gateway or registry.
  • With a caveatNothing — plain Node or PythonYou have to run a process locally before any call works.

Reading this the other way round — pick what you already run and see every product scored against it.

The integration surfaces behind those answers

  • Vercel AI SDKVia the OpenAI provider

    Use the OpenAI provider pointed at a custom base URL. Works, but you lose provider-specific options.

    Named: @ai-sdk/openai with a custom baseURL

    Vercel AI SDK integration is documented using the npm packages ai and @ai-sdk/openai; it creates a provider with createOpenAI. The example uses the gateway base URL http://127.0.0.1:3000/v1. (API7 Vercel AI SDK integration)

  • Cloudflare WorkersNot documented

    No Workers guidance either way. If you are edge-first, verify fetch-only compatibility yourself.

    n.a. (not documented)

  • KubernetesOfficial Helm chart

    A named, published chart. You can read its values file before committing to anything.

    Named: apache/apisix-helm-chart; API7 ships api7/gateway from https://charts.api7.ai

    Apache APISIX documents a Kubernetes Ingress Controller and Helm Charts. The official Apache repository is apache/apisix-helm-chart, containing the Apache APISIX Helm Chart and Apache APISIX Ingress Controller Helm Chart; a separate API7 chart is named api7/gateway in the https://charts.api7.ai repository. (Apache APISIX Helm Charts repository)

  • TerraformNot documented

    No Terraform surface published. Configuration is API or dashboard work.

    n.a. (not documented)

  • Existing API gatewayIt is the API gateway

    This product is the gateway. If you already run it for your other APIs, AI traffic becomes a plugin rather than a new hop.

    Apache APISIX is itself an open-source API gateway and AI gateway, built on NGINX and etcd, with an AI-focused plugin ecosystem. Documented AI plugins include ai-proxy, ai-proxy-multi, ai-request-rewrite, ai-rate-limiting, ai-prompt-decorator, ai-prompt-template, ai-prompt-guard, ai-aws-content-moderation, and ai-rag. (Introducing Apache APISIX AI Gateway)

  • Cloud identityReuses your cloud identity

    Authenticate with the identity you already run — IAM roles, workload identity or managed identities. No long-lived key to rotate.

    Yes: the ai-proxy plugin documents Amazon Bedrock authentication with AWS IAM credentials and SigV4 signing and Vertex AI authentication with a GCP service-account JSON (auth.gcp.service_account_json); Azure OpenAI uses an api-key header (no Entra/managed identity documented). (API7/APISIX ai-proxy plugin reference)

  • MCPMCP gateway or registry

    It sits in front of many MCP servers and governs access to them. This is the shape you want if MCP sprawl is the problem you are solving.

    Apache APISIX documents MCP support through gateway proxying and plugins: it can proxy Streamable HTTP traffic between MCP clients and backend MCP servers, and the mcp-bridge plugin converts stdio-based MCP servers to scalable HTTP SSE services. APISIX-MCP is also documented as an open-source npm/GitHub MCP server, available with npx -y apisix-mcp, for managing APISIX through its Admin API. (Apache APISIX MCP Protocol AI Gateway guide)

Python frameworks
  • LangChain
  • LangGraph
  • LlamaIndex

Python integrations are documented for LangChain through langchain-openai and ChatOpenAI, and for LangGraph through langgraph. LlamaIndex is documented through llama-index-llms-openai and the OpenAIResponses class; OpenAILike is also mentioned for Chat Completions compatibility. (API7 LangChain integration)

First-party client libraries None — you call it with an OpenAI-compatible client in whatever language you like

The API7 integration catalog lists OpenAI SDK and Anthropic SDK integrations. The documented TypeScript integration uses ai, @ai-sdk/openai, and @ai-sdk/openai-compatible; official Apache APISIX client libraries and supported client languages are not otherwise documented. (API7 AI Gateway integrations)

Agent features: APISIX-MCP documentation describes Cursor Agent mode and YOLO Mode automatically invoking MCP tools, including plugin analysis and route-management tools such as get_plugins_list, get_plugin_schema, create_route, update_route, and get_route. It also states that the workflow supports task decomposition, closed-loop validation, and iterative optimization; gateway-native sessions, threads, and A2A are not documented. (APISIX and AI/MCP intelligent API management)

After install you must configure a Route plus ai-proxy with provider auth and model; Kubernetes examples require kubectl apply -f ai-proxy-ic.yaml and declarative config uses adc sync -f adc.yaml; logger examples require setting up Kafka. Non-container installs need apisix init then apisix start, with config in conf/config.yaml and optional systemd management (AI Proxy | APISIX, Installation - Apache APISIX)

Apache APISIX is open source, Apache 2.0 licensed, platform agnostic, and described as having no vendor lock-in, portable configuration, and standalone, cluster, bare-metal, and Kubernetes deployment options. Its ecosystem includes an Ingress Controller, Helm Charts, Docker, Java/Go/Python plugin runners, MCP components, and integrations with OpenAI, DeepSeek, OpenAI-compatible services, AWS Comprehend, Azure OpenAI, and Azure AI Search; API7 documentation also describes open-source gateway, cloud, and hybrid-cloud access. (Apache APISIX AI Gateway)

Silence in the docs: 4 of the integration questions on this card have no published answer either way. That is recorded as undocumented, not as a no — but it does mean you would be verifying it yourself.

What it does well

  • Fastest published data-plane numbers here: ~18k QPS per core at under 0.2 ms added latency
  • Fully Apache-2.0 with no paid tier, gated features or vendor account required
  • Mature routing primitives: round-robin/consistent-hash/semantic load balancing, retries with time windows, active health checks and fallback strategies
  • Token-level logging variables and token-based rate limiting for LLM traffic

Where it falls short

  • Not AI-first: no response or semantic caching, budgets, virtual keys or prompt management in the AI plugins
  • Only about ten provider integrations documented, versus 20-140 for LLM-native gateways
  • Plugin configuration is Lua/OpenResty-shaped and the published performance numbers are for generic proxying, not LLM paths
  • No compliance certifications, support SLA or hosted control plane from the project itself

Choose it when

Teams already running APISIX that want basic multi-provider LLM proxying, retries and token-based rate limiting without adding another gateway.

Look elsewhere when

You want LLM-native features like semantic caching, spend budgets, virtual keys or a managed control plane out of the box.

Open source: You run the gateway yourself. Full control over the data path and no per-request vendor fee, in exchange for operating it.

How hard is it to leave?

Derived from six published facts, not from an opinion. The weights are fixed and the same for every product — see the arithmetic.

100/100Easy to leave
Portability score breakdown for Apache APISIX AI Gateway
What helps you leave Points Source
Works with standard OpenAI codeSwitching away is a base-URL change rather than a rewrite of every call site.22/22 vendor page
No vendor-specific SDK requiredA proprietary client library spreads through your codebase and has to be torn out again.10/10 vendor page
Can use your own provider accountsYour keys and billing relationship stay yours, so removing the gateway does not cut off model access.20/20 vendor page
Can be self-hostedYou can run it yourself instead of accepting a pricing or policy change.20/20—
Configuration lives in version controlRouting and budget rules are a file you keep, not dashboard state you would have to rebuild.16/16 vendor page
Your request history can be exportedYou leave with your own logs instead of abandoning them.12/12 vendor page

All six inputs are published, so this is scored against the full 100. This measures technical switching cost only. It does not price the engineering time to re-test prompts against a different routing stack.

Full specification

Every field we track. Blank fields say "Not published" rather than "No" — we do not infer an absence from silence. Switch to Technical in the header for the precise field names and the low-level details.

Overview

What kind of product Category
Open source Not verified
Marketplaces resell many providers behind one key. Gateways add governance on top. Open-source projects you run yourself. Cloud platforms are hyperscaler surfaces. Inference providers host models on their own hardware.
Who runs it Deployment model
Self-host only Not verified
Managed means the vendor operates it. Self-host means you run it on your own infrastructure. Both means you can choose.
Licence Licence
Apache-2.0 Verified 3 days ago
Proprietary products cannot be inspected or forked. Open licences such as MIT and Apache-2.0 let you audit, modify, and run the code without permission.
Company Company
Apache Software Foundation Verified 3 days ago
The organisation that maintains the product.
Who you would be signing with Vendor status
Run by a software foundation Not verified
Whether the product is still an independent company, has been acquired, is a large cloud vendor’s product line, is run by a software foundation, or has been put into maintenance mode. Maintenance mode means bug fixes and security patches only — no new features.
Last shipped an update Latest release
2026-08-20 Not verified
The date of the most recent release or version tag. A product that has not shipped in a year is a different risk from one that shipped last week, regardless of what its marketing site says.
GitHub stars GitHub stars
16,800 Verified 3 days ago
A rough proxy for community size on open-source projects. Not a quality measure.

Cost

Markup on model prices Token markup
None Verified 3 days ago
How much the product adds on top of what the underlying model provider charges. Zero means you pay the same per-token price you would pay the model provider directly.
Fee to add funds Credit purchase fee
None Verified 3 days ago
A percentage charged when you top up your balance, separate from token prices. It is easy to miss because it does not appear on the per-token price list.
Monthly cost per person Seat fee
None Verified 3 days ago
A recurring per-user platform charge that applies regardless of how much you use the models.
Can use your own provider accounts BYOK supported
Yes Verified 3 days ago
Bring Your Own Key: you keep direct contracts with OpenAI, Anthropic and others, and the gateway only routes traffic. This preserves negotiated rates and committed-spend discounts.
Cost of using your own accounts BYOK terms
Provider credentials are configured in the plugin; the project charges nothing. Not verified
What the product charges to route traffic through your own provider keys.
Free tier Free tier
Entire project is free under Apache-2.0, including the ai-proxy and ai-proxy-multi AI gateway plugins. Verified 3 days ago
What you can do without paying, useful for evaluation.
Enterprise plan from Enterprise plan from
Not published Not verified
Annual entry price for the enterprise tier, where one is published or credibly reported.
Cost to run it yourself Self-host cost
No license cost at all; you run APISIX and its etcd control store on your own infrastructure. The docs are co-branded with API7, a commercial vendor offering a supported distribution, whose pricing is not published on these pages. Verified 3 days ago
What self-hosting actually costs once you account for infrastructure and any paid tier.
How the vendor makes money Pricing model
Open source, no paid tier Not verified
The shape of the vendor’s bill: does the routing layer charge a percentage on top of tokens, a flat monthly fee, both, neither (because inference is the product), or nothing at all (open source with no paid tier).
How pricing works, briefly Pricing model detail
Apache 2.0 OSS project — no ASF pricing page, tier list or paid SKU exists. Commercial support is sold by a separate vendor, API7.ai: API7 Cloud Standard is $2 per 1M API calls (floored to $0.01) + $250/gateway group/month + $10/service/month; API7 Enterprise is annual licensing by API gateway CPU core with a custom quote. Not verified
A one-paragraph description that covers the caveats a pricing category cannot: introductory rates, per-feature meters, tier gating, and pricing that resets on a specific date.
Minimum commitment Minimum commitment
None for OSS. API7 Enterprise is annual per-CPU-core licensing. Not verified
Whether the vendor requires a minimum contract term, a minimum spend, or a provisioned-capacity purchase to get its published rate.
Charges that fire after you go over an allowance Overage terms
None for OSS. API7 Cloud is pay-as-you-go 'with no usage limitation', so cost grows with volume rather than being an overage penalty. Not verified
The line items that scale with usage after an included allowance is exhausted — log storage, extra requests, per-feature meters, data export — which is where cost estimates usually go wrong.
Prompt cache offered Cache mechanism
No gateway-owned cache Not verified
Whether the gateway offers its own response cache, what kind of match it does (exact request, prefix, semantic), or simply passes provider caching through unchanged.
Discount on cached input Cache-read discount
Not published Not verified
How much cheaper cached tokens are than fresh input, when the vendor publishes a single figure. Bundled-inference clouds usually price this per model instead of as one number.
Premium on cache writes Cache-write premium
Not published Not verified
How much more the first write of a cached prefix costs versus a plain input token. A high write premium and a low hit rate can leave you paying more than you save, so this matters as much as the read discount.
Who captures the cache saving Cache economics
The AI Gateway overview and ai-rate-limiting plugin docs do not document a response cache or provider-cache passthrough — usage fields like cached_tokens are surfaced but no gateway-owned cache is described. APISIX never prices tokens, so any caching saving is 100% the customer's. Not verified
Whether the customer keeps the full saving from caching or the vendor captures part of it — and any conditions attached (write premium, storage fees, best-effort hits).
What you can split spend by Cost attribution
Token consumption trackable by Route, Service, Consumer, Consumer Group or custom attributes. Dollar cost attribution not documented. Not verified
The dimensions the vendor documents for splitting spend — per key, per user, per team, per tag, per customer. Matters if you need to chargeback internally or bill an end customer.
How you get cost data out Cost export
None stated. Observability comes from the plugin ecosystem. Not verified
The mechanisms the vendor publishes for exporting cost and usage data: CSV, an API, webhooks, S3, a data warehouse, or nothing at all. Any per-unit price is included.
Who pays the model bill BYOK mode
Your keys only Not verified
Whether you bring your own provider accounts (BYOK), buy inference from this vendor, or can do either. This is the single biggest commercial difference between these products: it decides who holds the contract with the model provider and who carries the spend.

Spend governance in detail

Seven signals matter when a bill starts to hurt: who can spend, how much, on what, and who gets paged when it goes wrong. Everything below is drawn from the vendor’s own pricing and docs pages — how we read these.

  • Virtual or scoped keysYes

    Keys that carry their own budget and rate-limit policy, so an intern experiment cannot spend against a production budget.

    Supported in effect via Consumer and Consumer Group objects, though not branded as virtual keys.

  • Budget caps per keyYes

    A dollar or token ceiling attached to an individual key. Where enforcement is soft, one over-limit request still completes before the block kicks in.

    Token-quota caps rather than dollar budgets. Pre-request. ai-rate-limiting plugin controls tokens consumed within a time frame with limit_strategy of total_tokens/prompt_tokens/completion_tokens or an expression. Default local policy keeps counters per node — effective quota scales with node count.

  • Budget caps per team or workspaceYes

    A ceiling applied at a higher scope than one key — a team, a workspace, a customer, or an entire environment.

    Scoped by Route, Service, Consumer, Consumer Group or custom attributes.

  • Rate limiting as a cost controlYes

    Configurable request-per-time-window caps. Platform-set rate limits do not count as spend controls; user-configurable ones do.

    Free, pre-request.

  • Model allowlistsNot published

    A policy that constrains which models a key or team can call, keeping expensive frontier models out of the wrong hands.

    Not stated (routing/fallback across providers is; model allowlisting is not).

  • Spend alertsNo

    Alerts fired as spend approaches a threshold. Alerts that only fire after the meter has rolled over are marked as such.

    Not supported / not stated.

  • Webhook notificationsNo

    Programmatic notifications on spend events, so budget breaches can page an on-call or open a ticket.

    Not supported / not stated.

Enforcement:Enforced before each request

Catalog

Models available Models available
Not published Not verified
How many models you can call, shown as a range because several vendors publish different totals on different pages. Vendor-reported either way, so counts are not directly comparable — some count every provider variant of the same model separately.
Model providers reachable Upstream providers
10–20 Verified 3 days ago
How many distinct model providers or labs you can reach, shown as a range where the vendor’s own pages disagree. More providers usually means better redundancy when one has an outage.
Works with standard OpenAI code OpenAI-compatible API
Yes Verified 3 days ago
If yes, you can usually switch to it by changing one base URL, and switch away just as easily. This is the main defence against lock-in.
OpenAI chat endpoint POST /v1/chat/completions
Yes Not verified
The endpoint almost every application ports first. "Not documented" means the vendor never states it, which is different from a documented no.
Anthropic messages endpoint POST /v1/messages
Yes Not verified
Whether Anthropic-shaped calls work without rewriting them. Several products support this only as SDK compatibility or provider passthrough rather than a native endpoint — the detail page says which.
OpenAI Responses endpoint POST /v1/responses
Partly Not verified
The newer stateful OpenAI surface. Support is much thinner across this market than chat completions.
Embeddings endpoint POST /v1/embeddings
Yes Not verified
Whether you can generate vectors through the same gateway, or need a second integration for retrieval workloads.
Image generation endpoint POST /v1/images/generations
Not documented Not verified
Whether image models are reachable through the same surface as text.
Audio endpoints POST /v1/audio/*
Not documented Not verified
Speech-to-text and text-to-speech. Frequently the first gap in an otherwise complete gateway.
Batch jobs endpoint POST /v1/batches
Not documented Not verified
Asynchronous bulk processing, usually at a discount. Commonly undocumented, and commonly the reason a migration stalls late.
Needs the vendor’s own code library Requires a vendor-specific SDK
No Verified 3 days ago
A proprietary client library spreads through your codebase and has to be torn out again if you leave. “No” is the better answer here, and it means the standard OpenAI client works.
You can export your request history Logs / usage data export
Yes Verified 3 days ago
Whether you can get your own request logs, traces, or usage records back out — through an API, a bulk export, or a download. Decides whether you leave with your history or abandon it.
Settings can live in version control Declarative config-as-code
Yes Verified 3 days ago
Whether routing, fallback, and budget rules can be declared in a file you keep in Git, rather than existing only as settings clicked into a hosted dashboard.
Embeddings Embeddings
Yes Verified 3 days ago
Text-to-vector models, needed for search and retrieval features.
Image generation Image generation
Not published Not verified
Whether image models are reachable through the same interface.
Speech and audio Speech and audio
Not published Not verified
Text-to-speech or transcription models through the same interface.
Video generation Video generation
Not published Not verified
Whether video models are reachable through the same interface.
Batch processing Batch processing
Not published Not verified
Submitting large jobs for cheaper, slower processing. Often 50% off for work that is not time-sensitive.

Routing & reliability

Uptime it promises in writing Contractual SLA uptime
Not published Not verified
The uptime percentage in a published, contractual service level agreement. A public status page is not an SLA — it reports what happened, it does not promise anything or pay you back when it breaks.
Automatic failover Automatic failover
Yes Verified 3 days ago
When a model provider goes down or rate-limits you, traffic moves to a backup automatically instead of returning errors to your users.
Load balancing Load balancing
Yes Verified 3 days ago
Spreads requests across several providers or keys to raise your effective rate limit.
Rule-based routing Conditional routing
Yes Verified 3 days ago
Send different requests to different models based on rules — for example a cheap model for free users and a strong model for paying ones.
Response caching Response caching
Not published Not verified
Reuses the answer when the exact same request comes in again, which cuts both cost and latency.
Similar-question caching Semantic cache
Not published Not verified
Reuses an answer when a new question means roughly the same thing as an earlier one. Saves far more than exact-match caching but can return subtly wrong answers if tuned loosely.
Where you set the timeout Request timeout surface
In config Not verified

`ai-proxy` / `ai-proxy-multi` key `timeout`, integer milliseconds, range `1`–`60000`, **default `30000`**. Connection reuse is tuned with `keepalive`, `keepalive_timeout` (`60000`) and `keepalive_pool` (`30`) ([APISIX ai-proxy-multi](https://apisix.apache.org/docs/apisix/3.12/plugins/ai-proxy-multi/), [APISIX ai-proxy](https://apisix.apache.org/docs/apisix/3.12/plugins/ai-proxy/)).

Where a request timeout can be set: per request, in a config file, in the vendor dashboard, or nowhere. Nine of the twenty products documented here do not describe a request timeout at all, so the worst case of a hung upstream call is unknowable from the docs.
Where you set retries Retry policy surface
In config Not verified

Retry/fallback ordering is driven by `provider.priority`; no retry-count key, default count or backoff strategy is documented on the fetched plugin pages: `n.a.` ([APISIX ai-proxy-multi](https://apisix.apache.org/docs/apisix/3.12/plugins/ai-proxy-multi/)).

Where retry count and backoff are configured. Worth knowing alongside billing: a retried streaming call can be charged more than once.
Where you set fallbacks Fallback surface
In config Not verified

ORDERED via `provider.priority` (higher-priority providers are tried first, lower priorities act as fallbacks); WEIGHTED distribution is separately available via `provider.weight` ([APISIX ai-proxy-multi](https://apisix.apache.org/docs/apisix/3.12/plugins/ai-proxy-multi/)).

Where the fallback chain is defined. Almost every product claims fallback; the useful question is whether you can change it from code or only by hand in a dashboard.
Shape of the fallback chain Fallback shape
Weighted split Not verified
An ordered list tries targets in sequence; a weighted split sends a percentage of traffic to each, which is what you need to trial a new model on 5% of requests. Weighted splits are much rarer than the marketing implies.
Upstream health tracking Health checks / circuit breaking
In config Not verified

`config_file`, and it is both an upstream health check and unhealthy-node ejection: APISIX will "Enable health check on the upstream node and will automatically filter unhealthy nodes during load balancing", plus "Intelligent tracking of unhealthy upstream services" (circuit breaking) ([apache/apisix](https://github.com/apache/apisix)). The AI Gateway landing page also advertises "health checks and weighted balancing" without naming keys ([APISIX AI Gateway](https://apisix.apache.org/ai-gateway/)).

Whether the product notices a failing upstream and stops sending traffic to it, and whether you can tune the thresholds. This is what turns a provider outage into a blip rather than a sustained error rate, and only four of the twenty expose it.
Cross-region failover you control Multi-region failover surface
Not documented Not verified

`not_documented` for provider failover. The only HA statement is about the control store: "Support to configure multiple etcd addresses in the same cluster" ([apache/apisix](https://github.com/apache/apisix)).

Whether you can define what happens when a region degrades. A vendor running many regions is not the same as a vendor letting you configure failover between them; only three document a user-controlled mechanism.
Where you set load balancing Load balancing surface
In config Not verified

Weights supported: `provider.weight`, `balancer.algorithm` (`roundrobin` default, or `chash`) with `balancer.hash_on` / `balancer.key` ([APISIX ai-proxy-multi](https://apisix.apache.org/docs/apisix/3.12/plugins/ai-proxy-multi/)). The repo describes round-robin "with weight", consistent-hash balancing, and custom algorithms in the `balancer` phase ([apache/apisix](https://github.com/apache/apisix)).

Where traffic distribution across upstreams or keys is configured.

Operations

Usage dashboards and logs Observability
Yes Verified 3 days ago
Built-in visibility into what was sent, what came back, what it cost, and how long it took.
Spending limits Budget controls
Not published Not verified
Hard caps that stop spend before it becomes a surprise invoice. The single most valuable control for a small team.
Rate limits Rate limits
Yes Verified 3 days ago
Caps on request volume per key or per user, useful for protecting against abuse and runaway loops.
Separate keys per team or app Virtual keys
Not published Not verified
Issue scoped keys with their own budgets and permissions so you can attribute cost and revoke access without rotating everything.
Prompt versioning Prompt management
Not published Not verified
Store and version prompts outside your code so they can be changed without a deploy.
Quality testing Evals
Not published Not verified
Built-in tooling to score model output against test cases, so you can tell whether a model swap made things better or worse.
MCP support MCP support
Yes Verified 3 days ago
Native support for the Model Context Protocol, the emerging standard for connecting models to external tools.
What gets logged Logged content
Metadata only Not verified

`metadata_only` — and in fact less: in `ai-proxy`, `logging.summaries` and `logging.payloads` both default to `false`, so neither token summaries nor request/response bodies are logged unless enabled ([ai-proxy](https://apisix.apache.org/docs/apisix/next/plugins/ai-proxy/))

Whether full prompts and responses are stored, only metadata, or nothing. Full-body logging is the most useful debugging feature here and the one most likely to need a conversation with your compliance team.
You can turn logging off Body-logging opt-out
Yes Not verified

`logging.payloads: false` (the default) disables body logging while `logging.summaries: true` can keep token statistics ([ai-proxy](https://apisix.apache.org/docs/apisix/next/plugins/ai-proxy/))

Whether prompt and response bodies can be suppressed while still keeping usage metrics. Nineteen of the twenty document a way to do this; the mechanisms range from a per-request header to an organisation-wide setting.
Traces you can take elsewhere Distributed tracing
OpenTelemetry Not verified

OpenTelemetry natively via the `opentelemetry` plugin (binary OTLP over HTTP, configurable sampler, optional full request-lifecycle spans); the plugin is disabled by default. Multi-step agent traces are not modelled — spans are HTTP-request-scoped ([OpenTelemetry plugin](https://apisix.apache.org/docs/apisix/plugins/opentelemetry/))

Whether the product emits OpenTelemetry, a proprietary format, or nothing. OpenTelemetry means the traces land in the tooling you already run instead of only in the vendor’s dashboard.
Where telemetry can go Export destinations
OpenTelemetry, Prometheus, Kafka, HTTP logger, Loki, Datadog, Splunk, Google Cloud Logging Not verified

OTLP/HTTP collectors for traces ([OpenTelemetry plugin](https://apisix.apache.org/docs/apisix/plugins/opentelemetry/)); `http-logger` and `kafka-logger` for AI request logs, with the `llm_content_risk_level` variable available to loggers ([ai-proxy](https://apisix.apache.org/docs/apisix/next/plugins/ai-proxy/))

Documented sinks for logs and metrics. This is a good proxy for how replaceable the vendor’s own dashboard is: around twenty destinations means you never have to depend on it, while a CSV download means you do.
Can record user feedback Feedback capture API
No Not verified

`n.a.` ([AI Gateway](https://apisix.apache.org/ai-gateway/))

Whether there is an API to attach a rating or score to a logged request, which is what lets production traffic feed quality work later.
Scores live traffic Online eval hooks
No Not verified

`n.a.` ([AI Gateway](https://apisix.apache.org/ai-gateway/))

Whether automated scorers can run against real production requests, rather than only against a test set you assemble yourself.

Performance

Delay it adds Proxy overhead
0.2 ms Verified 3 days ago
Extra time the product itself adds to each request, on top of however long the model takes. Usually irrelevant next to multi-second model latency, but it matters for high-volume or streaming-sensitive workloads.
Requests per second ceiling Throughput
18,000 rps Verified 3 days ago
Published sustained request rate before the product becomes the bottleneck. Only relevant at genuinely high volume.
What the request path runs on Architecture class
Interpreted proxy Not verified

Repo language breakdown Lua 81.6% (OpenResty-style plugin runtime; Java/Go/Python/Node.js plugins run out-of-process via RPC, and Wasm is experimental) ([apache/apisix](https://github.com/apache/apisix)).

The comparable way to talk about latency here. An edge worker, a compiled Go or Rust binary, and a Python proxy have different overhead floors no matter which figures each vendor publishes. Products that never disclose their runtime are recorded as undisclosed rather than assumed.
You can run the request path yourself Self-hostable data plane
Yes Not verified

Runs "from bare-metal to Kubernetes", official Helm charts installed with `helm repo add apisix https://apache.github.io/apisix-helm-chart` then `helm install apisix apisix/apisix` ([apache/apisix](https://github.com/apache/apisix), [APISIX Helm chart](https://apisix.apache.org/docs/helm-chart/apisix/)).

Whether the component that actually carries your prompts can run on your own infrastructure. Distinct from a vendor offering a self-hosted control plane while still proxying traffic through their network.
Streaming responses Streaming support
Partly Not verified

SSE handling is documented for the `mcp-bridge` plugin, which converts stdio MCP servers into "scalable HTTP SSE services"; no LLM-streaming caveat is stated on the fetched AI plugin pages ([apache/apisix](https://github.com/apache/apisix)).

Whether token-by-token streaming is documented. The caveats matter more than the yes: some products cannot cancel a stream without still being billed, and several timeout and fallback mechanisms stop applying once the first token has been sent.

Security & compliance

Does your prompt reach their servers Prompt transits vendor
No Not verified

Apache-licensed software you deploy yourself. There is no foundation-operated hosted gateway, so no third party receives your prompts.

Whether the text you send passes through this company’s own infrastructure. If it does, every other promise on this page is a policy commitment rather than a physical impossibility. Self-hosted products can answer no outright.
What they keep if you change nothing Logging default
Not applicable — you own the logs Not verified

Token usage goes to your own access logs and observability stack. Whatever it keeps is your decision.

Defaults matter more than options. A product that stores full prompts and replies unless you find the right header will have stored them by the time you read the docs.
How long they keep it Default content retention (days)
Not published Not verified

Determined entirely by the log store you point it at.

Default retention for request content, in days. Zero means nothing is kept. Read the note: several products keep nothing as a rule but make timed exceptions for abuse review or specific models.
Could they train on your prompts Training on customer data
Not applicable Not verified

The project ships software rather than a data-processing service; no vendor receives prompts.

Whether the vendor may use your prompts and outputs to train models. “Not published” means we could not find any position, which is not the same as a no — ask for it in writing.
Where it runs, and what you can pin Region and residency control
Anywhere you run it. There are no vendor-managed regions. Not verified
Which regions are offered and whether you can force processing to stay in one. A global endpoint that silently picks a region is a different compliance story from an endpoint you pin yourself.
Where safety filters run Guardrail execution location
In your own infrastructure Not verified

Prompt guards, content moderation and auditing run in your own gateway before traffic reaches a model, checking content against allow and deny patterns you define.

A filter that strips personal data only helps if it runs before the data leaves your boundary. If guardrails execute in the vendor’s cloud, the vendor has already received whatever you wanted redacted.
Who else touches the data Subprocessor list
Not published Not verified
The published list of third parties the vendor passes your data to. No list means you cannot know the full chain, which most data-protection agreements require you to.
SOC 2 audited SOC 2 audited
Not published Not verified
An independent audit of security controls. Enterprise buyers and their procurement teams routinely require it.
Will sign a HIPAA agreement HIPAA BAA
Not published Not verified
Required before you may send protected health information through the service. Without a signed BAA, healthcare data is off limits.
GDPR commitments GDPR commitments
Not published Not verified
Published data processing terms for handling personal data of people in the EU and UK.
Can keep data in the EU EU data residency
Not published Not verified
Requests can be processed inside the EU rather than routed to US infrastructure. Often the deciding constraint for European customers.
Does not retain your data Zero data retention
Not published Not verified
Prompts and responses are not stored after the request completes. Sometimes a paid add-on rather than the default.
Strips personal data PII redaction
Not published Not verified
Detects and removes identifiers such as names, emails, and card numbers before the request reaches the model provider.
Content guardrails Content guardrails
Yes Verified 3 days ago
Policy checks on inputs and outputs — blocking unsafe content, enforcing formats, or catching prompt-injection attempts.
Runs fully disconnected Air-gapped deployment
Not published Not verified
Can be deployed in a network with no internet access, which some regulated and defence environments require.
Blocks personal data in prompts PII / DLP enforcement
Not documented Not verified

No PII/DLP plugin on the fetched AI plugin pages ([AI Gateway](https://apisix.apache.org/ai-gateway/), [ai-proxy](https://apisix.apache.org/docs/apisix/next/plugins/ai-proxy/))

Whether personal data detection sits on the request path and can stop the call, merely inspects and forwards it, or is not documented. A control that only reports is a logging feature, not a policy control.
Blocks prompt injection Injection / jailbreak enforcement
Can block the request Not verified

`ai-prompt-guard` matches allow/deny regex patterns against prompts and rejects with HTTP 400; detection is regex, not a classifier ([ai-prompt-guard](https://apisix.apache.org/docs/apisix/next/plugins/ai-prompt-guard/))

Whether injection and jailbreak detection can stop a request. Most products offering this call a partner classifier rather than shipping their own.
Blocks harmful content Toxicity / moderation enforcement
Can block the request Not verified

`ai-aws-content-moderation` sends the request body to AWS Comprehend and rejects when a category or the overall toxicity threshold is exceeded, returning `400 Bad Request` with e.g. "request body exceeds PROFANITY threshold" ([ai-aws-content-moderation](https://docs.api7.ai/hub/ai-aws-content-moderation))

Whether hate, violence, sexual and self-harm categories are checked inline and can stop a request, in either direction.
Your own policy rules Custom policy hooks
Can block the request Not verified

Policy form is PCRE-style allow/deny pattern lists in `ai-prompt-guard`, plus per-category numeric moderation thresholds ([ai-prompt-guard](https://apisix.apache.org/docs/apisix/next/plugins/ai-prompt-guard/), [ai-aws-content-moderation](https://docs.api7.ai/hub/ai-aws-content-moderation))

Whether you can add your own rule — a regex, a webhook, or your own classifier — rather than choosing from the vendor library.
Where guardrails run Guardrail execution location
Either, your choice Not verified
Whether guardrail evaluation happens inside your infrastructure or on the vendor’s servers. This decides whether the prompt you are trying to protect leaves your network in order to be checked.
If the guardrail itself fails Guardrail failure mode
Not documented Not verified

`not_documented` ([ai-aws-content-moderation](https://docs.api7.ai/hub/ai-aws-content-moderation))

What happens when the guardrail service times out or errors: does the request proceed unchecked, or is it blocked? This is the worst-documented field in the entire catalogue — only two vendors state it plainly, which means most teams are running a control whose failure behaviour they cannot know.
Third-party guardrail vendors Guardrail integrations
AWS Comprehend Not verified
Named external guardrail services the product can call. A long list means the product is a router for policy engines rather than a policy engine itself — which also means another vendor bill and another hop.

Compliance evidence

Graded by how strong the evidence is, not whether the word appears on the vendor’s website. An audited report and a marketing claim are different things, and only one of them will satisfy your own auditor.

Nothing to certify. This is software you run yourself, so no vendor receives your data and compliance is inherited from whatever you deploy it on — your own certifications, not the project’s.

Vendor source

Fit & integration

Work to try it Evaluation work shape
Run something locally first Not verified
The shape of the work on the vendor’s own quickstart, from swapping one base URL through to deploying infrastructure. An ordinal class rather than a duration, because elapsed time depends on accounts and quota we cannot see.
Work to run it Production work shape
Deploy it on your infrastructure Not verified
The same scale applied to the vendor’s recommended production path. For several products this is much heavier than the quickstart, which is exactly why both are recorded.
Steps on the quickstart Numbered quickstart steps
0 Not verified

The getting-started page is prose plus commands, and its verification call is not an LLM completion. The actual LLM call lives on the ai-proxy plugin reference page.

A literal count of numbered steps on the vendor’s quickstart, recorded as evidence beside the work shape. Zero means the page publishes no numbered procedure at all. Large counts usually mean interleaved language tracks rather than more work.
Can you self-host it today Self-host install documentation
Install command published Not verified

`git clone https://github.com/apache/apisix-docker.git`, `cd apisix-docker/example`, `docker-compose -p docker-apisix up -d`; ARM: `docker-compose -p docker-apisix -f docker-compose-arm64.yml up -d`; Helm: `helm repo add apisix https://charts.apiseven.com`, `helm repo update`, `helm install apisix apisix/apisix --create-namespace --namespace apisix`; RPM: `sudo yum install apisix`; DEB: `sudo apt install -y apisix` ([Installation - Apache APISIX](https://apisix.apache.org/docs/apisix/installation-guide/))

Whether an install command is actually published. Several products advertise self-hosting while publishing no command to start from, which a plain yes/no would hide.
Works with the OpenAI SDK OpenAI SDK drop-in
Partly Not verified

An OpenAI SDK integration is listed, but generic drop-in compatibility is not explicitly documented. The Vercel AI SDK example configures a custom `baseURL` of `http://127.0.0.1:3000/v1` with `@ai-sdk/openai`. ([API7 Vercel AI SDK integration](https://docs.api7.ai/ai-gateway/integrations/frameworks/vercel-ai-sdk))

Whether an existing OpenAI-compatible client can be pointed at it by changing the base URL and key.
Vercel AI SDK support AI SDK provider package
Via the OpenAI provider Not verified

Vercel AI SDK integration is documented using the npm packages `ai` and `@ai-sdk/openai`; it creates a provider with `createOpenAI`. The example uses the gateway base URL `http://127.0.0.1:3000/v1`. ([API7 Vercel AI SDK integration](https://docs.api7.ai/ai-gateway/integrations/frameworks/vercel-ai-sdk))

Whether a first-party AI SDK provider package exists, or only a community package, a documented workaround, or the generic OpenAI provider pointed at a custom base URL.
Python framework integrations Documented Python frameworks
LangChain, LangGraph, LlamaIndex Not verified

Python integrations are documented for LangChain through `langchain-openai` and `ChatOpenAI`, and for LangGraph through `langgraph`. LlamaIndex is documented through `llama-index-llms-openai` and the `OpenAIResponses` class; `OpenAILike` is also mentioned for Chat Completions compatibility. ([API7 LangChain integration](https://docs.api7.ai/ai-gateway/integrations/frameworks/langchain))

LangChain, LangGraph, LlamaIndex and similar orchestration frameworks with a documented integration.
Callable from Cloudflare Workers Cloudflare Workers support
Not documented Not verified

n.a. (not documented)

Whether the docs show calling this product from your own Worker. Deliberately separated from the several products whose own gateway runs on Workers, which is a fact about their infrastructure and not about your edge compatibility.
Kubernetes install Helm chart availability
Official Helm chart Not verified

Apache APISIX documents a Kubernetes Ingress Controller and Helm Charts. The official Apache repository is `apache/apisix-helm-chart`, containing the Apache APISIX Helm Chart and Apache APISIX Ingress Controller Helm Chart; a separate API7 chart is named `api7/gateway` in the `https://charts.api7.ai` repository. ([Apache APISIX Helm Charts repository](https://github.com/apache/apisix-helm-chart))

Whether a named, published Helm chart exists, versus Helm being referenced with no chart named, versus generic cluster documentation that has nothing to do with this product.
Terraform support Terraform provider or modules
Not documented Not verified

n.a. (not documented)

Whether you can declare this in code: an official provider, official modules, resources inside a hyperscaler’s provider, a community provider, or only Terraform code shipped in a repo.
Reuses your cloud identity Cloud IAM reuse
Reuses your cloud identity Not verified

Yes: the `ai-proxy` plugin documents Amazon Bedrock authentication with AWS IAM credentials and SigV4 signing and Vertex AI authentication with a GCP service-account JSON (`auth.gcp.service_account_json`); Azure OpenAI uses an `api-key` header (no Entra/managed identity documented). ([API7/APISIX ai-proxy plugin reference](https://docs.api7.ai/hub/ai-proxy))

Whether you can authenticate with IAM roles, workload identity or managed identities instead of another long-lived API key. Distinguished from products that only accept static upstream provider credentials.
Fits behind your API gateway API gateway integration
It is the API gateway Not verified

Apache APISIX is itself an open-source API gateway and AI gateway, built on NGINX and etcd, with an AI-focused plugin ecosystem. Documented AI plugins include `ai-proxy`, `ai-proxy-multi`, `ai-request-rewrite`, `ai-rate-limiting`, `ai-prompt-decorator`, `ai-prompt-template`, `ai-prompt-guard`, `ai-aws-content-moderation`, and `ai-rag`. ([Introducing Apache APISIX AI Gateway](https://apisix.apache.org/blog/2025/04/08/introducing-apisix-ai-gateway/))

Whether AI traffic can go through a gateway you already run, and who documented that — the product’s vendor or the gateway’s.
MCP support MCP surface shape
MCP gateway or registry Not verified

Apache APISIX documents MCP support through gateway proxying and plugins: it can proxy Streamable HTTP traffic between MCP clients and backend MCP servers, and the `mcp-bridge` plugin converts stdio-based MCP servers to scalable HTTP SSE services. APISIX-MCP is also documented as an open-source npm/GitHub MCP server, available with `npx -y apisix-mcp`, for managing APISIX through its Admin API. ([Apache APISIX MCP Protocol AI Gateway guide](https://apisix.apache.org/learning-center/mcp-protocol-ai-gateway/))

Which kind of MCP support this is: a gateway that governs many MCP servers, a hosted MCP server you connect to, MCP tools accepted inside the completion API, or client tooling. These are different products behind one acronym.
Needs your own provider key Upstream provider key required
Required Not verified

Effectively yes: the ai-proxy examples configure provider API keys or AWS credentials (Bedrock "requires AWS SigV4 credentials and a non-empty AWS region"), though the page does not state a general first-call rule ([AI Proxy | APISIX](https://apisix.apache.org/docs/apisix/plugins/ai-proxy/))

Whether an upstream provider account and key must exist before your first call works. A prerequisite rather than a step, and it can differ between the hosted and self-hosted forms of the same product.
Gate before models work Model access gate
Not documented Not verified

n.a. (not documented) ([AI Proxy | APISIX](https://apisix.apache.org/docs/apisix/plugins/ai-proxy/), [Getting Started with Apache APISIX](https://apisix.apache.org/docs/apisix/getting-started/README/))

Whether an enablement click, a quota grant, a paid tier or an approval form stands between a valid key and a working model call.
Official client languages First-party client SDK languages
Not published Not verified

The API7 integration catalog lists OpenAI SDK and Anthropic SDK integrations. The documented TypeScript integration uses `ai`, `@ai-sdk/openai`, and `@ai-sdk/openai-compatible`; official Apache APISIX client libraries and supported client languages are not otherwise documented. ([API7 AI Gateway integrations](https://docs.api7.ai/ai-gateway/integrations))

Languages with a first-party client library. An empty list can still mean the product is usable from any language via an OpenAI-compatible SDK.

How pricing actually works

No license cost at all; you run APISIX and its etcd control store on your own infrastructure. The docs are co-branded with API7, a commercial vendor offering a supported distribution, whose pricing is not published on these pages.

Back to top ↑

Common questions

Answered from the fields above, so these move when the catalog moves. Every figure quoted here appears in the specification with its source.

Does Apache APISIX AI Gateway charge a markup on model prices?

Apache APISIX AI Gateway adds no percentage markup to model prices. Adding funds is free.

Can Apache APISIX AI Gateway be self-hosted?

Yes — self-hosting is the only way to run Apache APISIX AI Gateway; there is no vendor-hosted option. The licence is Apache-2.0.

Is Apache APISIX AI Gateway SOC 2 audited, and will it sign a HIPAA BAA?

Apache APISIX AI Gateway publishes neither a SOC 2 report nor a HIPAA business associate agreement. Each of these is linked to the vendor's own page in the compliance section below.

Does Apache APISIX AI Gateway retain your prompts?

Zero data retention does not apply to Apache APISIX AI Gateway: it runs inside your own infrastructure, so prompts never reach a vendor. Only metadata is logged — not prompt or response bodies. Logging can be turned off.

Can you use your own provider keys with Apache APISIX AI Gateway?

Yes. Apache APISIX AI Gateway can route through your own accounts with the underlying model providers, so inference is billed to you directly. Provider credentials are configured in the plugin; the project charges nothing.

How many models does Apache APISIX AI Gateway support?

Apache APISIX AI Gateway publishes no total model count. It reaches 10–20 upstream providers. No model count is published; models are whatever the configured upstream provider exposes.

Back to top ↑

What has changed here

  1. catalog entry catalog entry Not published Added to the catalog source ↗
See this in the full changelog Back to top ↑

Read the head-to-head

These pairs have a written verdict, not just a table.

Apache APISIX AI Gateway vs Kong AI Gateway Read the verdict →

Usually weighed against

Apache APISIX AI Gateway vs Bifrost Compare → Apache APISIX AI Gateway vs LiteLLM Compare → Apache APISIX AI Gateway vs LLM Gateway Compare →