Managed gateway Managed or self-host Open core

Portkey

Open-core AI gateway with a hosted control plane for observability, prompt management and governance.

Built by Portkey, Inc. (Palo Alto Networks), founded 2023 · US company

Compare with others Estimate cost Visit site ↗

Access at a glance

Whether this product can work for you at all, before features matter: who pays the model bill, where it can run, and which of your existing API calls keep working. Every value is the vendor’s own claim, linked to the page it came from.

Portkey calls itself a "unified interface for interacting with over 250 AI models" and an "AI Gateway" that sits in front of provider APIs, adding routing, fallbacks, guardrails and observability (Portkey: what is Portkey, Portkey AI Gateway docs). It is a routing/observability proxy in front of many providers, available as an open-source gateway and as managed SaaS (portkey-ai/gateway on GitHub).

Who pays the model bill

Your keys only

You contract with each model provider directly and hold those accounts. The gateway never resells inference.

On the evidence fetched: Portkey requests run on the customer's own provider credentials held as virtual keys (Portkey virtual keys, Portkey BYO-LLM docs); the pricing page states no token markup or credit purchase model (Portkey pricing).

Merchant of record: n.a. Portkey's pricing page does not state who invoices model usage, and lists no token markup or credit top-up fee (Portkey pricing); with BYOK the provider key is the customer's own (Portkey virtual keys).

Key handling: Upstream credentials live in Portkey's "secure vault" as virtual keys, "stored securely, never exposed in code", with organization-level management and sharing across workspaces (Portkey virtual keys, Portkey BYO-LLM docs). Data is AES-256 encrypted in transit and at rest, storage of request/response bodies can be disabled, and Portkey holds ISO 27001 and SOC 2 (Portkey AI Gateway docs). In hybrid/VPC mode "all prompts and responses stay in your VPC. Only metrics leave (no sensitive data)" (Portkey hybrid deployment, 2024-10-01).

Where it can run

3 of 5 shapes documented
  • Vendor-hosted
  • Self-host
  • Your VPC
  • On-premise
  • Air-gapped

Dimmed shapes are not documented by the vendor, which is not the same as unsupported.

hosted SaaS (Portkey docs); self-host / open-source gateway via npx or Docker (portkey-ai/gateway); hybrid / customer-VPC data plane with Docker + Helm on AWS, Azure, GCP and OpenShift (Portkey hybrid deployment, page dated 2024-10-01; Portkey hybrid on GCP). On-prem and air-gapped: n.a. (not documented on the pages fetched).

The OSS gateway runs locally with npx @portkey-ai/gateway exposing http://localhost:8787/v1 (portkey-ai/gateway). Enterprise hybrid uses the portkey/gateway-enterprise:latest Docker image, Helm v3.0+ on Kubernetes v1.24+, deployed into the customer's own VPC: "Deploy Portkey's data plane in your VPC… All prompts and responses stay in your VPC. Only metrics leave (no sensitive data)" (Portkey hybrid deployment, 2024-10-01). Hybrid/VPC is presented as the enterprise offering (Portkey hybrid on GCP).

API surfaces your code can keep using

5 of 7 documented
  • OpenAI chatPOST /v1/chat/completionsYes

    POST https://api.portkey.ai/v1/chat/completions, usable by pointing the OpenAI SDK's base URL at Portkey (Portkey AI Gateway docs).

  • Anthropic messagesPOST /v1/messagesYes

    /messages is listed among the endpoints Portkey exposes for provider integrations (Portkey BYO-LLM docs).

  • OpenAI ResponsesPOST /v1/responsesNot documented

    n.a. (not stated on the Portkey pages fetched in this session).

  • EmbeddingsPOST /v1/embeddingsYes

    /embeddings documented alongside /chat/completions and /completions (Portkey BYO-LLM docs).

  • ImagesPOST /v1/images/generationsYes

    Image generation is listed among the modalities the gateway supports (Portkey AI Gateway docs). Endpoint path: n.a.

  • AudioPOST /v1/audio/*Yes *

    Yes, both directions: text-to-speech and speech-to-text (plus realtime) are listed as supported modalities (Portkey AI Gateway docs). Endpoint paths: n.a.

  • Batch jobsPOST /v1/batchesNot documented

    n.a. (no batch endpoint stated on the pages fetched).

An asterisk marks a qualified verdict: support that is indirect (SDK compatibility or provider passthrough rather than a native endpoint), or a gap that is narrower or wider than the label suggests. Read the note before porting.

Drop-in base-URL swap: point an existing OpenAI-compatible client at https://api.portkey.ai/v1 and pass x-portkey-api-key; a first-party portkey_ai SDK also exists but is not required (Portkey AI Gateway docs). Requests select the upstream through a virtual key / provider slug, and sensitive headers can be forwarded "without processing" (Portkey BYO-LLM docs). Streaming caveats: n.a.

How much it reaches

Models250–2,300vendor pages disagree
Upstream providers40–48vendor pages disagree

Models: The vendor publishes different totals on different pages; both bounds are shown.

Providers: The vendor publishes different totals on different pages; both bounds are shown.

Vendor numbers disagree: "over 250 AI models" (Portkey docs, undated), "1,600+ LLMs" (Portkey hybrid deployment, 2024-10-01), "2,300+ models" (portkey-ai/gateway, undated).

Vendor numbers disagree: "45+ providers" (Portkey hybrid deployment, 2024-10-01) and "2,300+ models across 40+ providers" (portkey-ai/gateway, undated); the integrations index lists 48 provider cards (Portkey LLM integrations, undated).

Whose models: All third-party routed: Portkey owns no models and proxies to provider APIs plus privately hosted models (Portkey LLM integrations, Portkey BYO-LLM docs).

Your own endpoints: Yes: add a "Local/Privately hosted provider" in Model Catalog → Add Provider, pick a compatible provider API specification and enter a "Custom Host", or pass customHost per request (Portkey BYO-LLM docs). Named support for vLLM/Ollama/SageMaker specifically: n.a.

How it behaves in production

What happens when an upstream model is slow, wrong, or down — and what you can see and stop while it happens. Reliability features are recorded as where you configure them, not whether the vendor lists them, because almost every product here lists all of them.

4 of 6 reachable from code4 of 4 can block4 documented destinations

When something goes wrong

Each row says where the knob is, not whether the feature is on the marketing page. A control you can only reach by hand in someone else’s dashboard cannot be reviewed or version-controlled.

  • Request timeoutPer request

    Your application decides per call, so one noisy endpoint can have its own timeout without a redeploy.

    Also available:In config

    per_request and config_file. Header x-portkey-request-timeout, SDK parameter requestTimeout, and config key request_timeout in milliseconds, settable at strategy level or per target with nested inheritance (Portkey request timeouts).

  • RetriesIn config

    Changing it means editing configuration and shipping it, so behaviour is uniform across traffic until you redeploy.

    retry.attempts up to 5, default retried status codes [429, 500, 502, 503, 504] overridable via on_status_codes; exponential backoff 1s / 2s / 4s / 8s / 16s (Portkey automatic retries).

  • Fallback to another modelIn config

    ORDERED: strategy.mode: fallback with an ordered targets array; default trigger is any non-2xx response (Portkey fallbacks).

  • Load balancingIn config

    Weights supported: strategy.mode: loadbalance with per-target weight (default 1, 0 disables a target, weights are normalized) (Portkey load balancing).

  • Upstream health trackingNot documented

    The vendor does not document this, so any behaviour you observe today is unversioned and may change.

    No upstream health check, circuit breaker or provider-ejection key appears on the fetched configs/fallback/load-balancing pages (Portkey configs).

  • Cross-region failoverNot documented

    You can list targets that happen to be in different regions, but no cross-region failover feature is documented (Portkey fallbacks).

Fallback chain: Ordered list — Try A, then B, then C. Simple and predictable, but every failover is all-or-nothing.

Defaults: retry.attempts up to 5, default retried status codes [429, 500, 502, 503, 504] overridable via on_status_codes; exponential backoff 1s / 2s / 4s / 8s / 16s (Portkey automatic retries).

Nested timeout inheritance (strategy level vs target level) is the most granular timeout model in this set; combined with retry.attempts: 5 and exponential backoff, worst-case wall-clock time should be computed before setting client timeouts (Portkey request timeouts, Portkey automatic retries).

How fast the hop is

Interpreted proxy

Runs on an interpreted or JIT runtime (Lua, Python, Node). Overhead is higher than a compiled binary and more sensitive to concurrency, though a Lua-on-nginx proxy and a Python one are far apart.

interpreted_proxy (JS runtime). Repo is TypeScript 96.0% and ships as an npm-launched service (Portkey-AI/gateway).

You can run the request path yourselfYes
StreamingYes

npx @portkey-ai/gateway (npm package @portkey-ai/gateway), with documented Docker, Cloudflare, AWS, Azure, GCP and Kubernetes deployments (Portkey-AI/gateway).

Streaming caveats: Supported, with an important documented caveat: the request timeout does not trigger if at least one chunk has already arrived, so a stalled mid-stream response is not cut off by request_timeout (Portkey request timeouts).

Published figures, grouped by what each one measured. Figures in different groups are different quantities and cannot be compared with one another — nor, in most cases, with another vendor’s figure in the same group.

Overhead added by the gateway

The only figures that speak to whether this product slows your application down. Still not comparable between vendors: each was measured on different hardware, at a different load, with a different payload.

  • <1 msVendor-published

    README headline. No percentile, no RPS, no payload, no hardware, no cache state.

    Source

Marketing claim, not a measurement

Fleet totals and unquantified claims. Recorded here because it is all the vendor published, not because it means anything operationally.

  • over 10B tokens/dayfleet totalVendor-published

    Aggregate traffic across all customers, not a measured per-instance capacity.

    Source

Both figures are vendor self-published with no methodology; neither is a third-party benchmark and neither is a claim about a competitor (Portkey-AI/gateway).

What it will stop

4 of 4 can block

Two separate questions per control: can it stop a request at all, and what does it do before you change any settings? A control that inspects and forwards is a logging feature, however it is named.

These controls can block a request, but will not until you change their settings:

  • Personal data in prompts — Only logs until you change it
  • Prompt injection and jailbreaks — Only logs until you change it
  • Harmful content — Only logs until you change it
  • Your own policies — Only logs until you change it
  • Personal data in promptsCan block the request

    Out of the box: Only logs until you change it

    "Detect PII" is one of 20+ deterministic checks; guardrail actions default to async: TRUE (log only, no added latency), and only when a check is set synchronous and Deny: TRUE does Portkey return HTTP 446 and block. With Deny: FALSE the request passes through with a 246 status and the verdict recorded (Guardrail checks, Guardrails)

  • Prompt injection and jailbreaksCan block the request

    Out of the box: Only logs until you change it

    Injection/jailbreak detection comes from partner guardrail vendors rather than a built-in classifier, and inherits the same async/sync + Deny semantics (Guardrail checks, Guardrails)

  • Harmful contentCan block the request

    Out of the box: Only logs until you change it

    "Moderate Content" is a built-in check subject to the same async default (Guardrail checks, Guardrails)

  • Your own policiesCan block the request

    Out of the box: Only logs until you change it

    Policies take the form of Regex Match, Contains / word lists, JSON Schema and JSON Keys validation, Model Whitelist, and Webhook (custom classifier hook) (Guardrail checks)

Where checks runEither, your choice
If the guardrail itself failsNot documented

Almost no vendor in this catalogue documents what happens when the guardrail service itself times out. If the control matters to you, this is a question worth asking before you sign.

The docs describe deny/allow verdict semantics but not behaviour when the guardrail service itself errors or times out (Guardrails)

Calls out to: Acuvity, Aporia, AWS Bedrock Guardrails, Azure Content Safety, Javelin, Lasso, Mistral, Pangea, Palo Alto Prisma AIRS, Patronus AI, Pillar Security, Qualifire. Each is a separate vendor relationship and a separate hop on the request path.

The single most important default in this report: Portkey guardrails ship asynchronous, meaning a freshly configured "PII guardrail" annotates the log and lets the request through. Enforcement requires both switching the check to synchronous and setting Deny (Guardrails).

What you can see

Exports to a few places
What gets loggedYour choice

You decide whether bodies are captured, by setting or by header.

You can turn bodies offYes

Org/workspace-level Metrics Only mode (Request logging settings), or per request the DO NOT TRACK control: set debug: false on the client or send header x-portkey-debug: false, keeping tokens, cost, and latency (Logs)

TracesOpenTelemetry

OpenTelemetry export is supported in two modes: analytics-only spans without prompts, and complete logs including full prompt/completion (marked experimental); trace IDs group multi-step requests (Portkey OTel, Feedback)

Full Logging stores "Complete request payloads", "Full response content", and all metrics; Metrics Only (Privacy Mode) stores usage, metadata, and errors "without sensitive content" (Request logging settings). Clicking a log entry reveals "the entire raw data with the request and response objects" (Logs)

Where telemetry can go

  • OpenTelemetry
  • Datadog
  • Grafana
  • LangSmith

Any OTLP-compatible backend, with Datadog, Grafana, and LangSmith named explicitly (Portkey OTel)

Records user feedbackYes
Scores live trafficPartly

Yes — POST https://api.portkey.ai/v1/feedback with trace_id, value (−10…10), optional weight and metadata; the trace ID is returned in the x-portkey-trace-id response header (Feedback)

Feedback data feeds Analytics and a Prompt Eval tab inside Portkey; the same guardrail checks can run asynchronously over live traffic as scoring. Both require Portkey's platform (Feedback, Guardrails)

Depends on the vendor’s SaaS: Enterprise deployments can run the gateway in customer infrastructure and export telemetry via OTel, but the log/analytics UI is Portkey's control plane (Portkey OTel)

Retention: Developer 3 days (10k logs/month), Production 30 days (100k logs/month, $9 per additional 100k), Enterprise unlimited logs with retention unspecified (Logs)

Whether it fits how you work

How much work stands between you and a first call, how different that is from running it in production, and whether it slots into the stack you already have. Recorded as the shape of the work rather than a number of minutes — how long it takes you depends on which accounts and quota you already hold, which no comparison can know.

to try: sdk installto run: sdk installfits 6 of 10 common stacks

Getting to a first call

3 numbered steps
Shape of the workInstall a package

You add a dependency and construct the vendor’s client. Small, but it is a code change and a package you now maintain.

Read off: the vendor’s own quickstart — 3 numbered steps.

Before step one

  • Your own provider keyRequired

    You need an upstream provider account and key before anything works. That is a prerequisite, not a step.

    Not explicitly stated, but the documented snippets reference a provider: provider="@YOUR_PROVIDER" and examples using YOUR_OPENAI_API_KEY; the self-hosted validation step creates "a virtual key" before the first call (Docs - what is Portkey, Deploy Portkey in Your Infrastructure)

  • Payment methodNo card needed to start

    Not stated; "Portkey's Gateway is open source and free to use. On managed version, Portkey offers a free plan with 10k requests per month." (Docs - what is Portkey)

  • Gate before models answerNot documented

    The docs do not say, so budget for a surprise on the first model you actually want.

    n.a. (not documented). The only permissions wording is on the API-key modal: "Based on your access level, you might see the relevant permissions on the API key modal" (Make Your First Request - Portkey Docs); no approval gate in the private-deployment guide either (Deploy Portkey in Your Infrastructure)

Everything you need first: A Portkey account and an API key from the Settings page, with permissions ticked on the API-key modal. No credit card, cloud account, cluster, or provider key stated (Make Your First Request - Portkey Docs)

Copyable snippet: incomplete. Yes, on the overview page: Portkey(api_key="YOUR_PORTKEY_API_KEY", provider="@YOUR_PROVIDER") then portkey.chat.completions.create(model="gpt-3.5-turbo", messages=[...]) and print(chat_complete.choices[0].message.content) (Docs - what is Portkey). The make-your-first-request page names the integration options but rendered no code block in the fetched content (Make Your First Request - Portkey Docs)

The vendor’s own time claim: Vendor claims, verbatim: "It takes 2 mins to integrate" (Docs - what is Portkey); page description "Integrate Portkey and analyze your first LLM call in 2 minutes!" (Make Your First Request - Portkey Docs); and for private deployment "Get your AI gateway running in 4 simple steps. Most teams finish in under 2 hours," with a breakdown of Total 2 hours / Prerequisites 5 min / Credentials 10 min / Deployment 30 min / Validation 15 min (Deploy Portkey in Your Infrastructure) Quoted, not verified. Marketing time claims assume every account and approval is already in place.

Running it in production

The same scale applied to the path the vendor recommends for production traffic. Kept separate from the quickstart because for several products here the two are barely related pieces of work.

Shape of the workInstall a package

You add a dependency and construct the vendor’s client. Small, but it is a code change and a package you now maintain.

If you self-host it insteadDeploy it on your infrastructure

This is an infrastructure project, not an integration. Expect charts or Terraform, networking, secrets and someone who owns the deployment.

Trying this and running it are almost unrelated pieces of work. The quickstart is not a preview of the production effort.

What production needs: Kubernetes v1.24+, Helm v3.0+, outbound internet access returning HTTP/2 200 from https://control.portkey.ai, a storage bucket (s3, gcs, mongo, or wasabi), an auto-deployed Redis cache, Portkey control-plane access with 1Password credentials, Docker registry username/password plus a gateway JWT, an ingress with class nginx, and a Portkey dashboard account at app.portkey.ai (Deploy Portkey in Your Infrastructure)

Can you run it yourself

Install command publishedInstall command published

There is a command you can copy and run, so you can evaluate the self-hosted path yourself today.

docker pull portkey/gateway-enterprise:latest; helm repo add portkey https://portkey-ai.github.io/helm; helm repo update; helm upgrade --install portkey-gateway portkey/gateway -n portkey -f values.yaml (plus kubectl create namespace portkey and an image-pull secret) (Deploy Portkey in Your Infrastructure)

How it fits your stack

6 of 10

Each row is a thing you might already run. “With a caveat” means it works but not the way the vendor’s marketing implies — read the reason, because that is usually where the surprise lives.

  • FitsThe OpenAI SDKDrop-in once set up — but first-call work is sdk install.
  • FitsThe Vercel AI SDK@portkey-ai/vercel-provider
  • NoCloudflare WorkersTheir own gateway runs on Workers. That is not the same as edge support for your code.
  • FitsKubernetescharts/portkey-gateway and charts/portkey-app in Portkey-AI/helm
  • FitsTerraform or OpenTofuPortkey-AI/portkey
  • NoAn existing API gatewayNothing published about running behind your gateway.
  • NoCloud IAM I already runNo identity integration published.
  • FitsLangChain or LlamaIndexLangChain, LlamaIndex
  • FitsMCP servers to governActs as an MCP gateway or registry.
  • With a caveatNothing — plain Node or PythonInstall a package, but an upstream provider key is a prerequisite.

Reading this the other way round — pick what you already run and see every product scored against it.

The integration surfaces behind those answers

  • Vercel AI SDKOfficial provider package

    Install the package, swap the model factory, done. Maintained by a party with a stake in it.

    Named: @portkey-ai/vercel-provider

    Portkey provides a Vercel AI SDK integration. The documentation names the provider package @portkey-ai/vercel-provider and uses createPortkey. (Portkey Vercel AI SDK integration)

  • Cloudflare WorkersTheir gateway runs on Workers, not yours

    The vendor runs its own gateway on Workers. That is a fact about their infrastructure and says nothing about calling them from your Worker — it usually means low latency to their edge, not edge compatibility for you.

    The Portkey Gateway runs on Cloudflare Workers and uses Cloudflare’s global edge network and data centers, according to a Portkey blog post. (Why Portkey chose TypeScript for its AI Gateway)

  • KubernetesOfficial Helm chart

    A named, published chart. You can read its values file before committing to anything.

    Named: charts/portkey-gateway and charts/portkey-app in Portkey-AI/helm

    Portkey provides official Kubernetes Helm charts in the Portkey-AI/helm repository: charts/portkey-gateway for Portkey Hybrid (Gateway) and charts/portkey-app for Portkey Full (Full App). (Portkey official Helm charts)

  • TerraformOfficial Terraform provider

    You can manage this product as first-class Terraform resources, versioned on the Registry.

    Named: Portkey-AI/portkey

    Portkey provides an official Terraform provider named Portkey-AI/portkey on the Terraform Registry. (Portkey Terraform provider)

  • Existing API gatewayNot documented

    Nothing published about sitting behind an existing gateway. Treat it as a separate hop you route to yourself.

    n.a. (not documented)

  • Cloud identityNot documented

    No identity integration published. Expect API keys in a secret store.

    n.a. (not documented)

  • MCPMCP gateway or registry

    It sits in front of many MCP servers and governs access to them. This is the shape you want if MCP sprawl is the problem you are solving.

    Portkey documents an MCP Gateway that centralizes authentication and team/user access control for MCP servers and tools, logs tool calls, and supports identity forwarding. It works with Claude Desktop, Cursor, VS Code, and other MCP-compatible clients. (Portkey Gateway GitHub repository)

Python frameworks
  • LangChain
  • LlamaIndex

Portkey documents integrations with LangChain and LlamaIndex. LangChain examples use langchain, langchain-openai, ChatOpenAI, createHeaders, and LangchainCallbackHandler; LlamaIndex examples use llama_index.llms.openai.OpenAI, ChatMessage, and portkey-ai. (Portkey LangChain agents integration)

First-party client libraries
  • Python
  • JavaScript

Official examples use the OpenAI SDK and Portkey SDKs: Python packages openai and portkey-ai/portkey_ai, JavaScript packages openai and portkey-ai, and the Python Portkey class. Portkey also documents the Anthropic package anthropic for the Messages API. (Portkey AI Gateway getting started)

Agent features: Portkey supports function calling and tool use. Its Responses API documents built-in tool use and reasoning, including web search, file search, Computer Use Assistant, and Remote MCP; its Assistants API documents persistent threads. (Portkey universal API)

Self-hosted validation ends with a health check, creating a virtual key, calling https://gateway.internal.mycompany.com/v1/chat/completions with x-portkey-api-key, and viewing logs/analytics; port-forward for testing via kubectl -n portkey port-forward svc/portkey-gateway 8787:8787. Portkey adds "20-40ms" latency per its own docs. Native integrations exist for OpenAI, Azure OpenAI, Anthropic, Langchain, LlamaIndex and Ollama (Deploy Portkey in Your Infrastructure, Docs - what is Portkey, Make Your First Request - Portkey Docs)

Portkey documents both the hosted gateway URL https://api.portkey.ai/v1 and local/self-hosted use at http://localhost:8787/v1; PORTKEY_API_KEY can be skipped when self-hosting. It states that the gateway supports 30+ providers and all models within those providers. (Portkey AI Gateway getting started)

Silence in the docs: 3 of the integration questions on this card have no published answer either way. That is recorded as undocumented, not as a no — but it does mean you would be verifying it yourself.

What it does well

  • MIT-licensed gateway core covering routing, fallbacks, conditional routing, semantic cache, budgets and guardrails
  • Full governance stack: RBAC, SSO/SCIM, audit logs, budget and rate limits, PII-redaction guardrails
  • SOC 2, HIPAA, GDPR and ISO 27001 compliance claims plus custom BAAs
  • MCP registry with OAuth 2.1 (PKCE) in the open-source gateway
  • Cheap entry point: free Developer tier, $49/month Production

Where it falls short

  • Pricing is metered on recorded logs, not tokens, so high-request workloads hit $9 per extra 100k requests
  • No public model-catalog endpoint - the /v1/models call requires an API key
  • Self-hosting loses the hosted dashboards, prompt management and guardrails UI
  • Third-party reviews note slower development cadence and a smaller self-hosting community than LiteLLM

Choose it when

Teams that want governance, guardrails and prompt management in one control plane, with the option to self-host the routing engine.

Look elsewhere when

You mainly need a thin, zero-config marketplace router and do not want to run or pay for a control plane.

Managed gateway: A hosted control plane in front of the models: routing, caching, guardrails, spend limits, and logs. You trade a little simplicity for governance.

How hard is it to leave?

Derived from six published facts, not from an opinion. The weights are fixed and the same for every product — see the arithmetic.

100/100Easy to leave
Portability score breakdown for Portkey
What helps you leave Points Source
Works with standard OpenAI codeSwitching away is a base-URL change rather than a rewrite of every call site.22/22 vendor page
No vendor-specific SDK requiredA proprietary client library spreads through your codebase and has to be torn out again.10/10 vendor page
Can use your own provider accountsYour keys and billing relationship stay yours, so removing the gateway does not cut off model access.20/20 vendor page
Can be self-hostedYou can run it yourself instead of accepting a pricing or policy change.20/20 vendor page
Configuration lives in version controlRouting and budget rules are a file you keep, not dashboard state you would have to rebuild.16/16 vendor page
Your request history can be exportedYou leave with your own logs instead of abandoning them.12/12 vendor page

All six inputs are published, so this is scored against the full 100. This measures technical switching cost only. It does not price the engineering time to re-test prompts against a different routing stack.

Full specification

Every field we track. Blank fields say "Not published" rather than "No" — we do not infer an absence from silence. Switch to Technical in the header for the precise field names and the low-level details.

Overview

What kind of product Category
Managed gateway Not verified
Marketplaces resell many providers behind one key. Gateways add governance on top. Open-source projects you run yourself. Cloud platforms are hyperscaler surfaces. Inference providers host models on their own hardware.
Who runs it Deployment model
Managed or self-host Verified 2 days ago
Managed means the vendor operates it. Self-host means you run it on your own infrastructure. Both means you can choose.
Licence Licence
Open core Verified 2 days ago
Proprietary products cannot be inspected or forked. Open licences such as MIT and Apache-2.0 let you audit, modify, and run the code without permission.
Company Company
Portkey, Inc. (Palo Alto Networks) Not verified
The organisation that maintains the product.
Who you would be signing with Vendor status
Acquired Not verified
Whether the product is still an independent company, has been acquired, is a large cloud vendor’s product line, is run by a software foundation, or has been put into maintenance mode. Maintenance mode means bug fixes and security patches only — no new features.
Last shipped an update Latest release
2026-08-21 Not verified

Enterprise Gateway v2.19.0. The open-source repository is far staler: last tag v1.15.2 on 12 January 2026.

The date of the most recent release or version tag. A product that has not shipped in a year is a different risk from one that shipped last week, regardless of what its marketing site says.
GitHub stars GitHub stars
12,848 Verified 2 days ago
A rough proxy for community size on open-source projects. Not a quality measure.

Cost

Markup on model prices Token markup
Not published Not verified
How much the product adds on top of what the underlying model provider charges. Zero means you pay the same per-token price you would pay the model provider directly.
Fee to add funds Credit purchase fee
Not published Not verified
A percentage charged when you top up your balance, separate from token prices. It is easy to miss because it does not appear on the per-token price list.
Monthly cost per person Seat fee
Not published Not verified
A recurring per-user platform charge that applies regardless of how much you use the models.
Can use your own provider accounts BYOK supported
Yes Verified 2 days ago
Bring Your Own Key: you keep direct contracts with OpenAI, Anthropic and others, and the gateway only routes traffic. This preserves negotiated rates and committed-spend discounts.
Cost of using your own accounts BYOK terms
Requests use your own provider credentials via provider integrations; enterprise deployments can also bring their own KMS encryption keys. Verified 2 days ago
What the product charges to route traffic through your own provider keys.
Free tier Free tier
Developer plan free: 10,000 recorded logs/month, 3-day log and 30-day metric retention, universal API, fallbacks, load balancing, retries, 3 prompt templates, simple caching, deterministic guardrails. Verified 2 days ago
What you can do without paying, useful for evaluation.
Enterprise plan from Enterprise plan from
Not published Not verified
Annual entry price for the enterprise tier, where one is published or credibly reported.
Cost to run it yourself Self-host cost
The gateway core is MIT-licensed and free to self-host (you pay only your own infrastructure and model providers); the Gateway 2.0 release moved circuit breakers, semantic cache, budget limits, model catalog, metadata governance and config management into open source. Self-hosting drops the hosted dashboards/control plane, and an On-Prem Enterprise Gateway (gRPC, SSO, SCIM, AWS KMS, RBAC, JWT, audit logs, multi-workspace) is sold separately. Verified 2 days ago
What self-hosting actually costs once you account for infrastructure and any paid tier.
How the vendor makes money Pricing model
Platform fee plus usage meters Not verified
The shape of the vendor’s bill: does the routing layer charge a percentage on top of tokens, a flat monthly fee, both, neither (because inference is the product), or nothing at all (open source with no paid tier).
How pricing works, briefly Pricing model detail
Free OSS self-host + free Dev tier + flat monthly platform fee ($49/mo Pro per docs) + request overage + Enterprise quote. Public pricing page shows tiers without dollar amounts and describes the free allowance as '10k recorded logs' per month; docs comparison lists 'Pro $49/Month' with '10k requests/month'. These two framings contradict. Not verified
A one-paragraph description that covers the caveats a pricing category cannot: introductory rates, per-feature meters, tier gating, and pricing that resets on a specific date.
Minimum commitment Minimum commitment
Not stated. Not verified
Whether the vendor requires a minimum contract term, a minimum spend, or a provisioned-capacity purchase to get its published rate.
Charges that fire after you go over an allowance Overage terms
Dev 10K requests/mo with 'No Overage Allowed' (docs) — but pricing page says exceeding the log limit only stops recording, not requests. Pro 100K then $9/mo per additional 100K up to 3M requests. Enterprise custom. Retention-overage price and export cost not stated. Not verified
The line items that scale with usage after an included allowance is exhausted — log storage, extra requests, per-feature meters, data export — which is where cost estimates usually go wrong.
Prompt cache offered Cache mechanism
Both exact and semantic Not verified
Whether the gateway offers its own response cache, what kind of match it does (exact request, prefix, semantic), or simply passes provider caching through unchanged.
Discount on cached input Cache-read discount
Not published Not verified
How much cheaper cached tokens are than fresh input, when the vendor publishes a single figure. Bundled-inference clouds usually price this per model instead of as one number.
Premium on cache writes Cache-write premium
Not published Not verified
How much more the first write of a cached prefix costs versus a plain input token. A high write premium and a low hit rate can leave you paying more than you save, so this matters as much as the read discount.
Who captures the cache saving Cache economics
Exact/'simple' caching on all tiers; semantic caching from the paid tier (Production/Pro). No cached-token pricing published. Semantic cache savings are paid-tier-only. Not verified
Whether the customer keeps the full saving from caching or the vendor captures part of it — and any conditions attached (write premium, storage fees, best-effort hits).
What you can split spend by Cost attribution
Custom metadata and filters on all tiers. Per key/user/team/tag/customer cost splits not explicitly stated. Not verified
The dimensions the vendor documents for splitting spend — per key, per user, per team, per tag, per customer. Matters if you need to chargeback internally or bill an end customer.
How you get cost data out Cost export
'Data Export to Data Lakes' on Enterprise only. CSV/API/webhook/S3 not stated; price not stated. Not verified
The mechanisms the vendor publishes for exporting cost and usage data: CSV, an API, webhooks, S3, a data warehouse, or nothing at all. Any per-unit price is included.
Who pays the model bill BYOK mode
Your keys only Not verified
Whether you bring your own provider accounts (BYOK), buy inference from this vendor, or can do either. This is the single biggest commercial difference between these products: it decides who holds the contract with the model provider and who carries the spend.

Spend governance in detail

Seven signals matter when a bill starts to hurt: who can spend, how much, on what, and who gets paged when it goes wrong. Everything below is drawn from the vendor’s own pricing and docs pages — how we read these.

  • Virtual or scoped keysYes

    Keys that carry their own budget and rate-limit policy, so an intern experiment cannot spend against a production budget.

    Virtual keys are the object budgets attach to.

  • Budget caps per keyYes — enterprise

    A dollar or token ceiling attached to an individual key. Where enforcement is soft, one over-limit request still completes before the block kicks in.

    Docs contradict on availability: 'currently only available to Enterprise' vs 'Available on Enterprise plan and select Pro customers'. Pricing page lists 'Granular Budget & Rate Limits' as Enterprise-only. Minimum $1 cost limit or 100 tokens. Pre-request; key auto-expires on exhaustion. No automatic reset by default.

  • Budget caps per team or workspaceYes

    A ceiling applied at a higher scope than one key — a team, a workspace, a customer, or an entire environment.

    Per-workspace limits supported. Team-level not stated.

  • Rate limiting as a cost controlYes — enterprise

    Configurable request-per-time-window caps. Platform-set rate limits do not count as spend controls; user-configurable ones do.

    'Granular Budget & Rate Limits' is an Enterprise bullet.

  • Model allowlistsNot published

    A policy that constrains which models a key or team can call, keeping expensive frontier models out of the wrong hands.

    Not stated.

  • Spend alertsYes

    Alerts fired as spend approaches a threshold. Alerts that only fire after the meter has rolled over are marked as such.

    'Alerts' in Production observability; spend-specific alerts not stated.

  • Webhook notificationsNot published

    Programmatic notifications on spend events, so budget breaches can page an on-call or open a ticket.

    Not stated.

Enforcement:Enforced before each request

Catalog

Models available Models available
250–2,300 Verified 2 days ago
How many models you can call, shown as a range because several vendors publish different totals on different pages. Vendor-reported either way, so counts are not directly comparable — some count every provider variant of the same model separately.
Model providers reachable Upstream providers
40–48 Not verified
How many distinct model providers or labs you can reach, shown as a range where the vendor’s own pages disagree. More providers usually means better redundancy when one has an outage.
Works with standard OpenAI code OpenAI-compatible API
Yes Verified 2 days ago
If yes, you can usually switch to it by changing one base URL, and switch away just as easily. This is the main defence against lock-in.
OpenAI chat endpoint POST /v1/chat/completions
Yes Not verified
The endpoint almost every application ports first. "Not documented" means the vendor never states it, which is different from a documented no.
Anthropic messages endpoint POST /v1/messages
Yes Not verified
Whether Anthropic-shaped calls work without rewriting them. Several products support this only as SDK compatibility or provider passthrough rather than a native endpoint — the detail page says which.
OpenAI Responses endpoint POST /v1/responses
Not documented Not verified
The newer stateful OpenAI surface. Support is much thinner across this market than chat completions.
Embeddings endpoint POST /v1/embeddings
Yes Not verified
Whether you can generate vectors through the same gateway, or need a second integration for retrieval workloads.
Image generation endpoint POST /v1/images/generations
Yes Not verified
Whether image models are reachable through the same surface as text.
Audio endpoints POST /v1/audio/*
Yes Not verified
Speech-to-text and text-to-speech. Frequently the first gap in an otherwise complete gateway.
Batch jobs endpoint POST /v1/batches
Not documented Not verified
Asynchronous bulk processing, usually at a discount. Commonly undocumented, and commonly the reason a migration stalls late.
Needs the vendor’s own code library Requires a vendor-specific SDK
No Verified 2 days ago
A proprietary client library spreads through your codebase and has to be torn out again if you leave. “No” is the better answer here, and it means the standard OpenAI client works.
You can export your request history Logs / usage data export
Yes Verified 2 days ago
Whether you can get your own request logs, traces, or usage records back out — through an API, a bulk export, or a download. Decides whether you leave with your history or abandon it.
Settings can live in version control Declarative config-as-code
Yes Verified 2 days ago
Whether routing, fallback, and budget rules can be declared in a file you keep in Git, rather than existing only as settings clicked into a hosted dashboard.
Embeddings Embeddings
Not published Not verified
Text-to-vector models, needed for search and retrieval features.
Image generation Image generation
Not published Not verified
Whether image models are reachable through the same interface.
Speech and audio Speech and audio
Not published Not verified
Text-to-speech or transcription models through the same interface.
Video generation Video generation
Not published Not verified
Whether video models are reachable through the same interface.
Batch processing Batch processing
Not published Not verified
Submitting large jobs for cheaper, slower processing. Often 50% off for work that is not time-sensitive.

Routing & reliability

Uptime it promises in writing Contractual SLA uptime
99.9% Not verified
The uptime percentage in a published, contractual service level agreement. A public status page is not an SLA — it reports what happened, it does not promise anything or pay you back when it breaks.
Automatic failover Automatic failover
Yes Verified 2 days ago
When a model provider goes down or rate-limits you, traffic moves to a backup automatically instead of returning errors to your users.
Load balancing Load balancing
Yes Verified 2 days ago
Spreads requests across several providers or keys to raise your effective rate limit.
Rule-based routing Conditional routing
Yes Verified 2 days ago
Send different requests to different models based on rules — for example a cheap model for free users and a strong model for paying ones.
Response caching Response caching
Yes Verified 2 days ago
Reuses the answer when the exact same request comes in again, which cuts both cost and latency.
Similar-question caching Semantic cache
Yes Verified 2 days ago
Reuses an answer when a new question means roughly the same thing as an earlier one. Saves far more than exact-match caching but can return subtly wrong answers if tuned loosely.
Where you set the timeout Request timeout surface
Per request Not verified

`per_request` and `config_file`. Header `x-portkey-request-timeout`, SDK parameter `requestTimeout`, and config key `request_timeout` in milliseconds, settable at strategy level or per target with nested inheritance ([Portkey request timeouts](https://docs.portkey.ai/docs/product/ai-gateway/request-timeouts)).

Where a request timeout can be set: per request, in a config file, in the vendor dashboard, or nowhere. Nine of the twenty products documented here do not describe a request timeout at all, so the worst case of a hung upstream call is unknowable from the docs.
Where you set retries Retry policy surface
In config Not verified

`retry.attempts` up to `5`, default retried status codes `[429, 500, 502, 503, 504]` overridable via `on_status_codes`; exponential backoff 1s / 2s / 4s / 8s / 16s ([Portkey automatic retries](https://docs.portkey.ai/docs/product/ai-gateway/automatic-retries)).

Where retry count and backoff are configured. Worth knowing alongside billing: a retried streaming call can be charged more than once.
Where you set fallbacks Fallback surface
In config Not verified

ORDERED: `strategy.mode: fallback` with an ordered `targets` array; default trigger is any non-2xx response ([Portkey fallbacks](https://docs.portkey.ai/docs/product/ai-gateway/fallbacks)).

Where the fallback chain is defined. Almost every product claims fallback; the useful question is whether you can change it from code or only by hand in a dashboard.
Shape of the fallback chain Fallback shape
Ordered list Not verified
An ordered list tries targets in sequence; a weighted split sends a percentage of traffic to each, which is what you need to trial a new model on 5% of requests. Weighted splits are much rarer than the marketing implies.
Upstream health tracking Health checks / circuit breaking
Not documented Not verified

No upstream health check, circuit breaker or provider-ejection key appears on the fetched configs/fallback/load-balancing pages ([Portkey configs](https://docs.portkey.ai/docs/product/ai-gateway/configs)).

Whether the product notices a failing upstream and stops sending traffic to it, and whether you can tune the thresholds. This is what turns a provider outage into a blip rather than a sustained error rate, and only four of the twenty expose it.
Cross-region failover you control Multi-region failover surface
Not documented Not verified

You can list targets that happen to be in different regions, but no cross-region failover feature is documented ([Portkey fallbacks](https://docs.portkey.ai/docs/product/ai-gateway/fallbacks)).

Whether you can define what happens when a region degrades. A vendor running many regions is not the same as a vendor letting you configure failover between them; only three document a user-controlled mechanism.
Where you set load balancing Load balancing surface
In config Not verified

Weights supported: `strategy.mode: loadbalance` with per-target `weight` (default `1`, `0` disables a target, weights are normalized) ([Portkey load balancing](https://docs.portkey.ai/docs/product/ai-gateway/load-balancing)).

Where traffic distribution across upstreams or keys is configured.

Operations

Usage dashboards and logs Observability
Yes Verified 2 days ago
Built-in visibility into what was sent, what came back, what it cost, and how long it took.
Spending limits Budget controls
Yes Verified 2 days ago
Hard caps that stop spend before it becomes a surprise invoice. The single most valuable control for a small team.
Rate limits Rate limits
Yes Verified 2 days ago
Caps on request volume per key or per user, useful for protecting against abuse and runaway loops.
Separate keys per team or app Virtual keys
Not published Not verified
Issue scoped keys with their own budgets and permissions so you can attribute cost and revoke access without rotating everything.
Prompt versioning Prompt management
Yes Verified 2 days ago
Store and version prompts outside your code so they can be changed without a deploy.
Quality testing Evals
Yes Verified 2 days ago
Built-in tooling to score model output against test cases, so you can tell whether a model swap made things better or worse.
MCP support MCP support
Yes Verified 2 days ago
Native support for the Model Context Protocol, the emerging standard for connecting models to external tools.
What gets logged Logged content
Your choice Not verified

Full Logging stores "Complete request payloads", "Full response content", and all metrics; Metrics Only (Privacy Mode) stores usage, metadata, and errors "without sensitive content" ([Request logging settings](https://portkey.ai/docs/product/administration/configuring-request-logging)). Clicking a log entry reveals "the entire raw data with the request and response objects" ([Logs](https://docs.portkey.ai/docs/product/observability/logs))

Whether full prompts and responses are stored, only metadata, or nothing. Full-body logging is the most useful debugging feature here and the one most likely to need a conversation with your compliance team.
You can turn logging off Body-logging opt-out
Yes Not verified

Org/workspace-level Metrics Only mode ([Request logging settings](https://portkey.ai/docs/product/administration/configuring-request-logging)), or per request the `DO NOT TRACK` control: set `debug: false` on the client or send header `x-portkey-debug: false`, keeping tokens, cost, and latency ([Logs](https://docs.portkey.ai/docs/product/observability/logs))

Whether prompt and response bodies can be suppressed while still keeping usage metrics. Nineteen of the twenty document a way to do this; the mechanisms range from a per-request header to an organisation-wide setting.
Traces you can take elsewhere Distributed tracing
OpenTelemetry Not verified

OpenTelemetry export is supported in two modes: analytics-only spans without prompts, and complete logs including full prompt/completion (marked experimental); trace IDs group multi-step requests ([Portkey OTel](https://portkey.ai/docs/product/enterprise-offering/otel/otel), [Feedback](https://docs.portkey.ai/docs/product/observability/feedback))

Whether the product emits OpenTelemetry, a proprietary format, or nothing. OpenTelemetry means the traces land in the tooling you already run instead of only in the vendor’s dashboard.
Where telemetry can go Export destinations
OpenTelemetry, Datadog, Grafana, LangSmith Not verified

Any OTLP-compatible backend, with Datadog, Grafana, and LangSmith named explicitly ([Portkey OTel](https://portkey.ai/docs/product/enterprise-offering/otel/otel))

Documented sinks for logs and metrics. This is a good proxy for how replaceable the vendor’s own dashboard is: around twenty destinations means you never have to depend on it, while a CSV download means you do.
Can record user feedback Feedback capture API
Yes Not verified

Yes — `POST https://api.portkey.ai/v1/feedback` with `trace_id`, `value` (−10…10), optional `weight` and `metadata`; the trace ID is returned in the `x-portkey-trace-id` response header ([Feedback](https://docs.portkey.ai/docs/product/observability/feedback))

Whether there is an API to attach a rating or score to a logged request, which is what lets production traffic feed quality work later.
Scores live traffic Online eval hooks
Partly Not verified

Feedback data feeds Analytics and a Prompt Eval tab inside Portkey; the same guardrail checks can run asynchronously over live traffic as scoring. Both require Portkey's platform ([Feedback](https://docs.portkey.ai/docs/product/observability/feedback), [Guardrails](https://docs.portkey.ai/docs/product/guardrails))

Whether automated scorers can run against real production requests, rather than only against a test set you assemble yourself.

Performance

Delay it adds Proxy overhead
Not published Not verified
Extra time the product itself adds to each request, on top of however long the model takes. Usually irrelevant next to multi-second model latency, but it matters for high-volume or streaming-sensitive workloads.
Requests per second ceiling Throughput
Not published Not verified
Published sustained request rate before the product becomes the bottleneck. Only relevant at genuinely high volume.
What the request path runs on Architecture class
Interpreted proxy Not verified

`interpreted_proxy` (JS runtime). Repo is TypeScript 96.0% and ships as an npm-launched service ([Portkey-AI/gateway](https://github.com/Portkey-AI/gateway)).

The comparable way to talk about latency here. An edge worker, a compiled Go or Rust binary, and a Python proxy have different overhead floors no matter which figures each vendor publishes. Products that never disclose their runtime are recorded as undisclosed rather than assumed.
You can run the request path yourself Self-hostable data plane
Yes Not verified

`npx @portkey-ai/gateway` (npm package `@portkey-ai/gateway`), with documented Docker, Cloudflare, AWS, Azure, GCP and Kubernetes deployments ([Portkey-AI/gateway](https://github.com/Portkey-AI/gateway)).

Whether the component that actually carries your prompts can run on your own infrastructure. Distinct from a vendor offering a self-hosted control plane while still proxying traffic through their network.
Streaming responses Streaming support
Yes Not verified

Supported, with an important documented caveat: the request timeout does not trigger if at least one chunk has already arrived, so a stalled mid-stream response is not cut off by `request_timeout` ([Portkey request timeouts](https://docs.portkey.ai/docs/product/ai-gateway/request-timeouts)).

Whether token-by-token streaming is documented. The caveats matter more than the yes: some products cannot cancel a stream without still being billed, and several timeout and fallback mechanisms stop applying once the first token has been sent.

Security & compliance

Does your prompt reach their servers Prompt transits vendor
Depends how you deploy it Not verified

Requests hit the nearest regional edge gateway on the hosted product, or you run the gateway in your own VPC or air-gapped, where prompts, responses and logs never leave. One caveat holds in every mode: operational metrics are always sent to Portkey's control plane.

Whether the text you send passes through this company’s own infrastructure. If it does, every other promise on this page is a policy commitment rather than a physical impossibility. Self-hosted products can answer no outright.
What they keep if you change nothing Logging default
Yes — prompts and replies Not verified

Hosted logging is on by default. Suppressing body storage is not a setting you can flip — it is enabled on request.

Defaults matter more than options. A product that stores full prompts and replies unless you find the right header will have stored them by the time you read the docs.
How long they keep it Default content retention (days)
90 days Not verified

Ninety days for logs and 365 for metrics by default. Self-hosted puts logs in your own S3-compatible store under your lifecycle policy.

Default retention for request content, in days. Zero means nothing is kept. Read the note: several products keep nothing as a rule but make timed exceptions for abuse review or specific models.
Could they train on your prompts Training on customer data
Not published — silence, not a no Not verified

Nothing in the privacy policy, terms, enterprise security page or security overview addresses training.

Whether the vendor may use your prompts and outputs to train models. “Not published” means we could not find any position, which is not the same as a no — ask for it in writing.
Where it runs, and what you can pin Region and residency control
Region pinning with data stored in your selected region and requests served in-region across a network of about 310 data centres, plus customer-VPC and fully disconnected air-gapped deployment. Not verified
Which regions are offered and whether you can force processing to stay in one. A global endpoint that silently picks a region is a different compliance story from an endpoint you pin yourself.
Where safety filters run Guardrail execution location
Either, depending on deployment Not verified

PII scrubbing is a gateway feature, and the gateway runs either at Portkey's edge or inside your VPC or air-gapped environment.

A filter that strips personal data only helps if it runs before the data leaves your boundary. If guardrails execute in the vendor’s cloud, the vendor has already received whatever you wanted redacted.
Who else touches the data Subprocessor list
Not published Not verified
The published list of third parties the vendor passes your data to. No list means you cannot know the full chain, which most data-protection agreements require you to.
SOC 2 audited SOC 2 audited
Yes Verified 2 days ago
An independent audit of security controls. Enterprise buyers and their procurement teams routinely require it.
Will sign a HIPAA agreement HIPAA BAA
Yes Verified 2 days ago
Required before you may send protected health information through the service. Without a signed BAA, healthcare data is off limits.
GDPR commitments GDPR commitments
Yes Verified 2 days ago
Published data processing terms for handling personal data of people in the EU and UK.
Can keep data in the EU EU data residency
Not published Not verified
Requests can be processed inside the EU rather than routed to US infrastructure. Often the deciding constraint for European customers.
Does not retain your data Zero data retention
Yes Verified 2 days ago
Prompts and responses are not stored after the request completes. Sometimes a paid add-on rather than the default.
Strips personal data PII redaction
Yes Verified 2 days ago
Detects and removes identifiers such as names, emails, and card numbers before the request reaches the model provider.
Content guardrails Content guardrails
Yes Verified 2 days ago
Policy checks on inputs and outputs — blocking unsafe content, enforcing formats, or catching prompt-injection attempts.
Runs fully disconnected Air-gapped deployment
Not published Not verified
Can be deployed in a network with no internet access, which some regulated and defence environments require.
Blocks personal data in prompts PII / DLP enforcement
Can block the request Not verified

"Detect PII" is one of 20+ deterministic checks; guardrail actions default to `async: TRUE` (log only, no added latency), and only when a check is set synchronous **and** `Deny: TRUE` does Portkey return HTTP 446 and block. With `Deny: FALSE` the request passes through with a 246 status and the verdict recorded ([Guardrail checks](https://portkey.ai/docs/product/guardrails/list-of-guardrail-checks), [Guardrails](https://docs.portkey.ai/docs/product/guardrails))

Whether personal data detection sits on the request path and can stop the call, merely inspects and forwards it, or is not documented. A control that only reports is a logging feature, not a policy control.
Blocks prompt injection Injection / jailbreak enforcement
Can block the request Not verified

Injection/jailbreak detection comes from partner guardrail vendors rather than a built-in classifier, and inherits the same async/sync + Deny semantics ([Guardrail checks](https://portkey.ai/docs/product/guardrails/list-of-guardrail-checks), [Guardrails](https://docs.portkey.ai/docs/product/guardrails))

Whether injection and jailbreak detection can stop a request. Most products offering this call a partner classifier rather than shipping their own.
Blocks harmful content Toxicity / moderation enforcement
Can block the request Not verified

"Moderate Content" is a built-in check subject to the same async default ([Guardrail checks](https://portkey.ai/docs/product/guardrails/list-of-guardrail-checks), [Guardrails](https://docs.portkey.ai/docs/product/guardrails))

Whether hate, violence, sexual and self-harm categories are checked inline and can stop a request, in either direction.
Your own policy rules Custom policy hooks
Can block the request Not verified

Policies take the form of Regex Match, Contains / word lists, JSON Schema and JSON Keys validation, Model Whitelist, and Webhook (custom classifier hook) ([Guardrail checks](https://portkey.ai/docs/product/guardrails/list-of-guardrail-checks))

Whether you can add your own rule — a regex, a webhook, or your own classifier — rather than choosing from the vendor library.
Where guardrails run Guardrail execution location
Either, your choice Not verified
Whether guardrail evaluation happens inside your infrastructure or on the vendor’s servers. This decides whether the prompt you are trying to protect leaves your network in order to be checked.
If the guardrail itself fails Guardrail failure mode
Not documented Not verified

The docs describe deny/allow verdict semantics but not behaviour when the guardrail service itself errors or times out ([Guardrails](https://docs.portkey.ai/docs/product/guardrails))

What happens when the guardrail service times out or errors: does the request proceed unchecked, or is it blocked? This is the worst-documented field in the entire catalogue — only two vendors state it plainly, which means most teams are running a control whose failure behaviour they cannot know.
Third-party guardrail vendors Guardrail integrations
Acuvity, Aporia, AWS Bedrock Guardrails, Azure Content Safety, Javelin, Lasso, Mistral, Pangea, Palo Alto Prisma AIRS, Patronus AI, Pillar Security, Qualifire Not verified
Named external guardrail services the product can call. A long list means the product is a router for policy engines rather than a policy engine itself — which also means another vendor bill and another hop.

Compliance evidence

Graded by how strong the evidence is, not whether the word appears on the vendor’s website. An audited report and a marketing claim are different things, and only one of them will satisfy your own auditor.

  • SOC 2 Claimed, no evidence published Type II said to be available via an unnamed trust portal
  • ISO 27001 Claimed, no evidence published asserted in docs; no report or portal named
  • GDPR DPA Claimed, no evidence published described as GDPR-compliant
  • HIPAA BAA Claimed, no evidence published described as HIPAA-compliant; no BAA offer found
  • FedRAMP Not published
  • ITAR Not published

Vendor source

Fit & integration

Work to try it Evaluation work shape
Install a package Not verified
The shape of the work on the vendor’s own quickstart, from swapping one base URL through to deploying infrastructure. An ordinal class rather than a duration, because elapsed time depends on accounts and quota we cannot see.
Work to run it Production work shape
Install a package Not verified
The same scale applied to the vendor’s recommended production path. For several products this is much heavier than the quickstart, which is exactly why both are recorded.
Steps on the quickstart Numbered quickstart steps
3 Not verified
A literal count of numbered steps on the vendor’s quickstart, recorded as evidence beside the work shape. Zero means the page publishes no numbered procedure at all. Large counts usually mean interleaved language tracks rather than more work.
Can you self-host it today Self-host install documentation
Install command published Not verified

`docker pull portkey/gateway-enterprise:latest`; `helm repo add portkey https://portkey-ai.github.io/helm`; `helm repo update`; `helm upgrade --install portkey-gateway portkey/gateway -n portkey -f values.yaml` (plus `kubectl create namespace portkey` and an image-pull secret) ([Deploy Portkey in Your Infrastructure](https://portkey.ai/docs/enterprise/hybrid2))

Whether an install command is actually published. Several products advertise self-hosting while publishing no command to start from, which a plain yes/no would hide.
Works with the OpenAI SDK OpenAI SDK drop-in
Yes Not verified

Yes. Portkey documents direct OpenAI SDK compatibility using the exact hosted base URL `https://api.portkey.ai/v1`; local deployments use `http://localhost:8787/v1`. ([Portkey AI Gateway getting started](https://docs.portkey.ai/docs/guides/getting-started/getting-started-with-ai-gateway))

Whether an existing OpenAI-compatible client can be pointed at it by changing the base URL and key.
Vercel AI SDK support AI SDK provider package
Official provider package Not verified

Portkey provides a Vercel AI SDK integration. The documentation names the provider package `@portkey-ai/vercel-provider` and uses `createPortkey`. ([Portkey Vercel AI SDK integration](https://docs.portkey.ai/docs/integrations/libraries/vercel))

Whether a first-party AI SDK provider package exists, or only a community package, a documented workaround, or the generic OpenAI provider pointed at a custom base URL.
Python framework integrations Documented Python frameworks
LangChain, LlamaIndex Not verified

Portkey documents integrations with LangChain and LlamaIndex. LangChain examples use `langchain`, `langchain-openai`, `ChatOpenAI`, `createHeaders`, and `LangchainCallbackHandler`; LlamaIndex examples use `llama_index.llms.openai.OpenAI`, `ChatMessage`, and `portkey-ai`. ([Portkey LangChain agents integration](https://portkey.ai/docs/integrations/agents/langchain-agents))

LangChain, LangGraph, LlamaIndex and similar orchestration frameworks with a documented integration.
Callable from Cloudflare Workers Cloudflare Workers support
Their gateway runs on Workers, not yours Not verified

The Portkey Gateway runs on Cloudflare Workers and uses Cloudflare’s global edge network and data centers, according to a Portkey blog post. ([Why Portkey chose TypeScript for its AI Gateway](https://portkey.ai/blog/why-we-chose-ts-over-python-to-build-potkeys-ai-gateway))

Whether the docs show calling this product from your own Worker. Deliberately separated from the several products whose own gateway runs on Workers, which is a fact about their infrastructure and not about your edge compatibility.
Kubernetes install Helm chart availability
Official Helm chart Not verified

Portkey provides official Kubernetes Helm charts in the `Portkey-AI/helm` repository: `charts/portkey-gateway` for Portkey Hybrid (Gateway) and `charts/portkey-app` for Portkey Full (Full App). ([Portkey official Helm charts](https://github.com/Portkey-AI/helm))

Whether a named, published Helm chart exists, versus Helm being referenced with no chart named, versus generic cluster documentation that has nothing to do with this product.
Terraform support Terraform provider or modules
Official Terraform provider Not verified

Portkey provides an official Terraform provider named `Portkey-AI/portkey` on the Terraform Registry. ([Portkey Terraform provider](https://registry.terraform.io/providers/Portkey-AI/portkey/latest/docs))

Whether you can declare this in code: an official provider, official modules, resources inside a hyperscaler’s provider, a community provider, or only Terraform code shipped in a repo.
Reuses your cloud identity Cloud IAM reuse
Not documented Not verified

n.a. (not documented)

Whether you can authenticate with IAM roles, workload identity or managed identities instead of another long-lived API key. Distinguished from products that only accept static upstream provider credentials.
Fits behind your API gateway API gateway integration
Not documented Not verified

n.a. (not documented)

Whether AI traffic can go through a gateway you already run, and who documented that — the product’s vendor or the gateway’s.
MCP support MCP surface shape
MCP gateway or registry Not verified

Portkey documents an MCP Gateway that centralizes authentication and team/user access control for MCP servers and tools, logs tool calls, and supports identity forwarding. It works with Claude Desktop, Cursor, VS Code, and other MCP-compatible clients. ([Portkey Gateway GitHub repository](https://github.com/portkey-ai/gateway))

Which kind of MCP support this is: a gateway that governs many MCP servers, a hosted MCP server you connect to, MCP tools accepted inside the completion API, or client tooling. These are different products behind one acronym.
Needs your own provider key Upstream provider key required
Required Not verified

Not explicitly stated, but the documented snippets reference a provider: `provider="@YOUR_PROVIDER"` and examples using `YOUR_OPENAI_API_KEY`; the self-hosted validation step creates "a virtual key" before the first call ([Docs - what is Portkey](https://portkey.ai/docs/introduction/what-is-portkey), [Deploy Portkey in Your Infrastructure](https://portkey.ai/docs/enterprise/hybrid2))

Whether an upstream provider account and key must exist before your first call works. A prerequisite rather than a step, and it can differ between the hosted and self-hosted forms of the same product.
Gate before models work Model access gate
Not documented Not verified

n.a. (not documented). The only permissions wording is on the API-key modal: "Based on your access level, you might see the relevant permissions on the API key modal" ([Make Your First Request - Portkey Docs](https://portkey.ai/docs/introduction/make-your-first-request)); no approval gate in the private-deployment guide either ([Deploy Portkey in Your Infrastructure](https://portkey.ai/docs/enterprise/hybrid2))

Whether an enablement click, a quota grant, a paid tier or an approval form stands between a valid key and a working model call.
Official client languages First-party client SDK languages
Python, JavaScript Not verified

Official examples use the OpenAI SDK and Portkey SDKs: Python packages `openai` and `portkey-ai`/`portkey_ai`, JavaScript packages `openai` and `portkey-ai`, and the Python `Portkey` class. Portkey also documents the Anthropic package `anthropic` for the Messages API. ([Portkey AI Gateway getting started](https://docs.portkey.ai/docs/guides/getting-started/getting-started-with-ai-gateway))

Languages with a first-party client library. An empty list can still mean the product is usable from any language via an OpenAI-compatible SDK.

Additional charges

These are the fees that do not appear on a per-token price list, and they are where estimates usually go wrong.

  • Production plan platform fee $49/month (100k recorded logs, 30-day log retention)
  • Production log/request overage +$9 per additional 100k requests

How pricing actually works

The gateway core is MIT-licensed and free to self-host (you pay only your own infrastructure and model providers); the Gateway 2.0 release moved circuit breakers, semantic cache, budget limits, model catalog, metadata governance and config management into open source. Self-hosting drops the hosted dashboards/control plane, and an On-Prem Enterprise Gateway (gRPC, SSO, SCIM, AWS KMS, RBAC, JWT, audit logs, multi-workspace) is sold separately.

Back to top ↑

Official links

12,848 GitHub stars — a proxy for community size, not for quality.

Independent coverage

Third-party analysis, walkthroughs and operator threads. We link criticism as readily as praise. Nothing here is written or published by the vendor, by a competitor listed on this site, or by an SEO content farm — how we vet these.

Written reviews and analysis 1

Video 2

Practitioner discussion 1

What has changed here

No tracked value on this page has moved since it was first researched. Every figure still carries its original citation and check date.

Back to top ↑

Read the head-to-head

These pairs have a written verdict, not just a table.

Portkey vs LiteLLM Read the verdict → Portkey vs Helicone Read the verdict →

Usually weighed against

Portkey vs Braintrust Gateway Compare → Portkey vs Helicone Compare → Portkey vs Kong AI Gateway Compare →