Helicone
Open-source LLM observability platform with an OpenAI-compatible AI gateway attached.
Access at a glance
Whether this product can work for you at all, before features matter: who pays the model bill, where it can run, and which of your existing API calls keep working. Every value is the vendor’s own claim, linked to the page it came from.
Observability platform that also proxies/routes: the repo calls it "Helicone AI Gateway… The fastest, lightest, and easiest-to-integrate AI Gateway on the market", "The NGINX of LLMs", "Open-source, lightweight, and built on Rust" (GitHub Helicone/ai-gateway); the self-host Docker page calls Helicone an "LLM observability platform" whose Jawn service is the "Jawn API + LLM Proxy" (Docker self-host).
Who pays the model bill
Your keys or their creditsYou can start on their credits and move to your own provider accounts later.
BYOK keys are tried first with Helicone-managed keys as fallback, and credits give access to "100+ other providers without signing up for each one" at "0% markup" (Provider routing, Gateway overview).
Merchant of record: Credits mode: Helicone invoices, at "0% markup", "customers pay exactly provider charges" (Gateway overview, Provider routing). BYOK/self-host mode: the upstream provider, since your own OPENAI_API_KEY/ANTHROPIC_API_KEY are used (GitHub). Explicit invoicing language: not stated.
Key handling: Self-hosted: provider keys stay in your .env/environment as PROVIDER_API_KEY values, and the gateway itself "handles API keys", with the Helicone key only needed when authentication is enabled (GitHub). Cloud: keys are added in Provider Settings and BYOK keys are attempted before Helicone-managed keys (Provider routing). Rate limits can be scoped per user, team, per-API-key or globally (GitHub). Encryption specifics and whether Helicone can read request bodies in cloud mode: n.a. on pages fetched.
Where it can run
2 of 5 shapes documented- Vendor-hosted
- Self-host
- Your VPC
- On-premise
- Air-gapped
Dimmed shapes are not documented by the vendor, which is not the same as unsupported.
Hosted SaaS (https://ai-gateway.helicone.ai/ai), self-host (local binary via npx @helicone/ai-gateway@latest, Docker/Docker Compose, Kubernetes/Helm, cloud deployment on AWS/GCP/Azure) (GitHub, Self-hosting overview); on-prem is implied by the NEXT_PUBLIC_IS_ON_PREM flag (Docker self-host). Hybrid/VPC and air-gapped: not documented.
Self-hosting options are Manual, Docker Compose, Kubernetes with Helm charts, and cloud infrastructure (Self-hosting overview); the all-in-one image is helicone/helicone-all-in-one:latest for a "local machine or server" or "Production Setup (Remote Server) — EC2, VPS, etc." (Docker). Important gating: in the self-hosted version "Other providers (Vertex AI, AWS Bedrock, Azure OpenAI) are not supported" — only OpenAI and Anthropic (Docker).
API surfaces your code can keep using
2 of 7 documented- OpenAI chat
POST /v1/chat/completionsYesSelf-hosted path
http://YOUR_IP:8585/v1/gateway/oai/v1/chat/completions; cloud/router usage isclient.chat.completions.createagainst a gateway base URL such ashttps://ai-gateway.helicone.ai/aiorhttp://localhost:8080/router/<name>(Docker, GitHub). - Anthropic messages
POST /v1/messagesYeshttp://YOUR_IP:8585/v1/gateway/anthropic/v1/messages(Docker). - OpenAI Responses
POST /v1/responsesNot documentedn.a. (not mentioned on the gateway or self-host pages fetched: GitHub, Provider routing, Integrations overview).
- Embeddings
POST /v1/embeddingsNot documentedn.a. (not documented on pages fetched: Provider routing, Integrations overview).
- Images
POST /v1/images/generationsNot documentedn.a. (same pages, not documented).
- Audio
POST /v1/audio/*Not documentedn.a. (same pages, not documented).
- Batch jobs
POST /v1/batchesNot documentedn.a. (same pages, not documented).
An asterisk marks a qualified verdict: support that is indirect (SDK compatibility or provider passthrough rather than a native endpoint), or a gap that is narrower or wider than the label suggests. Read the note before porting.
Drop-in base-URL swap with "any OpenAI SDK" — "No new SDKs to learn, no integrations to maintain" — but model names must become provider-qualified (gpt-4o → openai/gpt-4o) (GitHub). Routing selectors are expressed in the model string, e.g. gpt-4o-mini/openai for a specific provider or gpt-4o-mini/azure/clm1a2b3c for a configured deployment (Provider routing).
How much it reaches
Providers: The vendor publishes different totals on different pages; both bounds are shown.
"1 API. 100+ models" (GitHub); "100+ Models Available" (Integrations overview). Both undated.
"100+" providers on the gateway docs (undated) (Provider routing, Integrations overview, Gateway overview); the GitHub README says "OpenAI, Anthropic, Google, AWS Bedrock, and 20+ more providers" (GitHub); the self-hosted build supports only 2 (OpenAI, Anthropic) (Docker). See contradictions.
Whose models: Third-party routing to "LLM Providers" (OpenAI, Anthropic, AWS Bedrock, Google Vertex, "20+ more") (GitHub); with credits "we manage provider API keys for you" so requests still reach those providers (Gateway overview). No Helicone-owned inference hardware claimed.
Your own endpoints: n.a.: not documented on the pages fetched; the closest is a "configured deployment" ID for provider deployments such as Azure (Provider routing).
How it behaves in production
What happens when an upstream model is slow, wrong, or down — and what you can see and stop while it happens. Reliability features are recorded as where you configure them, not whether the vendor lists them, because almost every product here lists all of them.
3 of 6 reachable from code1 of 4 can block4 documented destinations
When something goes wrong
Each row says where the knob is, not whether the feature is on the marketing page. A control you can only reach by hand in someone else’s dashboard cannot be reviewed or version-controlled.
- Request timeoutNot documented
The vendor does not document this, so any behaviour you observe today is unversioned and may change.
not_documentedas a settable value. Timeout appears only as a failover trigger:408"Timeout errors" causes the gateway to try the next provider (Helicone provider routing). Retry headers includeHelicone-Retry-Min-Timeout/Helicone-Retry-Max-Timeout, but these bound backoff, not the request (Helicone retries). - RetriesPer request
Your application decides per call, so one noisy endpoint can have its own timeout without a redeploy.
per_requestvia headers:Helicone-Retry-Enabled,Helicone-Retry-Num(default5),Helicone-Retry-Factor(default2),Helicone-Retry-Min-Timeout(default1000ms),Helicone-Retry-Max-Timeout(default10000ms). Backoff is exponential: 1s → 2s → 4s → 8s → 10s, on429,500,502,503,504(Helicone retries). - Fallback to another modelPer request
per_requestfor the cloud gateway: ORDERED.Helicone-FallbacksJSON header withtarget-url,onCodesandbodyKeyOverrideper entry; the response headerHelicone-Fallback-Indexreports which entry served (Helicone gateway fallbacks). In the AI Gateway the ordered chain is expressed inside themodelstring, e.g.model: "gpt-4o-mini/azure,gpt-4o-mini/openai,gpt-4o-mini", "tried in the exact order you specify" (Helicone provider routing). - Load balancingIn config
Changing it means editing configuration and shipping it, so behaviour is uniform across traffic until you redeploy.
Weights ARE supported. YAML key
routers.<name>.load-balance.chat.strategywith strategies:model-latency, provider latency P2C + PeakEWMA, weighted distribution based on model weight, and cost optimization (Helicone/ai-gateway). In the cloud gateway, "equal-cost providers are load balanced" and BYOK keys are always tried before Helicone managed keys (Helicone provider routing). - Upstream health trackingFixed, cannot change
The behaviour is fixed by the vendor. Predictable, but you cannot tune it for your workload.
not_configurable, and it is passive unhealthy-provider avoidance rather than a configurable circuit breaker: the gateway is "always aware of provider uptimes and your rate limits" and instantly tries the next provider on429,401,400,408,500+(Helicone/ai-gateway, Helicone provider routing). - Cross-region failoverNot documented
not_documentedas failover. You can pin a specific regional deployment (model: "gpt-4o/azure/eu-frankfurt-deployment", "No data leaves the EU"), but that is region pinning, not cross-region failover (Helicone provider routing).
Fallback chain: Ordered list — Try A, then B, then C. Simple and predictable, but every failover is all-or-nothing.
Defaults: per_request via headers: Helicone-Retry-Enabled, Helicone-Retry-Num (default 5), Helicone-Retry-Factor (default 2), Helicone-Retry-Min-Timeout (default 1000 ms), Helicone-Retry-Max-Timeout (default 10000 ms).
Two distinct products share the brand: the header-driven cloud proxy (Helicone-Retry-*, Helicone-Fallbacks) and the Rust AI Gateway with YAML routers. Their config surfaces do not overlap, so docs for one do not describe the other (Helicone gateway fallbacks, Helicone/ai-gateway).
How fast the hop is
Compiled binaryA single compiled Go or Rust binary. The lowest overhead floor of the self-hostable options, and the easiest to reason about under load.
"built on Rust"; repo language breakdown Rust 96.7% (Helicone/ai-gateway).
yes. npm-launched binary npx @helicone/ai-gateway@latest --config config.yaml, plus Docker and Kubernetes self-hosting (Helicone/ai-gateway).
Streaming caveats: Supported; the fetched pages do not state a streaming-specific caveat for the AI Gateway (Helicone/ai-gateway). Note the retry headers are documented independently of streaming (Helicone retries).
Published figures, grouped by what each one measured. Figures in different groups are different quantities and cannot be compared with one another — nor, in most cases, with another vendor’s figure in the same group.
Overhead added by the gateway
The only figures that speak to whether this product slows your application down. Still not comparable between vendors: each was measured on different hardware, at a different load, with a different payload.
Sustained capacity
Requests or queries per second sustained on the stated hardware.
- ~3,000 req/ssustainedVendor-published
README table against a "Typical Setup" baseline of ~500. Hardware not stated.
Source
All vendor-published and self-measured. Flag: the comparison column ("Typical Setup", ~60-100ms, ~500 req/s) is a vendor figure about unnamed competing setups, with no methodology or named product — treat it as marketing, not a benchmark (Helicone/ai-gateway).
What it will stop
1 of 4 can blockTwo separate questions per control: can it stop a request at all, and what does it do before you change any settings? A control that inspects and forwards is a logging feature, however it is named.
One of these controls can block a request, but will not until you change its settings:
- Harmful content — Ships switched off
- Personal data in promptsNot documented
No PII scanner documented; the privacy control is body omission rather than detection (Omit logs)
- Prompt injection and jailbreaksNot documented
not_documented(Moderations) - Harmful contentCan block the request
Out of the box: Ships switched off
With the header
Helicone-Moderations-Enabled: true, Helicone calls OpenAI's moderation endpoint before the chat completion and returns HTTP 400 withPROMPT_FLAGGED_FOR_MODERATIONwhen flagged, so the request never reaches the model (Moderations) - Your own policiesNot documented
No customer-defined regex/word-list/JSON-schema policy engine documented (Moderations)
Almost no vendor in this catalogue documents what happens when the guardrail service itself times out. If the control matters to you, this is a question worth asking before you sign.
No stated behaviour if the moderation call fails or times out (Moderations)
Calls out to: OpenAI Moderation. Each is a separate vendor relationship and a separate hop on the request path.
Helicone's one true request-path control is moderation, and it is opt-in per request via a header rather than an org-wide policy — a per-call flag is easy for a client to omit (Moderations).
What you can see
Exports to a few placesPrompt and completion bodies are stored by default. Powerful for debugging, and a data-residency question you have to answer before you ship.
Headers Helicone-Omit-Request: true and Helicone-Omit-Response: true, or disable_content_tracing() in the SDK, keep metrics while dropping bodies (Omit logs)
The AI Gateway repo states "OpenTelemetry support for logs, metrics, and traces" alongside built-in Helicone observability; representation of multi-step agent traces is not detailed on the fetched pages (Helicone AI Gateway repo)
Request and response bodies (prompt and completion text) are logged unless omission is requested (Omit logs)
Where telemetry can go
- OpenTelemetry
- Datadog
- Webhook
- PostHog
PostHog via Helicone-Posthog-Key / Helicone-Posthog-Host headers (PostHog integration); OpenTelemetry logs/metrics/traces from the gateway (Helicone AI Gateway repo)
POST /v1/request/{id}/feedback with a boolean rating (Feedback); arbitrary numeric scores via POST /v1/request/{requestId}/score (Scores)
Scores can be pushed against logged requests, but the docs state plainly that "Helicone doesn't run evaluations for you" — you compute scores externally and post them (Scores)
Depends on the vendor’s SaaS: The gateway is fully open-source and self-hostable, and the repo presents observability as part of it; the fetched pages do not enumerate which dashboard features are cloud-only (Helicone AI Gateway repo)
Retention: Plan-based and configurable only by upgrading: Hobby 7 days, Pro 1 month, Team 3 months, Enterprise forever (Helicone pricing)
Whether it fits how you work
How much work stands between you and a first call, how different that is from running it in production, and whether it slots into the stack you already have. Recorded as the shape of the work rather than a number of minutes — how long it takes you depends on which accounts and quota you already hold, which no comparison can know.
to try: base url swapto run: base url swapfits 4 of 10 common stacks
Getting to a first call
2 numbered stepsYour existing OpenAI-compatible client keeps working. You change a base URL and a key, and nothing else in your code moves.
Read off: the vendor’s own quickstart — 2 numbered steps.
Before step one
- Your own provider keyNot hosted, yes self-hosted
The hosted service works with just its own key. Self-hosting means bringing your own provider keys — a different prerequisite for the same product.
No for the hosted gateway ("Want more control? You can bring your own provider keys instead," presented as optional) (Quickstart - Helicone). Yes in the self-hosted Docker path, whose first call sends
Authorization: Bearer $OPENAI_API_KEYalongsideHelicone-Auth(Docker - Helicone) - Payment methodNo card needed to start
Not stated as required for the first call; the signup link is labeled "Sign up for free" (Quickstart - Helicone) and pricing lists "Hobby — Free" plus "Pro — 7-day free trial" (Helicone Pricing)
- Gate before models answerNo gate
Every catalogue model is callable as soon as you have a key.
n.a. (not documented). No approval/quota/waitlist step; users can "Switch between 100+ models by just changing the model name" (Quickstart - Helicone)
Everything you need first: A Helicone account plus a Helicone API key. No credit card, cloud account, cluster, or provider key: "Helicone maintains the keys for you." (Quickstart - Helicone)
The vendor’s own time claim: Vendor claim, verbatim: "Get your first LLM request logged with Helicone in under 2 minutes using the AI Gateway." (Quickstart - Helicone) Quoted, not verified. Marketing time claims assume every account and approval is already in place.
Running it in production
The same scale applied to the path the vendor recommends for production traffic. Kept separate from the quickstart because for several products here the two are barely related pieces of work.
Your existing OpenAI-compatible client keeps working. You change a base URL and a key, and nothing else in your code moves.
Nothing works until you have a process running. Fine on a laptop, but it means there is no zero-install way to try it.
Getting to production is a step up in kind from the quickstart, not just more of the same.
What production needs: Hosted: credits on the account ("You simply add credits to your account, and we handle the rest") (Quickstart - Helicone). Self-hosted: Docker; ports 3000/8585/9080 reachable; internal PostgreSQL (5432) and ClickHouse (8123); MinIO S3 storage; public IP/domain for remote deployments; Docker volumes for persistence; a reverse proxy for HTTPS; and production env vars SITE_URL, BETTER_AUTH_URL, BETTER_AUTH_SECRET, NEXT_PUBLIC_APP_URL, NEXT_PUBLIC_HELICONE_JAWN_SERVICE, NEXT_PUBLIC_IS_ON_PREM=true, S3_ENDPOINT (Docker - Helicone)
Can you run it yourself
There is a command you can copy and run, so you can evaluate the self-hosted path yourself today.
docker pull helicone/helicone-all-in-one:latest then docker run -d --name helicone -p 3000:3000 -p 8585:8585 -p 9080:9080 helicone/helicone-all-in-one:latest (Docker - Helicone)
How it fits your stack
4 of 10Each row is a thing you might already run. “With a caveat” means it works but not the way the vendor’s marketing implies — read the reason, because that is usually where the surprise lives.
- FitsThe OpenAI SDKDrop-in: change the base URL and key, nothing else.
- FitsThe Vercel AI SDK@helicone/ai-sdk-provider
- NoCloudflare WorkersTheir own gateway runs on Workers. That is not the same as edge support for your code.
- FitsKuberneteshelicone-core, helicone-infrastructure, helicone-monitoring, helicone-argocd
- With a caveatTerraform or OpenTofuTerraform code in the repo only
- NoAn existing API gatewayNothing published about running behind your gateway.
- NoCloud IAM I already runNo identity integration published.
- With a caveatLangChain or LlamaIndexLlamaIndex only.
- With a caveatMCP servers to governMCP client tooling — governs nothing on your side.
- FitsNothing — plain Node or PythonChange one base URL.
Reading this the other way round — pick what you already run and see every product scored against it.
The integration surfaces behind those answers
- Vercel AI SDKOfficial provider package
Install the package, swap the model factory, done. Maintained by a party with a stake in it.
Named:
@helicone/ai-sdk-providerHelicone documents Vercel AI SDK integrations using
@helicone/ai-sdk-providerwithcreateHelicone,@ai-sdk/openaiwithcreateOpenAI, and@ai-sdk/gatewayfor gateway-specific functionality. (Helicone Vercel AI SDK integration) - Cloudflare WorkersTheir gateway runs on Workers, not yours
The vendor runs its own gateway on Workers. That is a fact about their infrastructure and says nothing about calling them from your Worker — it usually means low latency to their edge, not edge compatibility for you.
Helicone states that its proxy leverages Cloudflare Workers running on Cloudflare’s global network to process requests at the edge. (Helicone latency reference)
- KubernetesOfficial Helm chart
A named, published chart. You can read its values file before committing to anything.
Named:
helicone-core, helicone-infrastructure, helicone-monitoring, helicone-argocdHelicone explicitly supports self-hosting on Kubernetes and provides an official Helm chart. The chart repository is the Helicone Helm repository, with components including
helicone-core,helicone-infrastructure,helicone-monitoring, andhelicone-argocd; the Helm deployment useshelm upgrade --install helicone ./helicone -f values.yaml. (Helicone Kubernetes self-hosting) - TerraformTerraform code in the repo only
Terraform exists inside the repo or product but nothing is published for you to consume. You will be adapting their code, not calling a module.
Named:
terraform/s3, terraform/aurora and terraform/eks directories in the repo; no provider or module publishedHelicone documents Terraform directories for creating AWS S3, Aurora, and EKS resources:
terraform/s3,terraform/aurora, andterraform/eks. An official Helicone Terraform provider, module, or Terraform Registry name is not stated. (Helicone Kubernetes self-hosting) - Existing API gatewayNot documented
Nothing published about sitting behind an existing gateway. Treat it as a separate hop you route to yourself.
n.a. (not documented)
- Cloud identityNot documented
No identity integration published. Expect API keys in a secret store.
n.a. (not documented)
- MCPMCP client tooling
Libraries and helpers for building or calling MCP. You assemble it; the product does not operate MCP for you.
Helicone provides an MCP server available as the npm package
@helicone/mcp, including the toolsquery_requestsandquery_sessions. It supports MCP-compatible clients such as Claude Desktop, Claude Code, Cursor, and Codex, andquery_sessionsprovides session search, filtering, pagination, and session performance metrics. (Helicone MCP integration)
LlamaIndex is documented with the packages llama-index-llms-helicone, llama_index.core, and llama_index.llms.openai, including the Helicone and OpenAI classes. LangChain and LlamaIndex are also discussed in a comparison blog post, which mentions PromptTemplate, LLMChain, IndexToolConfig, and LlamaIndexTool; the LangChain details are from that comparison post rather than a dedicated integration page. (Helicone LlamaIndex integration)
Documented clients and packages include the OpenAI Python package (openai, from openai import OpenAI), the OpenAI JavaScript package (openai), @helicone/ai-sdk-provider, @ai-sdk/gateway, @ai-sdk/openai, llama-index-llms-helicone, boto3, and helicone-async. Bedrock examples also document the classes BedrockRuntimeClient, InvokeModelCommand, ConverseCommand, and HeliconeAsyncLogger. (Helicone client integrations)
Agent features: Helicone documentation shows tool/function calling, session tracking, and grouping related API calls. The Vercel AI SDK integration documents multi-step orchestration using Experimental_Agent with stopWhen: stepCountIs(5), while A2A is not documented. (Helicone Vercel AI SDK integration)
Hosted quickstart needs no library install at all (curl works). Self-hosted is materially narrower: "Self-hosted version supports OpenAI and Anthropic only," port 8585 "has no proxy authentication; firewall restriction is required," and email verification must be performed manually after creating an account at /signup (Quickstart - Helicone, Docker - Helicone)
Helicone documents hosted gateways including https://ai-gateway.helicone.ai, access to 100+ models or providers, logging and observability, routing, fallbacks, and BYOK through Helicone Providers. It also explicitly documents self-hosting a complete Helicone stack on Kubernetes/Helm, with AWS S3, Aurora, and EKS resources supported through Terraform; some older OpenAI integration pages are described as maintained but no longer actively developed. (Helicone quick start)
Silence in the docs: 2 of the integration questions on this card have no published answer either way. That is recorded as undocumented, not as a no — but it does mean you would be verifying it yourself.
What it does well
- 0% markup on tokens with credits or BYOK
- Apache-2.0 platform that can be self-hosted for full data control
- Strong observability: sessions, user analytics, custom properties, HQL, alerts
- Public model registry endpoint exposing per-endpoint pricing
- US or EU data region choice, SOC 2 Type II and HIPAA listed on plans
Where it falls short
- Acquired by Mintlify on 2026-03-03 and reported to be in maintenance mode with no new features planned
- Log- and storage-based pricing gets expensive for very high request counts
- Hobby plan drops logs above 10 logs/min, which can silently truncate agent traces
- Observability-first: routing, guardrails and PII redaction are weaker than Portkey or OpenRouter
- Standalone Rust gateway repo (GPL-3.0) has not had a functional commit since July 2025
Choose it when
Teams whose main need is per-request, per-user LLM observability with a thin gateway bolted on, ideally self-hosted.
Look elsewhere when
You need an actively developed routing-first gateway with guardrails and policy enforcement.
Managed gateway: A hosted control plane in front of the models: routing, caching, guardrails, spend limits, and logs. You trade a little simplicity for governance.
How hard is it to leave?
Derived from six published facts, not from an opinion. The weights are fixed and the same for every product — see the arithmetic.
| What helps you leave | Points | Source |
|---|---|---|
| Works with standard OpenAI codeSwitching away is a base-URL change rather than a rewrite of every call site. | 22/22 | vendor page |
| No vendor-specific SDK requiredA proprietary client library spreads through your codebase and has to be torn out again. | 10/10 | vendor page |
| Can use your own provider accountsYour keys and billing relationship stay yours, so removing the gateway does not cut off model access. | 20/20 | vendor page |
| Can be self-hostedYou can run it yourself instead of accepting a pricing or policy change. | 20/20 | vendor page |
| Configuration lives in version controlRouting and budget rules are a file you keep, not dashboard state you would have to rebuild. | 16/16 | vendor page |
| Your request history can be exportedYou leave with your own logs instead of abandoning them. | 12/12 | vendor page |
All six inputs are published, so this is scored against the full 100. This measures technical switching cost only. It does not price the engineering time to re-test prompts against a different routing stack.
Full specification
Every field we track. Blank fields say "Not published" rather than "No" — we do not infer an absence from silence. Switch to Technical in the header for the precise field names and the low-level details.
Overview
- What kind of product Category
- Managed gateway Not verified
- Marketplaces resell many providers behind one key. Gateways add governance on top. Open-source projects you run yourself. Cloud platforms are hyperscaler surfaces. Inference providers host models on their own hardware.
- Who runs it Deployment model
- Managed or self-host Verified 2 days ago
- Managed means the vendor operates it. Self-host means you run it on your own infrastructure. Both means you can choose.
- Licence Licence
- Apache-2.0 Verified 2 days ago
- Proprietary products cannot be inspected or forked. Open licences such as MIT and Apache-2.0 let you audit, modify, and run the code without permission.
- Company Company
- Helicone (Mintlify) Verified 2 days ago
- The organisation that maintains the product.
- Who you would be signing with Vendor status
- Maintenance mode — no new features Not verified
- Whether the product is still an independent company, has been acquired, is a large cloud vendor’s product line, is run by a software foundation, or has been put into maintenance mode. Maintenance mode means bug fixes and security patches only — no new features.
- Last shipped an update Latest release
- 2025-08-21 Not verified
Latest tagged GitHub release v2025.08.21-1. The hosted changelog stops at 26 November 2025 — the stalest product in this catalogue by roughly a year.
- The date of the most recent release or version tag. A product that has not shipped in a year is a different risk from one that shipped last week, regardless of what its marketing site says.
- GitHub stars GitHub stars
- 6,109 Verified 2 days ago
- A rough proxy for community size on open-source projects. Not a quality measure.
Cost
- Markup on model prices Token markup
- None Verified 2 days ago
- How much the product adds on top of what the underlying model provider charges. Zero means you pay the same per-token price you would pay the model provider directly.
- Fee to add funds Credit purchase fee
- Not published Not verified
- A percentage charged when you top up your balance, separate from token prices. It is easy to miss because it does not appear on the per-token price list.
- Monthly cost per person Seat fee
- None Verified 2 days ago
- A recurring per-user platform charge that applies regardless of how much you use the models.
- Can use your own provider accounts BYOK supported
- Yes Verified 2 days ago
- Bring Your Own Key: you keep direct contracts with OpenAI, Anthropic and others, and the gateway only routes traffic. This preserves negotiated rates and committed-spend discounts.
- Cost of using your own accounts BYOK terms
- Bring your own provider keys, or add Helicone credits and pay provider rates with 0% markup while Helicone manages provider keys. Verified 2 days ago
- What the product charges to route traffic through your own provider keys.
- Free tier Free tier
- Hobby: free, 1 seat, 10,000 requests/month, 1 GB storage, 7-day retention, 10 logs/min ingestion. Verified 2 days ago
- What you can do without paying, useful for evaluation.
- Enterprise plan from Enterprise plan from
- Not published Not verified
- Annual entry price for the enterprise tier, where one is published or credibly reported.
- Cost to run it yourself Self-host cost
- Main platform repo is Apache-2.0 and self-hostable (infrastructure cost only); the standalone Rust AI gateway repo is GPL-3.0 and was last pushed 2025-11-21, which an independent review flags as effectively stale. Verified 2 days ago
- What self-hosting actually costs once you account for infrastructure and any paid tier.
- How the vendor makes money Pricing model
- Platform fee plus usage meters Not verified
- The shape of the vendor’s bill: does the routing layer charge a percentage on top of tokens, a flat monthly fee, both, neither (because inference is the product), or nothing at all (open source with no paid tier).
- How pricing works, briefly Pricing model detail
- Flat per-tier platform fee (Hobby free, Pro $79/mo, Team $799/mo, Enterprise custom) plus usage-based requests/storage. Gateway credits at 0% markup. 50% off first year for startups; $100 credit for OSS; free for students. Not verified
- A one-paragraph description that covers the caveats a pricing category cannot: introductory rates, per-feature meters, tier gating, and pricing that resets on a specific date.
- Minimum commitment Minimum commitment
- None stated for Pro/Team. Enterprise not stated. Not verified
- Whether the vendor requires a minimum contract term, a minimum spend, or a provisioned-capacity purchase to get its published rate.
- Charges that fire after you go over an allowance Overage terms
- Pro/Team/Enterprise all include 10K requests + 1 GB storage with 'usage-based ↑' after — the per-unit overage rates are not published. Calculator implies storage pricing (0.30 GB → $0.97) without a stated rate. Not verified
- The line items that scale with usage after an included allowance is exhausted — log storage, extra requests, per-feature meters, data export — which is where cost estimates usually go wrong.
- Prompt cache offered Cache mechanism
- Exact-match cache Not verified
- Whether the gateway offers its own response cache, what kind of match it does (exact request, prefix, semantic), or simply passes provider caching through unchanged.
- Discount on cached input Cache-read discount
- Not published Not verified
- How much cheaper cached tokens are than fresh input, when the vendor publishes a single figure. Bundled-inference clouds usually price this per model instead of as one number.
- Premium on cache writes Cache-write premium
- Not published Not verified
- How much more the first write of a cached prefix costs versus a plain input token. A high write premium and a low hit rate can leave you paying more than you save, so this matters as much as the read discount.
- Who captures the cache saving Cache economics
- Exact-match gateway cache on Cloudflare's edge; cache key hashes seed, URL, full body, relevant headers and bucket index. Semantic cache not stated. Can combine with provider prompt caching. Helicone charges no cache surcharge, but cached responses count against your storage allowance. Not verified
- Whether the customer keeps the full saving from caching or the vendor captures part of it — and any conditions attached (write premium, storage fees, best-effort hits).
- What you can split spend by Cost attribution
- Per user via Helicone-User-Id header; per tag/property via Custom Properties (project, feature, workflow stage, ticket). Per key/team not stated. Costs are estimates based on the model returned in the response body, not billed pass-through. Not verified
- The dimensions the vendor documents for splitting spend — per key, per user, per team, per tag, per customer. Matters if you need to chargeback internally or bill an end customer.
- How you get cost data out Cost export
- Query API for filtered retrieval; property-filtered webhooks. CSV, S3 and warehouse export not stated; pricing page 'Data export' line has no format, tier or price. Not verified
- The mechanisms the vendor publishes for exporting cost and usage data: CSV, an API, webhooks, S3, a data warehouse, or nothing at all. Any per-unit price is included.
- Who pays the model bill BYOK mode
- Your keys or their credits Not verified
- Whether you bring your own provider accounts (BYOK), buy inference from this vendor, or can do either. This is the single biggest commercial difference between these products: it decides who holds the contract with the model provider and who carries the spend.
Spend governance in detail
Seven signals matter when a bill starts to hurt: who can spend, how much, on what, and who gets paged when it goes wrong. Everything below is drawn from the vendor’s own pricing and docs pages — how we read these.
- Virtual or scoped keysNot published
Keys that carry their own budget and rate-limit policy, so an intern experiment cannot spend against a production budget.
Not stated.
- Budget caps per keyYes
A dollar or token ceiling attached to an individual key. Where enforcement is soft, one over-limit request still completes before the block kicks in.
Not stated as budgets, but rate limits can be denominated in cost — policies control 'request count or cost in cents within a time window' via Helicone-RateLimit-Policy, enforced pre-request with a 429.
- Budget caps per team or workspaceNot published
A ceiling applied at a higher scope than one key — a team, a workspace, a customer, or an entire environment.
Not stated.
- Rate limiting as a cost controlYes
Configurable request-per-time-window caps. Platform-set rate limits do not count as spend controls; user-configurable ones do.
Global, per-user and per-property segments; pre-request. Helicone's rate limit is bounded by your provider's policy.
- Model allowlistsNot published
A policy that constrains which models a key or team can call, keeping expensive frontier models out of the wrong hands.
Not stated.
- Spend alertsYes
Alerts fired as spend approaches a threshold. Alerts that only fire after the meter has rolled over are marked as such.
'Alerts' listed as a feature without tier marks.
- Webhook notificationsYes
Programmatic notifications on spend events, so budget breaches can page an on-call or open a ticket.
'Webhooks' listed as a feature without tier marks.
Enforcement:Enforced before each request
Catalog
- Models available Models available
- ~100 Verified 2 days ago
- How many models you can call, shown as a range because several vendors publish different totals on different pages. Vendor-reported either way, so counts are not directly comparable — some count every provider variant of the same model separately.
- Model providers reachable Upstream providers
- 20–100 Verified 2 days ago
- How many distinct model providers or labs you can reach, shown as a range where the vendor’s own pages disagree. More providers usually means better redundancy when one has an outage.
- Works with standard OpenAI code OpenAI-compatible API
- Yes Verified 2 days ago
- If yes, you can usually switch to it by changing one base URL, and switch away just as easily. This is the main defence against lock-in.
- OpenAI chat endpoint POST /v1/chat/completions
- Yes Not verified
- The endpoint almost every application ports first. "Not documented" means the vendor never states it, which is different from a documented no.
- Anthropic messages endpoint POST /v1/messages
- Yes Not verified
- Whether Anthropic-shaped calls work without rewriting them. Several products support this only as SDK compatibility or provider passthrough rather than a native endpoint — the detail page says which.
- OpenAI Responses endpoint POST /v1/responses
- Not documented Not verified
- The newer stateful OpenAI surface. Support is much thinner across this market than chat completions.
- Embeddings endpoint POST /v1/embeddings
- Not documented Not verified
- Whether you can generate vectors through the same gateway, or need a second integration for retrieval workloads.
- Image generation endpoint POST /v1/images/generations
- Not documented Not verified
- Whether image models are reachable through the same surface as text.
- Audio endpoints POST /v1/audio/*
- Not documented Not verified
- Speech-to-text and text-to-speech. Frequently the first gap in an otherwise complete gateway.
- Batch jobs endpoint POST /v1/batches
- Not documented Not verified
- Asynchronous bulk processing, usually at a discount. Commonly undocumented, and commonly the reason a migration stalls late.
- Needs the vendor’s own code library Requires a vendor-specific SDK
- No Verified 2 days ago
- A proprietary client library spreads through your codebase and has to be torn out again if you leave. “No” is the better answer here, and it means the standard OpenAI client works.
- You can export your request history Logs / usage data export
- Yes Verified 2 days ago
- Whether you can get your own request logs, traces, or usage records back out — through an API, a bulk export, or a download. Decides whether you leave with your history or abandon it.
- Settings can live in version control Declarative config-as-code
- Yes Verified 2 days ago
- Whether routing, fallback, and budget rules can be declared in a file you keep in Git, rather than existing only as settings clicked into a hosted dashboard.
- Embeddings Embeddings
- Not published Not verified
- Text-to-vector models, needed for search and retrieval features.
- Image generation Image generation
- Not published Not verified
- Whether image models are reachable through the same interface.
- Speech and audio Speech and audio
- Not published Not verified
- Text-to-speech or transcription models through the same interface.
- Video generation Video generation
- Not published Not verified
- Whether video models are reachable through the same interface.
- Batch processing Batch processing
- Not published Not verified
- Submitting large jobs for cheaper, slower processing. Often 50% off for work that is not time-sensitive.
Routing & reliability
- Uptime it promises in writing Contractual SLA uptime
- Not published Not verified
- The uptime percentage in a published, contractual service level agreement. A public status page is not an SLA — it reports what happened, it does not promise anything or pay you back when it breaks.
- Automatic failover Automatic failover
- Yes Verified 2 days ago
- When a model provider goes down or rate-limits you, traffic moves to a backup automatically instead of returning errors to your users.
- Load balancing Load balancing
- Not published Not verified
- Spreads requests across several providers or keys to raise your effective rate limit.
- Rule-based routing Conditional routing
- Not published Not verified
- Send different requests to different models based on rules — for example a cheap model for free users and a strong model for paying ones.
- Response caching Response caching
- Yes Verified 2 days ago
- Reuses the answer when the exact same request comes in again, which cuts both cost and latency.
- Similar-question caching Semantic cache
- Not published Not verified
- Reuses an answer when a new question means roughly the same thing as an earlier one. Saves far more than exact-match caching but can return subtly wrong answers if tuned loosely.
- Where you set the timeout Request timeout surface
- Not documented Not verified
`not_documented` as a settable value. Timeout appears only as a failover trigger: `408` "Timeout errors" causes the gateway to try the next provider ([Helicone provider routing](https://docs.helicone.ai/gateway/provider-routing)). Retry headers include `Helicone-Retry-Min-Timeout` / `Helicone-Retry-Max-Timeout`, but these bound backoff, not the request ([Helicone retries](https://docs.helicone.ai/features/advanced-usage/retries)).
- Where a request timeout can be set: per request, in a config file, in the vendor dashboard, or nowhere. Nine of the twenty products documented here do not describe a request timeout at all, so the worst case of a hung upstream call is unknowable from the docs.
- Where you set retries Retry policy surface
- Per request Not verified
`per_request` via headers: `Helicone-Retry-Enabled`, `Helicone-Retry-Num` (default `5`), `Helicone-Retry-Factor` (default `2`), `Helicone-Retry-Min-Timeout` (default `1000` ms), `Helicone-Retry-Max-Timeout` (default `10000` ms). Backoff is exponential: 1s → 2s → 4s → 8s → 10s, on `429`, `500`, `502`, `503`, `504` ([Helicone retries](https://docs.helicone.ai/features/advanced-usage/retries)).
- Where retry count and backoff are configured. Worth knowing alongside billing: a retried streaming call can be charged more than once.
- Where you set fallbacks Fallback surface
- Per request Not verified
`per_request` for the cloud gateway: ORDERED. `Helicone-Fallbacks` JSON header with `target-url`, `onCodes` and `bodyKeyOverride` per entry; the response header `Helicone-Fallback-Index` reports which entry served ([Helicone gateway fallbacks](https://docs.helicone.ai/getting-started/integration-method/gateway-fallbacks)). In the AI Gateway the ordered chain is expressed inside the `model` string, e.g. `model: "gpt-4o-mini/azure,gpt-4o-mini/openai,gpt-4o-mini"`, "tried in the exact order you specify" ([Helicone provider routing](https://docs.helicone.ai/gateway/provider-routing)).
- Where the fallback chain is defined. Almost every product claims fallback; the useful question is whether you can change it from code or only by hand in a dashboard.
- Shape of the fallback chain Fallback shape
- Ordered list Not verified
- An ordered list tries targets in sequence; a weighted split sends a percentage of traffic to each, which is what you need to trial a new model on 5% of requests. Weighted splits are much rarer than the marketing implies.
- Upstream health tracking Health checks / circuit breaking
- Fixed, cannot change Not verified
`not_configurable`, and it is passive unhealthy-provider avoidance rather than a configurable circuit breaker: the gateway is "always aware of provider uptimes and your rate limits" and instantly tries the next provider on `429`, `401`, `400`, `408`, `500+` ([Helicone/ai-gateway](https://github.com/Helicone/ai-gateway), [Helicone provider routing](https://docs.helicone.ai/gateway/provider-routing)).
- Whether the product notices a failing upstream and stops sending traffic to it, and whether you can tune the thresholds. This is what turns a provider outage into a blip rather than a sustained error rate, and only four of the twenty expose it.
- Cross-region failover you control Multi-region failover surface
- Not documented Not verified
`not_documented` as failover. You can pin a specific regional deployment (`model: "gpt-4o/azure/eu-frankfurt-deployment"`, "No data leaves the EU"), but that is region pinning, not cross-region failover ([Helicone provider routing](https://docs.helicone.ai/gateway/provider-routing)).
- Whether you can define what happens when a region degrades. A vendor running many regions is not the same as a vendor letting you configure failover between them; only three document a user-controlled mechanism.
- Where you set load balancing Load balancing surface
- In config Not verified
Weights ARE supported. YAML key `routers.<name>.load-balance.chat.strategy` with strategies: `model-latency`, provider latency P2C + PeakEWMA, weighted distribution based on model weight, and cost optimization ([Helicone/ai-gateway](https://github.com/Helicone/ai-gateway)). In the cloud gateway, "equal-cost providers are load balanced" and BYOK keys are always tried before Helicone managed keys ([Helicone provider routing](https://docs.helicone.ai/gateway/provider-routing)).
- Where traffic distribution across upstreams or keys is configured.
Operations
- Usage dashboards and logs Observability
- Yes Verified 2 days ago
- Built-in visibility into what was sent, what came back, what it cost, and how long it took.
- Spending limits Budget controls
- Not published Not verified
- Hard caps that stop spend before it becomes a surprise invoice. The single most valuable control for a small team.
- Rate limits Rate limits
- Yes Verified 2 days ago
- Caps on request volume per key or per user, useful for protecting against abuse and runaway loops.
- Separate keys per team or app Virtual keys
- Not published Not verified
- Issue scoped keys with their own budgets and permissions so you can attribute cost and revoke access without rotating everything.
- Prompt versioning Prompt management
- Yes Verified 2 days ago
- Store and version prompts outside your code so they can be changed without a deploy.
- Quality testing Evals
- Yes Verified 2 days ago
- Built-in tooling to score model output against test cases, so you can tell whether a model swap made things better or worse.
- MCP support MCP support
- Not published Not verified
- Native support for the Model Context Protocol, the emerging standard for connecting models to external tools.
- What gets logged Logged content
- Full prompts and responses Not verified
Request and response bodies (prompt and completion text) are logged unless omission is requested ([Omit logs](https://docs.helicone.ai/features/advanced-usage/omit-logs))
- Whether full prompts and responses are stored, only metadata, or nothing. Full-body logging is the most useful debugging feature here and the one most likely to need a conversation with your compliance team.
- You can turn logging off Body-logging opt-out
- Yes Not verified
Headers `Helicone-Omit-Request: true` and `Helicone-Omit-Response: true`, or `disable_content_tracing()` in the SDK, keep metrics while dropping bodies ([Omit logs](https://docs.helicone.ai/features/advanced-usage/omit-logs))
- Whether prompt and response bodies can be suppressed while still keeping usage metrics. Nineteen of the twenty document a way to do this; the mechanisms range from a per-request header to an organisation-wide setting.
- Traces you can take elsewhere Distributed tracing
- OpenTelemetry Not verified
The AI Gateway repo states "OpenTelemetry support for logs, metrics, and traces" alongside built-in Helicone observability; representation of multi-step agent traces is not detailed on the fetched pages ([Helicone AI Gateway repo](https://github.com/Helicone/ai-gateway))
- Whether the product emits OpenTelemetry, a proprietary format, or nothing. OpenTelemetry means the traces land in the tooling you already run instead of only in the vendor’s dashboard.
- Where telemetry can go Export destinations
- OpenTelemetry, Datadog, Webhook, PostHog Not verified
PostHog via `Helicone-Posthog-Key` / `Helicone-Posthog-Host` headers ([PostHog integration](https://docs.helicone.ai/gateway/integrations/posthog)); OpenTelemetry logs/metrics/traces from the gateway ([Helicone AI Gateway repo](https://github.com/Helicone/ai-gateway))
- Documented sinks for logs and metrics. This is a good proxy for how replaceable the vendor’s own dashboard is: around twenty destinations means you never have to depend on it, while a CSV download means you do.
- Can record user feedback Feedback capture API
- Yes Not verified
`POST /v1/request/{id}/feedback` with a boolean `rating` ([Feedback](https://docs.helicone.ai/features/advanced-usage/feedback)); arbitrary numeric scores via `POST /v1/request/{requestId}/score` ([Scores](https://docs.helicone.ai/features/advanced-usage/scores))
- Whether there is an API to attach a rating or score to a logged request, which is what lets production traffic feed quality work later.
- Scores live traffic Online eval hooks
- Partly Not verified
Scores can be pushed against logged requests, but the docs state plainly that "Helicone doesn't run evaluations for you" — you compute scores externally and post them ([Scores](https://docs.helicone.ai/features/advanced-usage/scores))
- Whether automated scorers can run against real production requests, rather than only against a test set you assemble yourself.
Performance
- Delay it adds Proxy overhead
- Not published Not verified
- Extra time the product itself adds to each request, on top of however long the model takes. Usually irrelevant next to multi-second model latency, but it matters for high-volume or streaming-sensitive workloads.
- Requests per second ceiling Throughput
- Not published Not verified
- Published sustained request rate before the product becomes the bottleneck. Only relevant at genuinely high volume.
- What the request path runs on Architecture class
- Compiled binary Not verified
"built on Rust"; repo language breakdown Rust 96.7% ([Helicone/ai-gateway](https://github.com/Helicone/ai-gateway)).
- The comparable way to talk about latency here. An edge worker, a compiled Go or Rust binary, and a Python proxy have different overhead floors no matter which figures each vendor publishes. Products that never disclose their runtime are recorded as undisclosed rather than assumed.
- You can run the request path yourself Self-hostable data plane
- Yes Not verified
`yes`. npm-launched binary `npx @helicone/ai-gateway@latest --config config.yaml`, plus Docker and Kubernetes self-hosting ([Helicone/ai-gateway](https://github.com/Helicone/ai-gateway)).
- Whether the component that actually carries your prompts can run on your own infrastructure. Distinct from a vendor offering a self-hosted control plane while still proxying traffic through their network.
- Streaming responses Streaming support
- Yes Not verified
Supported; the fetched pages do not state a streaming-specific caveat for the AI Gateway ([Helicone/ai-gateway](https://github.com/Helicone/ai-gateway)). Note the retry headers are documented independently of streaming ([Helicone retries](https://docs.helicone.ai/features/advanced-usage/retries)).
- Whether token-by-token streaming is documented. The caveats matter more than the yes: some products cannot cancel a stream without still being billed, and several timeout and fallback mechanisms stop applying once the first token has been sent.
Security & compliance
- Does your prompt reach their servers Prompt transits vendor
- Depends how you deploy it Not verified
Hosted cloud in the US or EU, or self-hosted entirely inside your own perimeter.
- Whether the text you send passes through this company’s own infrastructure. If it does, every other promise on this page is a policy commitment rather than a physical impossibility. Self-hosted products can answer no outright.
- What they keep if you change nothing Logging default
- Yes — prompts and replies Not verified
Bodies are stored unless you use Omit Logs. Note the limit of that feature: it stops storage but the request and response are still sent to Helicone's backend.
- Defaults matter more than options. A product that stores full prompts and replies unless you find the right header will have stored them by the time you read the docs.
- How long they keep it Default content retention (days)
- 7 days Not verified
Seven days on Hobby, one month on Pro, three months on Team, indefinite on Enterprise.
- Default retention for request content, in days. Zero means nothing is kept. Read the note: several products keep nothing as a rule but make timed exceptions for abuse review or specific models.
- Could they train on your prompts Training on customer data
- Not published — silence, not a no Not verified
No statement found in the privacy policy, terms or data-autonomy docs. Get it in writing.
- Whether the vendor may use your prompts and outputs to train models. “Not published” means we could not find any position, which is not the same as a no — ask for it in writing.
- Where it runs, and what you can pin Region and residency control
- US or EU for the hosted service, or self-host on your own Docker, Kubernetes or cloud. Not verified
- Which regions are offered and whether you can force processing to stay in one. A global endpoint that silently picks a region is a different compliance story from an endpoint you pin yourself.
- Where safety filters run Guardrail execution location
- No guardrails offered Not verified
No guardrails or PII-redaction feature found. The nearest control is omitting logs.
- A filter that strips personal data only helps if it runs before the data leaves your boundary. If guardrails execute in the vendor’s cloud, the vendor has already received whatever you wanted redacted.
- Who else touches the data Subprocessor list
- Not published Not verified
- The published list of third parties the vendor passes your data to. No list means you cannot know the full chain, which most data-protection agreements require you to.
- SOC 2 audited SOC 2 audited
- Yes Verified 2 days ago
- An independent audit of security controls. Enterprise buyers and their procurement teams routinely require it.
- Will sign a HIPAA agreement HIPAA BAA
- Yes Verified 2 days ago
- Required before you may send protected health information through the service. Without a signed BAA, healthcare data is off limits.
- GDPR commitments GDPR commitments
- Yes Verified 2 days ago
- Published data processing terms for handling personal data of people in the EU and UK.
- Can keep data in the EU EU data residency
- Yes Verified 2 days ago
- Requests can be processed inside the EU rather than routed to US infrastructure. Often the deciding constraint for European customers.
- Does not retain your data Zero data retention
- Yes Verified 2 days ago
- Prompts and responses are not stored after the request completes. Sometimes a paid add-on rather than the default.
- Strips personal data PII redaction
- No Verified 2 days ago
- Detects and removes identifiers such as names, emails, and card numbers before the request reaches the model provider.
- Content guardrails Content guardrails
- No Verified 2 days ago
- Policy checks on inputs and outputs — blocking unsafe content, enforcing formats, or catching prompt-injection attempts.
- Runs fully disconnected Air-gapped deployment
- Not published Not verified
- Can be deployed in a network with no internet access, which some regulated and defence environments require.
- Blocks personal data in prompts PII / DLP enforcement
- Not documented Not verified
No PII scanner documented; the privacy control is body omission rather than detection ([Omit logs](https://docs.helicone.ai/features/advanced-usage/omit-logs))
- Whether personal data detection sits on the request path and can stop the call, merely inspects and forwards it, or is not documented. A control that only reports is a logging feature, not a policy control.
- Blocks prompt injection Injection / jailbreak enforcement
- Not documented Not verified
`not_documented` ([Moderations](https://docs.helicone.ai/features/advanced-usage/moderations))
- Whether injection and jailbreak detection can stop a request. Most products offering this call a partner classifier rather than shipping their own.
- Blocks harmful content Toxicity / moderation enforcement
- Can block the request Not verified
With the header `Helicone-Moderations-Enabled: true`, Helicone calls OpenAI's moderation endpoint before the chat completion and returns HTTP 400 with `PROMPT_FLAGGED_FOR_MODERATION` when flagged, so the request never reaches the model ([Moderations](https://docs.helicone.ai/features/advanced-usage/moderations))
- Whether hate, violence, sexual and self-harm categories are checked inline and can stop a request, in either direction.
- Your own policy rules Custom policy hooks
- Not documented Not verified
No customer-defined regex/word-list/JSON-schema policy engine documented ([Moderations](https://docs.helicone.ai/features/advanced-usage/moderations))
- Whether you can add your own rule — a regex, a webhook, or your own classifier — rather than choosing from the vendor library.
- Where guardrails run Guardrail execution location
- Either, your choice Not verified
- Whether guardrail evaluation happens inside your infrastructure or on the vendor’s servers. This decides whether the prompt you are trying to protect leaves your network in order to be checked.
- If the guardrail itself fails Guardrail failure mode
- Not documented Not verified
No stated behaviour if the moderation call fails or times out ([Moderations](https://docs.helicone.ai/features/advanced-usage/moderations))
- What happens when the guardrail service times out or errors: does the request proceed unchecked, or is it blocked? This is the worst-documented field in the entire catalogue — only two vendors state it plainly, which means most teams are running a control whose failure behaviour they cannot know.
- Third-party guardrail vendors Guardrail integrations
- OpenAI Moderation Not verified
- Named external guardrail services the product can call. A long list means the product is a router for policy engines rather than a policy engine itself — which also means another vendor bill and another hop.
Compliance evidence
Graded by how strong the evidence is, not whether the word appears on the vendor’s website. An audited report and a marketing claim are different things, and only one of them will satisfy your own auditor.
- SOC 2 Certified Type II, report on request
- ISO 27001 Not published
- GDPR DPA Claimed, no evidence published described as GDPR-compliant in a blog post
- HIPAA BAA Claimed, no evidence published cloud described as HIPAA-compliant; no executable BAA offer found
- FedRAMP Not published
- ITAR Not published
Fit & integration
- Work to try it Evaluation work shape
- Change one base URL Not verified
- The shape of the work on the vendor’s own quickstart, from swapping one base URL through to deploying infrastructure. An ordinal class rather than a duration, because elapsed time depends on accounts and quota we cannot see.
- Work to run it Production work shape
- Change one base URL Not verified
- The same scale applied to the vendor’s recommended production path. For several products this is much heavier than the quickstart, which is exactly why both are recorded.
- Steps on the quickstart Numbered quickstart steps
- 2 Not verified
- A literal count of numbered steps on the vendor’s quickstart, recorded as evidence beside the work shape. Zero means the page publishes no numbered procedure at all. Large counts usually mean interleaved language tracks rather than more work.
- Can you self-host it today Self-host install documentation
- Install command published Not verified
`docker pull helicone/helicone-all-in-one:latest` then `docker run -d --name helicone -p 3000:3000 -p 8585:8585 -p 9080:9080 helicone/helicone-all-in-one:latest` ([Docker - Helicone](https://docs.helicone.ai/getting-started/self-host/docker))
- Whether an install command is actually published. Several products advertise self-hosting while publishing no command to start from, which a plain yes/no would hide.
- Works with the OpenAI SDK OpenAI SDK drop-in
- Yes Not verified
Helicone documents OpenAI SDK-compatible integrations by changing the endpoint and authentication. Documented base URLs include `https://oai.helicone.ai/v1`, `https://ai-gateway.helicone.ai`, `https://ai-gateway.helicone.ai/v1`, and `https://vercel.helicone.ai/v1`, depending on the integration. ([Helicone OpenAI Python integration](https://docs.helicone.ai/integrations/openai/python))
- Whether an existing OpenAI-compatible client can be pointed at it by changing the base URL and key.
- Vercel AI SDK support AI SDK provider package
- Official provider package Not verified
Helicone documents Vercel AI SDK integrations using `@helicone/ai-sdk-provider` with `createHelicone`, `@ai-sdk/openai` with `createOpenAI`, and `@ai-sdk/gateway` for gateway-specific functionality. ([Helicone Vercel AI SDK integration](https://docs.helicone.ai/gateway/integrations/vercel-ai-sdk))
- Whether a first-party AI SDK provider package exists, or only a community package, a documented workaround, or the generic OpenAI provider pointed at a custom base URL.
- Python framework integrations Documented Python frameworks
- LlamaIndex Not verified
LlamaIndex is documented with the packages `llama-index-llms-helicone`, `llama_index.core`, and `llama_index.llms.openai`, including the `Helicone` and `OpenAI` classes. LangChain and LlamaIndex are also discussed in a comparison blog post, which mentions `PromptTemplate`, `LLMChain`, `IndexToolConfig`, and `LlamaIndexTool`; the LangChain details are from that comparison post rather than a dedicated integration page. ([Helicone LlamaIndex integration](https://docs.helicone.ai/gateway/integrations/llamaindex))
- LangChain, LangGraph, LlamaIndex and similar orchestration frameworks with a documented integration.
- Callable from Cloudflare Workers Cloudflare Workers support
- Their gateway runs on Workers, not yours Not verified
Helicone states that its proxy leverages Cloudflare Workers running on Cloudflare’s global network to process requests at the edge. ([Helicone latency reference](https://docs.helicone.ai/references/latency-affect))
- Whether the docs show calling this product from your own Worker. Deliberately separated from the several products whose own gateway runs on Workers, which is a fact about their infrastructure and not about your edge compatibility.
- Kubernetes install Helm chart availability
- Official Helm chart Not verified
Helicone explicitly supports self-hosting on Kubernetes and provides an official Helm chart. The chart repository is the Helicone Helm repository, with components including `helicone-core`, `helicone-infrastructure`, `helicone-monitoring`, and `helicone-argocd`; the Helm deployment uses `helm upgrade --install helicone ./helicone -f values.yaml`. ([Helicone Kubernetes self-hosting](https://docs.helicone.ai/getting-started/self-host/kubernetes))
- Whether a named, published Helm chart exists, versus Helm being referenced with no chart named, versus generic cluster documentation that has nothing to do with this product.
- Terraform support Terraform provider or modules
- Terraform code in the repo only Not verified
Helicone documents Terraform directories for creating AWS S3, Aurora, and EKS resources: `terraform/s3`, `terraform/aurora`, and `terraform/eks`. An official Helicone Terraform provider, module, or Terraform Registry name is not stated. ([Helicone Kubernetes self-hosting](https://docs.helicone.ai/getting-started/self-host/kubernetes))
- Whether you can declare this in code: an official provider, official modules, resources inside a hyperscaler’s provider, a community provider, or only Terraform code shipped in a repo.
- Reuses your cloud identity Cloud IAM reuse
- Not documented Not verified
n.a. (not documented)
- Whether you can authenticate with IAM roles, workload identity or managed identities instead of another long-lived API key. Distinguished from products that only accept static upstream provider credentials.
- Fits behind your API gateway API gateway integration
- Not documented Not verified
n.a. (not documented)
- Whether AI traffic can go through a gateway you already run, and who documented that — the product’s vendor or the gateway’s.
- MCP support MCP surface shape
- MCP client tooling Not verified
Helicone provides an MCP server available as the npm package `@helicone/mcp`, including the tools `query_requests` and `query_sessions`. It supports MCP-compatible clients such as Claude Desktop, Claude Code, Cursor, and Codex, and `query_sessions` provides session search, filtering, pagination, and session performance metrics. ([Helicone MCP integration](https://docs.helicone.ai/integrations/tools/mcp))
- Which kind of MCP support this is: a gateway that governs many MCP servers, a hosted MCP server you connect to, MCP tools accepted inside the completion API, or client tooling. These are different products behind one acronym.
- Needs your own provider key Upstream provider key required
- Not hosted, yes self-hosted Not verified
No for the hosted gateway ("Want more control? You can bring your own provider keys instead," presented as optional) ([Quickstart - Helicone](https://docs.helicone.ai/getting-started/quick-start)). Yes in the self-hosted Docker path, whose first call sends `Authorization: Bearer $OPENAI_API_KEY` alongside `Helicone-Auth` ([Docker - Helicone](https://docs.helicone.ai/getting-started/self-host/docker))
- Whether an upstream provider account and key must exist before your first call works. A prerequisite rather than a step, and it can differ between the hosted and self-hosted forms of the same product.
- Gate before models work Model access gate
- No gate Not verified
n.a. (not documented). No approval/quota/waitlist step; users can "Switch between 100+ models by just changing the model name" ([Quickstart - Helicone](https://docs.helicone.ai/getting-started/quick-start))
- Whether an enablement click, a quota grant, a paid tier or an approval form stands between a valid key and a working model call.
- Official client languages First-party client SDK languages
- Python, TypeScript Not verified
Documented clients and packages include the OpenAI Python package (`openai`, `from openai import OpenAI`), the OpenAI JavaScript package (`openai`), `@helicone/ai-sdk-provider`, `@ai-sdk/gateway`, `@ai-sdk/openai`, `llama-index-llms-helicone`, `boto3`, and `helicone-async`. Bedrock examples also document the classes `BedrockRuntimeClient`, `InvokeModelCommand`, `ConverseCommand`, and `HeliconeAsyncLogger`. ([Helicone client integrations](https://docs.helicone.ai/integrations/openai/python))
- Languages with a first-party client library. An empty list can still mean the product is usable from any language via an OpenAI-compatible SDK.
Additional charges
These are the fees that do not appear on a per-token price list, and they are where estimates usually go wrong.
- Pro plan $79/month, unlimited seats, 10k requests + 1 GB free then usage-based
- Team plan $799/month, 5 organizations, 3-month retention
- Usage overage Metered per request and per GB of storage beyond included amounts
How pricing actually works
Main platform repo is Apache-2.0 and self-hostable (infrastructure cost only); the standalone Rust AI gateway repo is GPL-3.0 and was last pushed 2025-11-21, which an independent review flags as effectively stale.
Official links
- Website helicone.ai ↗
- Documentation docs.helicone.ai ↗
- Pricing helicone.ai ↗
- Status page status.helicone.ai ↗
- Source code github.com ↗
- Models API api.helicone.ai ↗
6,109 GitHub stars — a proxy for community size, not for quality.
Independent coverage
Third-party analysis, walkthroughs and operator threads. We link criticism as readily as praise. Nothing here is written or published by the vendor, by a competitor listed on this site, or by an SEO content farm — how we vet these.
Written reviews and analysis 1
- Helicone's Migration from Postgres to ClickHouse for Advanced LLM Monitoring Third-party engineering writeup of Helicone's proxy/observability architecture and why its request-log workload outgrew Postgres, with the actual migration path described.
What has changed here
No tracked value on this page has moved since it was first researched. Every figure still carries its original citation and check date.
Back to top ↑Read the head-to-head
These pairs have a written verdict, not just a table.