Managed only Proprietary

Fireworks AI Inference provider

Name: Fireworks AI
Author: Fireworks AI

Fireworks AI is an inference provider: an OpenAI-compatible API in front of ~100 models. You pay for inference itself rather than a routing fee. It cannot be self-hosted. Zero data retention, a HIPAA BAA and SOC 2 are published. Beyond chat it also serves embeddings, image generation, audio and batch jobs.

Last verified August 29, 2026 · 29 of 113 fields dated · 43 sources

US company

Compare with others Estimate cost Visit site ↗

Access at a glance

Whether this product can work for you at all, before features matter: who pays the model bill, where it can run, and which of your existing API calls keep working. Every value is the vendor’s own claim, linked to the page it came from.

"Fireworks AI is the fastest platform for building with open source AI models" (Fireworks introduction), exposing a "Fireworks AI REST API" for "various language, image and embedding models using an API Key" that "also lets you automate management of models, deployments, datasets, and more" (Fireworks API reference introduction). It is a first-party inference and training platform, not a router.

Who pays the model bill

Their credits only

You buy inference from this vendor. There is no path to bring your own provider account.

In the sense that all usage is billed by Fireworks on its own API key: serverless is "pay per token" with postpaid billing and "$1 in free credits", on-demand is "pay per GPU second" (Fireworks pricing, 2026-05-22; Models overview). There are no upstream provider keys to bring; requests authenticate with FIREWORKS_API_KEY (On-demand deployments).

Merchant of record: Fireworks: it bills serverless usage per token (postpaid), dedicated/on-demand per GPU-second with "no extra charges for start-up times", fine-tuning per 1M training tokens and reinforcement fine-tuning per GPU hour billed per second, with preemptible capacity carrying "no charge to hold dedicated capacity" (Fireworks pricing, 2026-05-22; On-demand deployments).

Key handling: Single-vendor key model: API keys are created with firectl api-key create or in the dashboard at https://app.fireworks.ai/settings/users/api-keys, and every request carries Authorization: Bearer <API_KEY>; account-scoped quota APIs (List/Get/Update Quota) exist (Fireworks API reference introduction). No third-party provider secrets are stored, and inference necessarily runs on Fireworks infrastructure, so Fireworks processes request content; the models overview asserts "your data is yours" (Models overview). Encryption/KMS specifics: n.a.

Where it can run

1 of 5 shapes documented

Vendor-hosted
Self-host
Your VPC
On-premise
Air-gapped

Dimmed shapes are not documented by the vendor, which is not the same as unsupported.

hosted cloud service in two shapes: serverless inference ("use popular models instantly with pay-per-token pricing", pre-deployed by Fireworks and shared across users) and dedicated GPU deployments ("deploy with high performance on dedicated GPUs with fast autoscaling and minimal cold starts"), plus training (Fireworks introduction; Models overview). Self-host, hybrid/VPC, on-prem, air-gapped: n.a. — the models overview page explicitly does not mention any of them (Models overview). Enterprise deployments are quote-based: "contact us for enterprise deployments with faster speeds, lower costs, and higher rate limits" (Fireworks pricing, 2026-05-22).

Serverless models are "pre-deployed by the Fireworks team for the community to use", shared, with "no SLA guarantees for up-time or latency", best-effort availability and at least two weeks' deprecation notice; "neither custom base models nor LoRA addons are supported for serverless inference" and "all user-provided models, including trained models, require a dedicated deployment" (Models overview). Dedicated deployments are "private to you", give access to "a wide array of hardware", support base models and LoRA addons, and are billed by GPU-second; deployments are addressed as accounts/<ACCOUNT_ID>/deployments/<DEPLOYMENT_ID> or accounts/fireworks/models/gpt-oss-120b#<DEPLOYMENT_NAME> (Models overview; On-demand deployments). Region-restricted deployments carry a 1.5x premium (Fireworks pricing, 2026-05-22).

API surfaces your code can keep using

5 of 7 documented

OpenAI chatPOST /v1/chat/completionsYes
POST https://api.fireworks.ai/inference/v1/chat/completions, configured as an OpenAI SDK base_url/baseURL of https://api.fireworks.ai/inference/v1 (On-demand deployments).
Anthropic messagesPOST /v1/messagesNot documented
n.a. (not documented on the pages fetched).
OpenAI ResponsesPOST /v1/responsesNot documented
n.a.
EmbeddingsPOST /v1/embeddingsYes
Embedding models are part of the REST API and are priced per 1M input tokens by base-model size (up to 150M: $0.008; 150M–350M: $0.016; Qwen3 8B: $0.10), with "embeddings & reranking" positioned for "search & context retrieval" (Fireworks API reference introduction; Fireworks pricing, 2026-05-22; Fireworks introduction).
ImagesPOST /v1/images/generationsYes *
Yes as a supported model family: the REST API covers "language, image and embedding models" and the catalog spans "text, vision, audio, image, and embeddings" (Fireworks API reference introduction; Fireworks introduction). Endpoint path: n.a.
AudioPOST /v1/audio/*Yes *
Yes as a supported modality ("text, vision, audio, image, and embeddings"), but no STT/TTS endpoint path is documented on the pages fetched (Fireworks introduction).
Batch jobsPOST /v1/batchesYes
"Batch Inference — run async inference jobs at scale, faster and cheaper" (Fireworks introduction). Endpoint path: n.a.

An asterisk marks a qualified verdict: support that is indirect (SDK compatibility or provider passthrough rather than a native endpoint), or a gap that is narrower or wider than the label suggests. Read the note before porting.

Drop-in base-URL swap with the OpenAI SDK (https://api.fireworks.ai/inference/v1) or curl; a first-party Fireworks Python SDK also exists but is not required (On-demand deployments). All requests need Authorization: Bearer <API_KEY> and Content-Type: application/json (Fireworks API reference introduction). Operational caveat: a deployment scaled to zero returns "a 503 error immediately" while it scales up, so "applications should implement retry logic" (On-demand deployments). Models must be deployed before use — "a model must be deployed before it can be used for inference" (Models overview).

How much it reaches

Models~100

Upstream providersNot published

"100+ Supported Models" spanning "text, vision, audio, image, and embeddings" (Fireworks introduction, undated). The pricing page names specific models (GLM 5.3, Qwen3 8B, Qwen3-235B, gpt-oss-120B, DeepSeek V3, Kimi K2, Qwen 3.5 9B, Qwen 3.6 27B, Kimi K3) without a total (Fireworks pricing, 2026-05-22).

First-party only: Fireworks serves models on its own platform ("the fastest platform for building with open source AI models") and does not route to third-party provider APIs, so no provider count exists (Fireworks introduction; Models overview).

Whose models: Vendor-operated hardware serving open-weight and customer models: deployments provide "dedicated GPUs for your models" and preemptible deployments borrow "idle reserved GPUs"; Fireworks maintains "a library of common base models" and customers can add their own (On-demand deployments; Models overview). Whether that hardware is Fireworks-owned or leased from clouds is not stated: n.a. (On-demand deployments).

Your own endpoints: Bring-your-own-weights rather than bring-your-own-endpoint: "upload your own models (for supported architectures) from Hugging Face or elsewhere", plus LoRA addon upload/training, all requiring a dedicated deployment (On-demand deployments; Models overview). Registering an external self-hosted endpoint (vLLM, Ollama, SageMaker, private URL): n.a.

How it behaves in production

What happens when an upstream model is slow, wrong, or down — and what you can see and stop while it happens. Reliability features are recorded as where you configure them, not whether the vendor lists them, because almost every product here lists all of them.

1 of 6 reachable from codenothing documented on the request pathno documented export

When something goes wrong

Each row says where the knob is, not whether the feature is on the marketing page. A control you can only reach by hand in someone else’s dashboard cannot be reviewed or version-controlled.

Request timeoutNot documented
The vendor does not document this, so any behaviour you observe today is unversioned and may change.
not_documented for inference requests. The only documented timeout is control-plane: --wait-timeout on firectl deployment create with --wait, default 1 hour (Fireworks on-demand deployments).
RetriesNot documented
not_documented as a platform feature; it is pushed to the client: when a deployment is scaled to zero, requests return 503 immediately and "Your application should implement retry logic to handle this". Default count and backoff: n.a. (Fireworks on-demand deployments).
Fallback to another modelNot documented
No model or provider fallback list (Fireworks on-demand deployments).
Load balancingNot documented
Deployment shapes can be optimized for speed, throughput or cost, but no load-balancing policy or weights (Fireworks on-demand deployments).
Upstream health trackingNot documented
not_documented as configurable. Deployment health is observable via state (CREATING, READY, UPDATING, DELETING, DELETED, FAILED) and replica counts (min_replica_count, ready_replica_count, desired_replica_count, max_replica_count), with guidance to "Verify it's healthy" before cutting traffic over — this is deployment status, not upstream ejection or circuit breaking (Fireworks on-demand deployments).
Cross-region failoverIn config
Changing it means editing configuration and shipping it, so behaviour is uniform across traffic until you redeploy.
config_file at creation time, and it is genuinely a capacity-failover control: --region accepts GLOBAL, US, EUROPE, APAC; GLOBAL is recommended for production workloads needing "geographic availability or capacity failover". Without --region a deployment is "pinned to a single datacenter at creation time and will not be automatically migrated later", and placement "cannot be changed in place" — you create a replacement deployment (Fireworks on-demand deployments).

Reliability here is deployment topology, not routing config: choose --region GLOBAL at creation, avoid preemptible for production, keep min_replica_count above zero to avoid cold 503s, and implement your own retries. Region placement being immutable is the biggest planning constraint (Fireworks on-demand deployments).

How fast the hop is

Undisclosed vendor service

The vendor does not disclose what the request path runs on, so no overhead floor can be inferred at all.

vendor_saas — an inference provider, not a gateway; the serving runtime is described by engine name (FireAttention V4) rather than a disclosed process runtime, and the control surface is the firectl CLI plus https://api.fireworks.ai/inference/v1 (Fireworks FireAttention V4, Fireworks on-demand deployments).

You can run the request path yourselfNo

StreamingYes

Fireworks distributes firectl binaries (Homebrew, macOS arm64/x86_64, Linux x86_64, Windows 64-bit) and an open-source benchmarking tool, but not a self-hostable serving plane (firectl CLI, Fireworks performance benchmarking).

Streaming caveats: Streaming is available through the OpenAI-compatible API surface; the fetched pages state no streaming-specific caveat, but they do state hard availability caveats: "A preemptible deployment can be preempted mid-request and disappear with no warning", "Never point production or latency-sensitive traffic at a preemptible deployment", and scaled-to-zero deployments return 503 immediately (Fireworks on-demand deployments).

Published figures, grouped by what each one measured. Figures in different groups are different quantities and cannot be compared with one another — nor, in most cases, with another vendor’s figure in the same group.

Model output speed

How fast the model emits tokens. This is an inference-provider metric and has nothing to do with routing overhead.

>250 tokens/soutput speedVendor-published
DeepSeek V3 0324, FireAttention V4 at FP4, NVIDIA B200, 8 GPUs with NVLink, speculation disabled.
Source

Vendor self-published, and explicitly comparative against rival stacks (SGLang on H200 FP8, TRT-LLM on B200 FP4). Fireworks itself flags the comparison as incomplete: "We were not able to fully validate TRT-LLM FP4 performance on B200 due to setup complexity", plus "Speculation success being very prompt-dependent makes it hard to compare across implementations" (Fireworks FireAttention V4). Fireworks also appears as a compared provider in Groq's cited ArtificialAnalysis.ai benchmark — a third-party figure published by a competitor (Groq LPU benchmark blog).

What it will stop

nothing documented on the request path

No request-path policy controls are documented. That is not a fault in a product built purely for routing — but it means anything you need blocked has to be blocked before the call reaches here.

What you can see

No documented export

What gets loggedMetadata only

Token counts, latency and model names are stored, but not the text itself.

You can turn bodies offYes

store=False on Responses API calls prevents storage; ZDR is the default elsewhere (Data handling)

TracesVendor format only

Via integrations rather than natively: Weights & Biases Weave and MLflow tracing are documented (MLOps and observability integrations)

Zero data retention applies by default and prompts/generations are not logged; the exception is the Responses API, where store=True is the default and stored responses are deleted after 30 days (Data handling)

Where telemetry can go

No documented export. Whatever this product records stays in its own interface, so it cannot become part of the monitoring you already run.

A Prometheus-compatible metrics endpoint at https://api.fireworks.ai/v1/accounts/<account_id>/metrics, scrapeable by Prometheus, the OpenTelemetry Collector, the Datadog Agent, Vector, Grafana, and New Relic (Exporting metrics); W&B and MLflow for traces (MLOps and observability integrations)

Records user feedbackNo

Scores live trafficNo

n.a. (MLOps and observability integrations)

n.a. — evaluation happens in W&B/MLflow, not against Fireworks-stored traffic (there is none by default) (MLOps and observability integrations, Data handling)

Retention: No prompt/generation retention by default; Responses API stored objects are deleted after 30 days (Data handling)

Whether it fits how you work

How much work stands between you and a first call, how different that is from running it in production, and whether it slots into the stack you already have. Recorded as the shape of the work rather than a number of minutes — how long it takes you depends on which accounts and quota you already hold, which no comparison can know.

to try: sdk installto run: sdk installfits 5 of 10 common stacks

Getting to a first call

2 numbered steps

Shape of the workInstall a package

You add a dependency and construct the vendor’s client. Small, but it is a code change and a package you now maintain.

Read off: the vendor’s own quickstart — 2 numbered steps.

Before step one

Your own provider keyNot needed
You can make a first call with only this product’s key. No upstream provider account needed.
. A Fireworks API key is used (Fireworks quickstart)
Payment methodNo card needed to start
Not stated as required: "Get started with $1 in free credits." (Fireworks - Pricing)
Gate before models answerNo gate
Every catalogue model is callable as soon as you have a key.
n.a. (not documented). No approval, enablement, quota or region gate on either page (Fireworks quickstart, Fireworks - Pricing)

Everything you need first: A Fireworks API key created in the dashboard and exported as FIREWORKS_API_KEY. Account, credit card, cloud account, cluster and provider key are not mentioned as prerequisites (Fireworks quickstart)

The vendor’s own time claim: Vendor claims, verbatim: "Make your first Serverless API call in minutes" and "help you make your first API call in minutes" (Fireworks quickstart); pricing page adds "Get started in seconds with per token pricing, zero setup and no cold starts" (Fireworks - Pricing) Quoted, not verified. Marketing time claims assume every account and approval is already in place.

Running it in production

The same scale applied to the path the vendor recommends for production traffic. Kept separate from the quickstart because for several products here the two are barely related pieces of work.

Shape of the workInstall a package

You add a dependency and construct the vendor’s client. Small, but it is a code change and a package you now maintain.

What production needs: n.a. (not documented). No database, Redis, cluster, load balancer, control plane or licence requirement is stated (Fireworks quickstart, Fireworks - Pricing)

Can you run it yourself

Install command publishedNo self-hosting

This runs on the vendor’s infrastructure only.

How it fits your stack

5 of 10

Each row is a thing you might already run. “With a caveat” means it works but not the way the vendor’s marketing implies — read the reason, because that is usually where the surprise lives.

FitsThe OpenAI SDKDrop-in once set up — but first-call work is sdk install.
FitsThe Vercel AI SDK@ai-sdk/fireworks
NoCloudflare WorkersNo Workers guidance published.
With a caveatKubernetesHelm is supported but no chart is named.
With a caveatTerraform or OpenTofuOfficial modules
With a caveatAn existing API gatewayDocumented by API7, not this vendor.
FitsCloud IAM I already runReuses IAM roles, workload identity or managed identities.
FitsLangChain or LlamaIndexLangChain, LlamaIndex, CrewAI, PydanticAI, Strands
With a caveatMCP servers to governMCP tools in the API — governs nothing on your side.
FitsNothing — plain Node or PythonInstall a package.

Reading this the other way round — pick what you already run and see every product scored against it.

The integration surfaces behind those answers

Vercel AI SDKOfficial provider package
Install the package, swap the model factory, done. Maintained by a party with a stake in it.
Named: @ai-sdk/fireworks
Fireworks has an official Vercel AI SDK provider package, @ai-sdk/fireworks, with provider instances fireworks and createFireworks. (Vercel AI SDK Fireworks provider)
Cloudflare WorkersNot documented
No Workers guidance either way. If you are edge-first, verify fetch-only compatibility yourself.
n.a. (not documented)
KubernetesHelm supported, chart not named
Helm is referenced but no chart or repository is named, so you cannot inspect it in advance or pin a version.
Named: Helm charts and templates for EKS BYOC clusters; no chart name or repo stated
Fireworks documents Kubernetes and Amazon EKS deployments, including deployment with Helm charts or Helm deployment templates. No official chart name, repository, or operator name is stated. (Fireworks airgapped EKS deployment)
TerraformOfficial modules
Vendor-published modules exist. Some are off-Registry Git references, so pin them explicitly.
Named: fw-ai-external/aws-cluster/fireworksai on the Registry, example version 0.1.2
Fireworks documents the official Terraform module fw-ai-external/aws-cluster/fireworksai at registry.terraform.io/modules/fw-ai-external/aws-cluster/fireworksai/latest, with example version 0.1.2. (Fireworks EKS BYOC documentation)
Existing API gatewayThe gateway vendor documents it
The integration is documented by the gateway vendor rather than this one. A working recipe, without a support commitment from this product’s vendor.
Documented by:API7
Not documented by Fireworks, but API7's APISIX-based AI gateway documents a fireworks-ai catalog provider that connects through the openai adapter with api_base = https://api.fireworks.ai/inference/v1. (API7 AI Gateway - Fireworks AI provider)
Cloud identityReuses your cloud identity
Authenticate with the identity you already run — IAM roles, workload identity or managed identities. No long-lived key to rotate.
For BYOC on AWS, Fireworks documents AWS IAM roles for the cluster and support for AWS_PROFILE when firectl uploads model files to S3. BYOC API examples use api_key="unused" because the API key is not used, while airgapped deployment stores a metering key in AWS Secrets Manager. (Fireworks EKS BYOC documentation)
MCPMCP tools in the API
The completion API accepts MCP tool definitions, so the model can call MCP tools. A model capability, not an MCP control plane.
Fireworks supports MCP tools in its Responses API; the tools field accepts MCP tools and response tools support the mcp type. Fireworks also documents MCP server usage with an example SSE server URL such as https://gitmcp.io/docs. (Fireworks Responses API reference)

Python frameworks

LangChain
LlamaIndex
CrewAI
PydanticAI
Strands

Fireworks documents integrations with LangChain and LlamaIndex, and also lists CrewAI, PydanticAI, Strands Agents, and AWS AgentCore as supported agent frameworks. The LangChain JavaScript integration uses package @langchain/fireworks and class Fireworks; package and class names for the other documented integrations are not stated. (Fireworks agent-framework integrations)

First-party client libraries

Python
TypeScript

Documented clients include the Fireworks Python SDK, package fireworks-ai with class Fireworks; the OpenAI Python package openai with class OpenAI; the Anthropic Python package anthropic with class anthropic.Anthropic; the OpenAI JavaScript/TypeScript package openai with class OpenAI; and the Anthropic JavaScript/TypeScript package @anthropic-ai/sdk with class Anthropic. Fireworks also documents the @ai-sdk/fireworks JavaScript/TypeScript provider and curl. (Fireworks quickstart)

Agent features: The Fireworks Responses API supports tool calls, max_tool_calls, parallel tool calls, tool_choice, streaming, and conversational continuation with previous_response_id. Fireworks also describes server-side handling of the agentic loop—reasoning, tool selection, and execution—and server-side state for long conversations. (Fireworks Responses API reference)

The Python SDK is in alpha, so installs need --pre (pip install --pre fireworks-ai, poetry add --pre fireworks-ai, uv add --pre fireworks-ai); OpenAI/Anthropic-compatible paths avoid the alpha SDK (pip install openai, pip install anthropic, npm install openai, npm install @anthropic-ai/sdk). Priority tier is selected with service_tier: "priority" (Fireworks quickstart)

Fireworks documents hosted serverless API endpoints and BYOC deployments, including private-preview deployments in customer-managed cloud, managed Kubernetes, GPU-cloud, or on-premises environments. Airgapped EKS deployments keep images, models, and dependencies in the organization’s infrastructure and cannot be managed through the Fireworks web app or firectl; the Fireworks Python SDK is documented as alpha and requires --pre for the latest version. (Fireworks BYOC setup)

Silence in the docs: 1 of the integration questions on this card have no published answer either way. That is recorded as undocumented, not as a no — but it does mean you would be verifying it yourself.

What it does well

100+ open-weight models with a drop-in OpenAI-compatible API
Selectable DEFAULT, PRIORITY and FAST service tiers plus batch inference for cost control
Both supervised and reinforcement fine-tuning, with managed LoRA priced from $0.50 per 1M tokens
States SOC 2 Type II and HIPAA compliance, with a public trust center and documented CMEK encryption states
Region-restricted deployments available when workloads must stay in a specific geography

Where it falls short

Single-source provider: no closed frontier models, no independent catalog, and no cross-provider routing or failover
Only a one-time $1 evaluation credit, which is too small for realistic load testing
GDPR, EU residency specifics and zero-data-retention are not documented on vendor pages found
Serverless is multi-tenant and priority tiers cost more, so predictable latency effectively requires paying up or going dedicated

Choose it when

Teams wanting fast serving of open-weight models with tiered latency options and both supervised and reinforcement fine-tuning.

Look elsewhere when

You need frontier closed models, multi-provider failover, or documented EU data residency and zero retention.

Inference provider: Hosts open-weight models on its own hardware. Often the cheapest or fastest route to a specific open model, but it is one source, not a router.

How hard is it to leave?

Derived from six published facts, not from an opinion. The weights are fixed and the same for every product — see the arithmetic.

44/64Hard to leave

Portability score breakdown for Fireworks AI
What helps you leave	Points	Source
Works with standard OpenAI codeSwitching away is a base-URL change rather than a rewrite of every call site.	22/22	vendor page
No vendor-specific SDK requiredA proprietary client library spreads through your codebase and has to be torn out again.	10/10	vendor page
Can use your own provider accountsYour keys and billing relationship stay yours, so removing the gateway does not cut off model access.	not published	—
Can be self-hostedYou can run it yourself instead of accepting a pricing or policy change.	0/20	—
Configuration lives in version controlRouting and budget rules are a file you keep, not dashboard state you would have to rebuild.	not published	—
Your request history can be exportedYou leave with your own logs instead of abandoning them.	12/12	vendor page

Read the fine print: Export covers rated usage and cost via firectl, not request logs.

2 of the 6 inputs are not published, so the highest reachable score here is 64 rather than 100. That is a gap in the public documentation, not a mark against the product — no points are deducted, they simply cannot be claimed. This measures technical switching cost only. It does not price the engineering time to re-test prompts against a different routing stack.

Full specification

Every field we track. Blank fields say "Not published" rather than "No" — we do not infer an absence from silence. Switch to Technical in the header for the precise field names and the low-level details.

Overview

What kind of product Category: Inference provider Not verified; Marketplaces resell many providers behind one key. Gateways add governance on top. Open-source projects you run yourself. Cloud platforms are hyperscaler surfaces. Inference providers host models on their own hardware.
Who runs it Deployment model: Managed only Not verified; Managed means the vendor operates it. Self-host means you run it on your own infrastructure. Both means you can choose.
Licence Licence: Proprietary Not verified; Proprietary products cannot be inspected or forked. Open licences such as MIT and Apache-2.0 let you audit, modify, and run the code without permission.
Company Company: Fireworks AI Verified 3 days ago; The organisation that maintains the product.
Who you would be signing with Vendor status: Independent company Not verified; Whether the product is still an independent company, has been acquired, is a large cloud vendor’s product line, is run by a software foundation, or has been put into maintenance mode. Maintenance mode means bug fixes and security patches only — no new features.
Last shipped an update Latest release: 2026-08-27 Not verified; The date of the most recent release or version tag. A product that has not shipped in a year is a different risk from one that shipped last week, regardless of what its marketing site says.
GitHub stars GitHub stars: Not published Not verified; A rough proxy for community size on open-source projects. Not a quality measure.

Cost

Markup on model prices Token markup: Not published Not verified; How much the product adds on top of what the underlying model provider charges. Zero means you pay the same per-token price you would pay the model provider directly.
Fee to add funds Credit purchase fee: Not published Not verified; A percentage charged when you top up your balance, separate from token prices. It is easy to miss because it does not appear on the per-token price list.
Monthly cost per person Seat fee: Not published Not verified; A recurring per-user platform charge that applies regardless of how much you use the models.
Can use your own provider accounts BYOK supported: Not published Not verified; Bring Your Own Key: you keep direct contracts with OpenAI, Anthropic and others, and the gateway only routes traffic. This preserves negotiated rates and committed-spend discounts.
Cost of using your own accounts BYOK terms: Not published Not verified; What the product charges to route traffic through your own provider keys.
Free tier Free tier: $1 in free credits on signup, then postpaid billing. Verified 3 days ago; What you can do without paying, useful for evaluation.
Enterprise plan from Enterprise plan from: Not published Not verified; Annual entry price for the enterprise tier, where one is published or credibly reported.
Cost to run it yourself Self-host cost: No self-host option; Fireworks sets its own per-token serverless prices rather than marking up another vendor's list price, with DEFAULT, PRIORITY and FAST service tiers resolved live per model. Dedicated capacity is billed per GPU-hour, and batch inference plus supervised and reinforcement fine-tuning are priced separately. Verified 3 days ago; What self-hosting actually costs once you account for infrastructure and any paid tier.
How the vendor makes money Pricing model: Bundled — inference is the product Not verified; The shape of the vendor’s bill: does the routing layer charge a percentage on top of tokens, a flat monthly fee, both, neither (because inference is the product), or nothing at all (open source with no paid tier).
How pricing works, briefly Pricing model detail: Pay-for-own-inference (Fireworks is the provider). Per-token serverless, per-GPU-second on-demand, per-token/GPU-hour training, enterprise quote. No separate gateway or platform fee — routing layer is $0. Not verified; A one-paragraph description that covers the caveats a pricing category cannot: introductory rates, per-feature meters, tier gating, and pricing that resets on a specific date.
Minimum commitment Minimum commitment: None stated for inference. Fine-tuning has no stated minimum on the pricing page. Not verified; Whether the vendor requires a minimum contract term, a minimum spend, or a provisioned-capacity purchase to get its published rate.
Charges that fire after you go over an allowance Overage terms: None stated. Serverless is postpaid, so cost scales with tokens. Note: 'Checkpoint storage for serverless models is included during private preview' implies a future charge. Not verified; The line items that scale with usage after an included allowance is exhausted — log storage, extra requests, per-feature meters, data export — which is where cost estimates usually go wrong.
Prompt cache offered Cache mechanism: Prefix cache Not verified; Whether the gateway offers its own response cache, what kind of match it does (exact request, prefix, semantic), or simply passes provider caching through unchanged.
Discount on cached input Cache-read discount: 50% Not verified; How much cheaper cached tokens are than fresh input, when the vendor publishes a single figure. Bundled-inference clouds usually price this per model instead of as one number.
Premium on cache writes Cache-write premium: None Not verified; How much more the first write of a cached prefix costs versus a plain input token. A high write premium and a low hit rate can leave you paying more than you save, so this matters as much as the read discount.
Who captures the cache saving Cache economics: Prompt caching with discounted cached tokens (prefix-style caching for serverless; mechanism not labelled exact-match or semantic). Default 50% discount, but exact discount varies by model. Pricing page separately lists 'Cached Prefill / 1M' rates for training that are 80% below prefill (e.g. Qwen 3.5 9B $0.132 vs $0.66; Kimi K3 $2.17 vs $10.87). No cache-write charge documented — close to a free win, but only serverless models are covered. Not verified; Whether the customer keeps the full saving from caching or the vendor captures part of it — and any conditions attached (write premium, storage fees, best-effort hits).
What you can split spend by Cost attribution: Not stated. Not verified; The dimensions the vendor documents for splitting spend — per key, per user, per team, per tag, per customer. Matters if you need to chargeback internally or bill an end customer.
How you get cost data out Cost export: Not stated. Not verified; The mechanisms the vendor publishes for exporting cost and usage data: CSV, an API, webhooks, S3, a data warehouse, or nothing at all. Any per-unit price is included.
Who pays the model bill BYOK mode: Their credits only Not verified; Whether you bring your own provider accounts (BYOK), buy inference from this vendor, or can do either. This is the single biggest commercial difference between these products: it decides who holds the contract with the model provider and who carries the spend.

Spend governance in detail

Seven signals matter when a bill starts to hurt: who can spend, how much, on what, and who gets paged when it goes wrong. Everything below is drawn from the vendor’s own pricing and docs pages — how we read these.

Virtual or scoped keysNot published
Keys that carry their own budget and rate-limit policy, so an intern experiment cannot spend against a production budget.
Not stated.
Budget caps per keyNot published
A dollar or token ceiling attached to an individual key. Where enforcement is soft, one over-limit request still completes before the block kicks in.
Not stated.
Budget caps per team or workspaceNot published
A ceiling applied at a higher scope than one key — a team, a workspace, a customer, or an entire environment.
Not stated.
Rate limiting as a cost controlYes
Configurable request-per-time-window caps. Platform-set rate limits do not count as spend controls; user-configurable ones do.
'High rate limits' on serverless; higher on Enterprise. Platform-set, not a configurable spend control.
Model allowlistsNot published
A policy that constrains which models a key or team can call, keeping expensive frontier models out of the wrong hands.
Not stated.
Spend alertsNot published
Alerts fired as spend approaches a threshold. Alerts that only fire after the meter has rolled over are marked as such.
Not stated.
Webhook notificationsNot published
Programmatic notifications on spend events, so budget breaches can page an on-call or open a ticket.
Not stated.

Enforcement:Enforcement point not stated

Catalog

Models available Models available: ~100 Verified 3 days ago; How many models you can call, shown as a range because several vendors publish different totals on different pages. Vendor-reported either way, so counts are not directly comparable — some count every provider variant of the same model separately.
Model providers reachable Upstream providers: Not published Not verified; How many distinct model providers or labs you can reach, shown as a range where the vendor’s own pages disagree. More providers usually means better redundancy when one has an outage.
Works with standard OpenAI code OpenAI-compatible API: Yes Verified 3 days ago; If yes, you can usually switch to it by changing one base URL, and switch away just as easily. This is the main defence against lock-in.
OpenAI chat endpoint POST /v1/chat/completions: Yes Not verified; The endpoint almost every application ports first. "Not documented" means the vendor never states it, which is different from a documented no.
Anthropic messages endpoint POST /v1/messages: Not documented Not verified; Whether Anthropic-shaped calls work without rewriting them. Several products support this only as SDK compatibility or provider passthrough rather than a native endpoint — the detail page says which.
OpenAI Responses endpoint POST /v1/responses: Not documented Not verified; The newer stateful OpenAI surface. Support is much thinner across this market than chat completions.
Embeddings endpoint POST /v1/embeddings: Yes Not verified; Whether you can generate vectors through the same gateway, or need a second integration for retrieval workloads.
Image generation endpoint POST /v1/images/generations: Yes Not verified; Whether image models are reachable through the same surface as text.
Audio endpoints POST /v1/audio/*: Yes Not verified; Speech-to-text and text-to-speech. Frequently the first gap in an otherwise complete gateway.
Batch jobs endpoint POST /v1/batches: Yes Not verified; Asynchronous bulk processing, usually at a discount. Commonly undocumented, and commonly the reason a migration stalls late.
Needs the vendor’s own code library Requires a vendor-specific SDK: No Verified 3 days ago; A proprietary client library spreads through your codebase and has to be torn out again if you leave. “No” is the better answer here, and it means the standard OpenAI client works.
You can export your request history Logs / usage data export: Yes Verified 3 days ago; Whether you can get your own request logs, traces, or usage records back out — through an API, a bulk export, or a download. Decides whether you leave with your history or abandon it.
Settings can live in version control Declarative config-as-code: Not published Not verified; Whether routing, fallback, and budget rules can be declared in a file you keep in Git, rather than existing only as settings clicked into a hosted dashboard.
Embeddings Embeddings: Yes Verified 3 days ago; Text-to-vector models, needed for search and retrieval features.
Image generation Image generation: Yes Verified 3 days ago; Whether image models are reachable through the same interface.
Speech and audio Speech and audio: Yes Verified 3 days ago; Text-to-speech or transcription models through the same interface.
Video generation Video generation: Not published Not verified; Whether video models are reachable through the same interface.
Batch processing Batch processing: Yes Verified 3 days ago; Submitting large jobs for cheaper, slower processing. Often 50% off for work that is not time-sensitive.

Routing & reliability

Uptime it promises in writing Contractual SLA uptime: Not published Not verified; The uptime percentage in a published, contractual service level agreement. A public status page is not an SLA — it reports what happened, it does not promise anything or pay you back when it breaks.
Automatic failover Automatic failover: Not published Not verified; When a model provider goes down or rate-limits you, traffic moves to a backup automatically instead of returning errors to your users.
Load balancing Load balancing: Not published Not verified; Spreads requests across several providers or keys to raise your effective rate limit.
Rule-based routing Conditional routing: Not published Not verified; Send different requests to different models based on rules — for example a cheap model for free users and a strong model for paying ones.
Response caching Response caching: Not published Not verified; Reuses the answer when the exact same request comes in again, which cuts both cost and latency.
Similar-question caching Semantic cache: Not published Not verified; Reuses an answer when a new question means roughly the same thing as an earlier one. Saves far more than exact-match caching but can return subtly wrong answers if tuned loosely.
Where you set the timeout Request timeout surface: Not documented Not verified
`not_documented` for inference requests. The only documented timeout is control-plane: `--wait-timeout` on `firectl deployment create` with `--wait`, **default 1 hour** ([Fireworks on-demand deployments](https://docs.fireworks.ai/guides/ondemand-deployments)).; Where a request timeout can be set: per request, in a config file, in the vendor dashboard, or nowhere. Nine of the twenty products documented here do not describe a request timeout at all, so the worst case of a hung upstream call is unknowable from the docs.
Where you set retries Retry policy surface: Not documented Not verified
`not_documented` as a platform feature; it is pushed to the client: when a deployment is scaled to zero, requests return `503` immediately and "Your application should implement retry logic to handle this". Default count and backoff: `n.a.` ([Fireworks on-demand deployments](https://docs.fireworks.ai/guides/ondemand-deployments)).; Where retry count and backoff are configured. Worth knowing alongside billing: a retried streaming call can be charged more than once.
Where you set fallbacks Fallback surface: Not documented Not verified
No model or provider fallback list ([Fireworks on-demand deployments](https://docs.fireworks.ai/guides/ondemand-deployments)).; Where the fallback chain is defined. Almost every product claims fallback; the useful question is whether you can change it from code or only by hand in a dashboard.
Shape of the fallback chain Fallback shape: Shape not documented Not verified; An ordered list tries targets in sequence; a weighted split sends a percentage of traffic to each, which is what you need to trial a new model on 5% of requests. Weighted splits are much rarer than the marketing implies.
Upstream health tracking Health checks / circuit breaking: Not documented Not verified
`not_documented` as configurable. Deployment health is observable via `state` (`CREATING`, `READY`, `UPDATING`, `DELETING`, `DELETED`, `FAILED`) and replica counts (`min_replica_count`, `ready_replica_count`, `desired_replica_count`, `max_replica_count`), with guidance to "Verify it's healthy" before cutting traffic over — this is deployment status, not upstream ejection or circuit breaking ([Fireworks on-demand deployments](https://docs.fireworks.ai/guides/ondemand-deployments)).; Whether the product notices a failing upstream and stops sending traffic to it, and whether you can tune the thresholds. This is what turns a provider outage into a blip rather than a sustained error rate, and only four of the twenty expose it.
Cross-region failover you control Multi-region failover surface: In config Not verified
`config_file` at creation time, and it is genuinely a capacity-failover control: `--region` accepts `GLOBAL`, `US`, `EUROPE`, `APAC`; `GLOBAL` is recommended for production workloads needing "geographic availability or capacity failover". Without `--region` a deployment is "pinned to a single datacenter at creation time and will not be automatically migrated later", and placement "cannot be changed in place" — you create a replacement deployment ([Fireworks on-demand deployments](https://docs.fireworks.ai/guides/ondemand-deployments)).; Whether you can define what happens when a region degrades. A vendor running many regions is not the same as a vendor letting you configure failover between them; only three document a user-controlled mechanism.
Where you set load balancing Load balancing surface: Not documented Not verified
Deployment shapes can be optimized for speed, throughput or cost, but no load-balancing policy or weights ([Fireworks on-demand deployments](https://docs.fireworks.ai/guides/ondemand-deployments)).; Where traffic distribution across upstreams or keys is configured.

Operations

Usage dashboards and logs Observability: Not published Not verified; Built-in visibility into what was sent, what came back, what it cost, and how long it took.
Spending limits Budget controls: Not published Not verified; Hard caps that stop spend before it becomes a surprise invoice. The single most valuable control for a small team.
Rate limits Rate limits: Not published Not verified; Caps on request volume per key or per user, useful for protecting against abuse and runaway loops.
Separate keys per team or app Virtual keys: Not published Not verified; Issue scoped keys with their own budgets and permissions so you can attribute cost and revoke access without rotating everything.
Prompt versioning Prompt management: Not published Not verified; Store and version prompts outside your code so they can be changed without a deploy.
Quality testing Evals: Not published Not verified; Built-in tooling to score model output against test cases, so you can tell whether a model swap made things better or worse.
MCP support MCP support: Not published Not verified; Native support for the Model Context Protocol, the emerging standard for connecting models to external tools.
What gets logged Logged content: Metadata only Not verified
Zero data retention applies by default and prompts/generations are not logged; the exception is the Responses API, where `store=True` is the default and stored responses are deleted after 30 days ([Data handling](https://docs.fireworks.ai/guides/security_compliance/data_handling)); Whether full prompts and responses are stored, only metadata, or nothing. Full-body logging is the most useful debugging feature here and the one most likely to need a conversation with your compliance team.
You can turn logging off Body-logging opt-out: Yes Not verified
`store=False` on Responses API calls prevents storage; ZDR is the default elsewhere ([Data handling](https://docs.fireworks.ai/guides/security_compliance/data_handling)); Whether prompt and response bodies can be suppressed while still keeping usage metrics. Nineteen of the twenty document a way to do this; the mechanisms range from a per-request header to an organisation-wide setting.
Traces you can take elsewhere Distributed tracing: Vendor format only Not verified
Via integrations rather than natively: Weights & Biases Weave and MLflow tracing are documented ([MLOps and observability integrations](https://docs.fireworks.ai/ecosystem/integrations/mlops-observability)); Whether the product emits OpenTelemetry, a proprietary format, or nothing. OpenTelemetry means the traces land in the tooling you already run instead of only in the vendor’s dashboard.
Where telemetry can go Export destinations: Not published Not verified
A Prometheus-compatible metrics endpoint at `https://api.fireworks.ai/v1/accounts/<account_id>/metrics`, scrapeable by Prometheus, the OpenTelemetry Collector, the Datadog Agent, Vector, Grafana, and New Relic ([Exporting metrics](https://docs.fireworks.ai/deployments/exporting-metrics)); W&B and MLflow for traces ([MLOps and observability integrations](https://docs.fireworks.ai/ecosystem/integrations/mlops-observability)); Documented sinks for logs and metrics. This is a good proxy for how replaceable the vendor’s own dashboard is: around twenty destinations means you never have to depend on it, while a CSV download means you do.
Can record user feedback Feedback capture API: No Not verified
`n.a.` ([MLOps and observability integrations](https://docs.fireworks.ai/ecosystem/integrations/mlops-observability)); Whether there is an API to attach a rating or score to a logged request, which is what lets production traffic feed quality work later.
Scores live traffic Online eval hooks: No Not verified
`n.a.` — evaluation happens in W&B/MLflow, not against Fireworks-stored traffic (there is none by default) ([MLOps and observability integrations](https://docs.fireworks.ai/ecosystem/integrations/mlops-observability), [Data handling](https://docs.fireworks.ai/guides/security_compliance/data_handling)); Whether automated scorers can run against real production requests, rather than only against a test set you assemble yourself.

Performance

Delay it adds Proxy overhead: Not published Not verified; Extra time the product itself adds to each request, on top of however long the model takes. Usually irrelevant next to multi-second model latency, but it matters for high-volume or streaming-sensitive workloads.
Requests per second ceiling Throughput: Not published Not verified; Published sustained request rate before the product becomes the bottleneck. Only relevant at genuinely high volume.
What the request path runs on Architecture class: Undisclosed vendor service Not verified
`vendor_saas` — an inference provider, not a gateway; the serving runtime is described by engine name (FireAttention V4) rather than a disclosed process runtime, and the control surface is the `firectl` CLI plus `https://api.fireworks.ai/inference/v1` ([Fireworks FireAttention V4](https://fireworks.ai/blog/fireattention-v4-fp4-b200), [Fireworks on-demand deployments](https://docs.fireworks.ai/guides/ondemand-deployments)).; The comparable way to talk about latency here. An edge worker, a compiled Go or Rust binary, and a Python proxy have different overhead floors no matter which figures each vendor publishes. Products that never disclose their runtime are recorded as undisclosed rather than assumed.
You can run the request path yourself Self-hostable data plane: No Not verified
Fireworks distributes `firectl` binaries (Homebrew, macOS arm64/x86_64, Linux x86_64, Windows 64-bit) and an open-source benchmarking tool, but not a self-hostable serving plane ([firectl CLI](https://docs.fireworks.ai/tools-sdks/firectl/firectl), [Fireworks performance benchmarking](https://docs.fireworks.ai/deployments/benchmarking)).; Whether the component that actually carries your prompts can run on your own infrastructure. Distinct from a vendor offering a self-hosted control plane while still proxying traffic through their network.
Streaming responses Streaming support: Yes Not verified
Streaming is available through the OpenAI-compatible API surface; the fetched pages state no streaming-specific caveat, but they do state hard availability caveats: "A preemptible deployment can be preempted mid-request and disappear with no warning", "Never point production or latency-sensitive traffic at a preemptible deployment", and scaled-to-zero deployments return `503` immediately ([Fireworks on-demand deployments](https://docs.fireworks.ai/guides/ondemand-deployments)).; Whether token-by-token streaming is documented. The caveats matter more than the yes: some products cannot cancel a stream without still being billed, and several timeout and fallback mechanisms stop applying once the first token has been sent.

Security & compliance

Does your prompt reach their servers Prompt transits vendor: Yes Not verified
Inference runs on Fireworks' own global hardware fleet.; Whether the text you send passes through this company’s own infrastructure. If it does, every other promise on this page is a policy commitment rather than a physical impossibility. Self-hosted products can answer no outright.
What they keep if you change nothing Logging default: Nothing stored by default Not verified
No prompt or generation data is logged or stored for any open model without explicit opt-in — it exists only in volatile memory for the life of the request. Metadata such as token counts is logged as needed to run the service.; Defaults matter more than options. A product that stores full prompts and replies unless you find the right header will have stored them by the time you read the docs.
How long they keep it Default content retention (days): Nothing kept by default Not verified
Zero by default. The Responses API is the exception: storage defaults to on, held 30 days, and you must pass store=False to opt out.; Default retention for request content, in days. Zero means nothing is kept. Read the note: several products keep nothing as a rule but make timed exceptions for abuse review or specific models.
Could they train on your prompts Training on customer data: Only if you opt in Not verified
Nothing is stored without explicit opt-in, and opting in enables features such as FireOptimizer. Worth noting what is absent: no blanket “we never train on your data” statement was found.; Whether the vendor may use your prompts and outputs to train models. “Not published” means we could not find any position, which is not the same as a no — ask for it in writing.
Where it runs, and what you can pin Region and residency control: Region groupings for global (the default), US, Europe and Asia-Pacific, plus around 19 pinnable single regions including Iowa, Frankfurt, Iceland and Tokyo. Not verified; Which regions are offered and whether you can force processing to stay in one. A global endpoint that silently picks a region is a different compliance story from an endpoint you pin yourself.
Where safety filters run Guardrail execution location: No guardrails offered Not verified
No guardrails are applied to models by default; Fireworks expects customers to implement their own.; A filter that strips personal data only helps if it runs before the data leaves your boundary. If guardrails execute in the vendor’s cloud, the vendor has already received whatever you wanted redacted.
Who else touches the data Subprocessor list: Not published Not verified; The published list of third parties the vendor passes your data to. No list means you cannot know the full chain, which most data-protection agreements require you to.
SOC 2 audited SOC 2 audited: Yes Verified 3 days ago; An independent audit of security controls. Enterprise buyers and their procurement teams routinely require it.
Will sign a HIPAA agreement HIPAA BAA: Yes Verified 3 days ago; Required before you may send protected health information through the service. Without a signed BAA, healthcare data is off limits.
GDPR commitments GDPR commitments: Not published Not verified; Published data processing terms for handling personal data of people in the EU and UK.
Can keep data in the EU EU data residency: Not published Not verified; Requests can be processed inside the EU rather than routed to US infrastructure. Often the deciding constraint for European customers.
Does not retain your data Zero data retention: Not published Not verified; Prompts and responses are not stored after the request completes. Sometimes a paid add-on rather than the default.
Strips personal data PII redaction: Not published Not verified; Detects and removes identifiers such as names, emails, and card numbers before the request reaches the model provider.
Content guardrails Content guardrails: Not published Not verified; Policy checks on inputs and outputs — blocking unsafe content, enforcing formats, or catching prompt-injection attempts.
Runs fully disconnected Air-gapped deployment: Not published Not verified; Can be deployed in a network with no internet access, which some regulated and defence environments require.
Blocks personal data in prompts PII / DLP enforcement: Not documented Not verified
No PII guardrail feature; the data-handling posture is zero retention rather than detection ([Data handling](https://docs.fireworks.ai/guides/security_compliance/data_handling)); Whether personal data detection sits on the request path and can stop the call, merely inspects and forwards it, or is not documented. A control that only reports is a logging feature, not a policy control.
Blocks prompt injection Injection / jailbreak enforcement: Not documented Not verified
`not_documented` as a platform feature ([Data handling](https://docs.fireworks.ai/guides/security_compliance/data_handling)); Whether injection and jailbreak detection can stop a request. Most products offering this call a partner classifier rather than shipping their own.
Blocks harmful content Toxicity / moderation enforcement: Not documented Not verified
`not_documented` — no inline moderation filter is documented on the fetched pages; the data-handling and account-policy docs describe retention and model access only ([Data handling](https://docs.fireworks.ai/guides/security_compliance/data_handling), [Model access policy](https://docs.fireworks.ai/accounts/model-access-policy)); Whether hate, violence, sexual and self-harm categories are checked inline and can stop a request, in either direction.
Your own policy rules Custom policy hooks: Not documented Not verified
`not_documented` ([Data handling](https://docs.fireworks.ai/guides/security_compliance/data_handling)); Whether you can add your own rule — a regex, a webhook, or your own classifier — rather than choosing from the vendor library.
Where guardrails run Guardrail execution location: Not documented Not verified; Whether guardrail evaluation happens inside your infrastructure or on the vendor’s servers. This decides whether the prompt you are trying to protect leaves your network in order to be checked.
If the guardrail itself fails Guardrail failure mode: Not documented Not verified
`not_documented` ([Data handling](https://docs.fireworks.ai/guides/security_compliance/data_handling)); What happens when the guardrail service times out or errors: does the request proceed unchecked, or is it blocked? This is the worst-documented field in the entire catalogue — only two vendors state it plainly, which means most teams are running a control whose failure behaviour they cannot know.
Third-party guardrail vendors Guardrail integrations: Not published Not verified; Named external guardrail services the product can call. A long list means the product is a router for policy engines rather than a policy engine itself — which also means another vendor bill and another hop.

Compliance evidence

Graded by how strong the evidence is, not whether the word appears on the vendor’s website. An audited report and a marketing claim are different things, and only one of them will satisfy your own auditor.

SOC 2 Certified Type II
ISO 27001 Certified achieved, alongside 27701 and 42001
GDPR DPA Claimed, no evidence published controls mapped to GDPR and CCPA; no executable DPA page retrieved
HIPAA BAA Claimed, no evidence published described as HIPAA-certified and compliant; no BAA document retrieved
FedRAMP Not published
ITAR Not published

Vendor source

Fit & integration

Work to try it Evaluation work shape: Install a package Not verified; The shape of the work on the vendor’s own quickstart, from swapping one base URL through to deploying infrastructure. An ordinal class rather than a duration, because elapsed time depends on accounts and quota we cannot see.
Work to run it Production work shape: Install a package Not verified; The same scale applied to the vendor’s recommended production path. For several products this is much heavier than the quickstart, which is exactly why both are recorded.
Steps on the quickstart Numbered quickstart steps: 2 Not verified; A literal count of numbered steps on the vendor’s quickstart, recorded as evidence beside the work shape. Zero means the page publishes no numbered procedure at all. Large counts usually mean interleaved language tracks rather than more work.
Can you self-host it today Self-host install documentation: No self-hosting Not verified; Whether an install command is actually published. Several products advertise self-hosting while publishing no command to start from, which a plain yes/no would hide.
Works with the OpenAI SDK OpenAI SDK drop-in: Yes Not verified
Yes. For hosted inference, Fireworks documents the OpenAI SDK with `base_url="https://api.fireworks.ai/inference/v1"`; BYOC examples use `base_url="http://<YOUR-LOAD-BALANCER-HOSTNAME>/v1"` and `api_key="unused"`. ([Fireworks OpenAI compatibility docs](https://docs.fireworks.ai/tools-sdks/openai-compatibility)); Whether an existing OpenAI-compatible client can be pointed at it by changing the base URL and key.
Vercel AI SDK support AI SDK provider package: Official provider package Not verified
Fireworks has an official Vercel AI SDK provider package, `@ai-sdk/fireworks`, with provider instances `fireworks` and `createFireworks`. ([Vercel AI SDK Fireworks provider](https://ai-sdk.dev/providers/ai-sdk-providers/fireworks)); Whether a first-party AI SDK provider package exists, or only a community package, a documented workaround, or the generic OpenAI provider pointed at a custom base URL.
Python framework integrations Documented Python frameworks: LangChain, LlamaIndex, CrewAI, PydanticAI, Strands Not verified
Fireworks documents integrations with LangChain and LlamaIndex, and also lists CrewAI, PydanticAI, Strands Agents, and AWS AgentCore as supported agent frameworks. The LangChain JavaScript integration uses package `@langchain/fireworks` and class `Fireworks`; package and class names for the other documented integrations are not stated. ([Fireworks agent-framework integrations](https://docs.fireworks.ai/ecosystem/integrations/agent-frameworks)); LangChain, LangGraph, LlamaIndex and similar orchestration frameworks with a documented integration.
Callable from Cloudflare Workers Cloudflare Workers support: Not documented Not verified
n.a. (not documented); Whether the docs show calling this product from your own Worker. Deliberately separated from the several products whose own gateway runs on Workers, which is a fact about their infrastructure and not about your edge compatibility.
Kubernetes install Helm chart availability: Helm supported, chart not named Not verified
Fireworks documents Kubernetes and Amazon EKS deployments, including deployment with Helm charts or Helm deployment templates. No official chart name, repository, or operator name is stated. ([Fireworks airgapped EKS deployment](https://docs.fireworks.ai/ecosystem/integrations/eks/airgapped)); Whether a named, published Helm chart exists, versus Helm being referenced with no chart named, versus generic cluster documentation that has nothing to do with this product.
Terraform support Terraform provider or modules: Official modules Not verified
Fireworks documents the official Terraform module `fw-ai-external/aws-cluster/fireworksai` at `registry.terraform.io/modules/fw-ai-external/aws-cluster/fireworksai/latest`, with example version `0.1.2`. ([Fireworks EKS BYOC documentation](https://docs.fireworks.ai/ecosystem/integrations/eks-bring-your-own-cloud)); Whether you can declare this in code: an official provider, official modules, resources inside a hyperscaler’s provider, a community provider, or only Terraform code shipped in a repo.
Reuses your cloud identity Cloud IAM reuse: Reuses your cloud identity Not verified
For BYOC on AWS, Fireworks documents AWS IAM roles for the cluster and support for `AWS_PROFILE` when `firectl` uploads model files to S3. BYOC API examples use `api_key="unused"` because the API key is not used, while airgapped deployment stores a metering key in AWS Secrets Manager. ([Fireworks EKS BYOC documentation](https://docs.fireworks.ai/ecosystem/integrations/eks-bring-your-own-cloud)); Whether you can authenticate with IAM roles, workload identity or managed identities instead of another long-lived API key. Distinguished from products that only accept static upstream provider credentials.
Fits behind your API gateway API gateway integration: The gateway vendor documents it Not verified
Not documented by Fireworks, but API7's APISIX-based AI gateway documents a `fireworks-ai` catalog provider that connects through the `openai` adapter with `api_base` = `https://api.fireworks.ai/inference/v1`. ([API7 AI Gateway - Fireworks AI provider](https://docs.api7.ai/ai-gateway/providers/fireworks-ai)); Whether AI traffic can go through a gateway you already run, and who documented that — the product’s vendor or the gateway’s.
MCP support MCP surface shape: MCP tools in the API Not verified
Fireworks supports MCP tools in its Responses API; the `tools` field accepts MCP tools and response tools support the `mcp` type. Fireworks also documents MCP server usage with an example SSE server URL such as `https://gitmcp.io/docs`. ([Fireworks Responses API reference](https://docs.fireworks.ai/api-reference/post-responses)); Which kind of MCP support this is: a gateway that governs many MCP servers, a hosted MCP server you connect to, MCP tools accepted inside the completion API, or client tooling. These are different products behind one acronym.
Needs your own provider key Upstream provider key required: Not needed Not verified
No. A Fireworks API key is used ([Fireworks quickstart](https://docs.fireworks.ai/getting-started/quickstart)); Whether an upstream provider account and key must exist before your first call works. A prerequisite rather than a step, and it can differ between the hosted and self-hosted forms of the same product.
Gate before models work Model access gate: No gate Not verified
n.a. (not documented). No approval, enablement, quota or region gate on either page ([Fireworks quickstart](https://docs.fireworks.ai/getting-started/quickstart), [Fireworks - Pricing](https://fireworks.ai/pricing)); Whether an enablement click, a quota grant, a paid tier or an approval form stands between a valid key and a working model call.
Official client languages First-party client SDK languages: Python, TypeScript Not verified
Documented clients include the Fireworks Python SDK, package `fireworks-ai` with class `Fireworks`; the OpenAI Python package `openai` with class `OpenAI`; the Anthropic Python package `anthropic` with class `anthropic.Anthropic`; the OpenAI JavaScript/TypeScript package `openai` with class `OpenAI`; and the Anthropic JavaScript/TypeScript package `@anthropic-ai/sdk` with class `Anthropic`. Fireworks also documents the `@ai-sdk/fireworks` JavaScript/TypeScript provider and `curl`. ([Fireworks quickstart](https://docs.fireworks.ai/getting-started/quickstart)); Languages with a first-party client library. An empty list can still mean the product is usable from any language via an OpenAI-compatible SDK.

Additional charges

These are the fees that do not appear on a per-token price list, and they are where estimates usually go wrong.

Region-restricted deployments 1.5x premium over standard deployment pricing
On-demand GPUs H100/H200 $8.00/hour, B200 $13.00, B300 $15.00, GB300 $20.00
Managed LoRA supervised fine-tuning $0.50-$10.00 per 1M tokens depending on model size
Embeddings $0.008-$0.10 per 1M input tokens

How pricing actually works

No self-host option; Fireworks sets its own per-token serverless prices rather than marking up another vendor's list price, with DEFAULT, PRIORITY and FAST service tiers resolved live per model. Dedicated capacity is billed per GPU-hour, and batch inference plus supervised and reinforcement fine-tuning are priced separately.

Common questions

Answered from the fields above, so these move when the catalog moves. Every figure quoted here appears in the specification with its source.

Does Fireworks AI charge a markup on model prices?

Fireworks AI does not resell other providers' models with a fee on top: inference is the product, so you pay its own published per-token rates. There is no separate routing charge to add to them.

Can Fireworks AI be self-hosted?

No. Fireworks AI is available only as a service the vendor operates; there is no self-hosted build. The licence is Proprietary.

Is Fireworks AI SOC 2 audited, and will it sign a HIPAA BAA?

A SOC 2 report is available for Fireworks AI and it will sign a HIPAA business associate agreement. Each of these is linked to the vendor's own page in the compliance section below.

Does Fireworks AI retain your prompts?

Fireworks AI publishes a zero-data-retention position. Only metadata is logged — not prompt or response bodies. Logging can be turned off. Stated retention is zero days. It trains on customer data only if you opt in.

Can you use your own provider keys with Fireworks AI?

Fireworks AI does not publish whether you can supply your own upstream provider keys.

How many models does Fireworks AI support?

Fireworks AI states ~100 models. Docs state "100+ Supported Models" across text, vision, audio, image and embeddings; the list-models API returns a totalSize with a maximum pageSize of 200. The figure on this page is dated and carries its source.

Official links

Independent coverage

Third-party analysis, walkthroughs and operator threads. We link criticism as readily as praise. Nothing here is written or published by the vendor, by a competitor listed on this site, or by an SEO content farm — how we vet these.

Written reviews and analysis 2

Fireworks.ai: The PyTorch Team's Bet on Inference as the New Runtime WorkOS · Jan 2026 Zack Proser's deep dive into Fireworks' FireAttention stack, serverless vs on-demand deployment model and where its inference economics come from.
Running LLaMA 4 Maverick Full 1M-Token Context on Fireworks AI with Aish (Substack) · Apr 2025 Hands-on account of pushing Llama 4 Maverick's full 1M-token context through the Fireworks API, with the scripts and practical limits described.

Video 2

Lin Qiao, Fireworks AI | RAISE Summit 2026 YouTube - SiliconANGLE theCUBE · Jul 2026 theCUBE's hosts press Fireworks' CEO on inference economics and platform strategy in a third-party-produced conference interview.
Infrastructure for AI at Scale - With Benny Chen (Fireworks AI) YouTube - The Information Bottleneck · Jun 2026 Research-podcast hosts interview a Fireworks co-founder on GPU sourcing, kernels, routing and cross-region training that underpin the service.

Practitioner discussion 2

Fireworks.ai AST critical issues (stay away until they fix them) Reddit - r/speechtech · Dec 2025 Founder documents a 35-day Fireworks speech-to-text defect with dates, connection-count symptoms, cost impact and the workaround used, plus corroborating replies.
LLaMa 4 behaving differently on Groq vs Fireworks AI Reddit - r/LocalLLaMA · Apr 2025 Head-to-head thread on identical-model output differences between the two providers, with notes on Fireworks' fine-tune hosting and tier/rate-limit behaviour.

What has changed here

Aug 29, 2026 catalog entry catalog entry Not published Added to the catalog source ↗

See this in the full changelog Back to top ↑

Read the head-to-head

These pairs have a written verdict, not just a table.

Fireworks AI vs Together AI Read the verdict →

Usually weighed against

Fireworks AI vs Groq Compare → Fireworks AI vs Together AI Compare →